2024-03-18T23:37:14.571 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T23:37:14.571 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225 --name pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/orig.config.yaml 2024-03-18T23:37:14.624 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T23:37:14.713 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225 branch: wip-batrick-testing-20240318.181317 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7609225' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240318.181317 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240318.181317 sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3005 sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 sleep_before_teardown: 0 subset: 433/512 suite: fs suite_branch: wip-batrick-testing-20240318.181317 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKT4V/jSmR2AUCPmV1XiqgbRRuNTuEtBjyRpbTG6vOINJ39jsdZ4avpnWXR0v+gT/lYh/TJ3u4Vp633n6CyKk6Q= smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQG07FLKYQrbnejlw3LW7M+r1RZzqt2qa+Kk61wdFO+IOonC412GvvNGqL+yuVjgq5TRduVwBLZR4MmRAfy0zE= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5kGPY/UwRYLKvcCPSzHA3TWRcRdxDHmrkVH05qGP5PI3OlHGm6zpz6ZMPrKWOEme1EGs/S5+3wi3x3DFKGkSk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_21:05:49 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-18T23:37:14.713 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa; will attempt to use it 2024-03-18T23:37:14.714 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks 2024-03-18T23:37:14.714 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T23:37:14.716 INFO:teuthology.task.internal:Checking packages... 2024-03-18T23:37:14.737 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b6ae24918b03b1b8a19c9263a50655bba80bb810' 2024-03-18T23:37:14.738 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T23:37:14.738 INFO:teuthology.packaging:ref: None 2024-03-18T23:37:14.738 INFO:teuthology.packaging:tag: None 2024-03-18T23:37:14.738 INFO:teuthology.packaging:branch: wip-batrick-testing-20240318.181317 2024-03-18T23:37:14.738 INFO:teuthology.packaging:sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T23:37:14.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240318.181317 2024-03-18T23:37:14.985 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2206.gb6ae2491 2024-03-18T23:37:14.987 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T23:37:14.995 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T23:37:14.995 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T23:37:15.023 INFO:teuthology.task.internal:Saving configuration 2024-03-18T23:37:15.043 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T23:37:15.049 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:37:15.075 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 23:31:17.216453', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKT4V/jSmR2AUCPmV1XiqgbRRuNTuEtBjyRpbTG6vOINJ39jsdZ4avpnWXR0v+gT/lYh/TJ3u4Vp633n6CyKk6Q='} 2024-03-18T23:37:15.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 23:31:17.219350', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQG07FLKYQrbnejlw3LW7M+r1RZzqt2qa+Kk61wdFO+IOonC412GvvNGqL+yuVjgq5TRduVwBLZR4MmRAfy0zE='} 2024-03-18T23:37:15.120 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 23:31:17.217886', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5kGPY/UwRYLKvcCPSzHA3TWRcRdxDHmrkVH05qGP5PI3OlHGm6zpz6ZMPrKWOEme1EGs/S5+3wi3x3DFKGkSk='} 2024-03-18T23:37:15.120 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T23:37:15.127 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-18T23:37:15.127 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-18T23:37:15.127 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-18T23:37:15.127 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T23:37:15.259 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f10b3059160>, signals=[15]) 2024-03-18T23:37:15.259 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T23:37:15.266 INFO:teuthology.task.internal:Opening connections... 2024-03-18T23:37:15.266 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-03-18T23:37:15.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:37:15.371 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-03-18T23:37:15.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:37:15.472 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-03-18T23:37:15.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:37:15.562 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T23:37:15.569 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-03-18T23:37:15.587 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-03-18T23:37:15.587 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-03-18T23:37:15.644 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T23:37:15.645 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T23:37:15.646 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-03-18T23:37:15.667 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-03-18T23:37:15.684 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-03-18T23:37:15.684 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-03-18T23:37:15.741 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-03-18T23:37:15.741 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-03-18T23:37:15.741 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T23:37:15.742 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T23:37:15.743 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-03-18T23:37:15.767 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-03-18T23:37:15.785 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-03-18T23:37:15.785 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-03-18T23:37:15.842 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-03-18T23:37:15.842 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T23:37:15.843 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T23:37:15.844 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-03-18T23:37:15.858 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T23:37:15.869 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T23:37:15.925 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T23:37:15.925 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-03-18T23:37:15.929 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-03-18T23:37:15.932 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-03-18T23:37:15.949 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T23:37:15.998 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T23:37:15.999 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-03-18T23:37:16.001 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-03-18T23:37:16.004 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-03-18T23:37:16.032 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T23:37:16.110 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T23:37:16.124 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-18T23:37:16.124 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-18T23:37:16.125 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T23:37:16.125 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T23:37:16.126 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T23:37:16.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:37:16.143 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-03-18T23:37:16.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:37:16.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:37:16.145 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-18T23:37:16.145 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-03-18T23:37:16.200 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-361.el9.x86_64 2024-03-18T23:37:16.200 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.14.0-361.el9.x86_64 2024-03-18T23:37:16.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum install -y kernel 2024-03-18T23:37:16.203 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-361.el9.x86_64 2024-03-18T23:37:16.203 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-361.el9.x86_64 2024-03-18T23:37:16.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-03-18T23:37:16.204 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-361.el9.x86_64 2024-03-18T23:37:16.205 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-361.el9.x86_64 2024-03-18T23:37:16.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-03-18T23:37:18.424 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-18T23:37:18.586 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-9 - AppStream 9.1 MB/s | 6.7 MB 00:00 2024-03-18T23:37:18.740 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 7.8 MB/s | 6.7 MB 00:00 2024-03-18T23:37:19.816 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-18T23:37:20.021 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-03-18T23:37:20.265 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-18T23:37:20.640 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-18T23:37:22.889 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-03-18T23:37:24.794 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-03-18T23:37:31.665 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:09 2024-03-18T23:50:30.585 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 25 kB/s | 19 MB 13:06 2024-03-18T23:50:32.778 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 25 kB/s | 19 MB 13:06 2024-03-18T23:50:34.802 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.1 MB 00:00 2024-03-18T23:50:36.362 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-03-18T23:50:37.095 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.1 MB 00:00 2024-03-18T23:50:37.549 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-18T23:50:38.971 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-03-18T23:50:39.871 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 8.0 kB/s | 6.1 MB 13:04 2024-03-18T23:50:41.846 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-18T23:50:42.596 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-18T23:50:43.106 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-18T23:50:44.753 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T23:50:44.754 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T23:50:44.848 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-18T23:50:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 76 M 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 123 M 2024-03-18T23:50:44.856 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-18T23:50:45.298 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 206 kB/s | 44 kB 00:00 2024-03-18T23:50:46.813 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 21 MB 00:07 2024-03-18T23:50:47.923 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-03-18T23:50:48.373 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 4.5 MB/s | 15 MB 00:03 2024-03-18T23:50:50.046 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T23:50:50.046 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T23:50:50.132 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-03-18T23:50:50.142 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-03-18T23:50:50.142 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-03-18T23:50:50.142 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-03-18T23:50:50.142 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-03-18T23:50:50.142 INFO:teuthology.orchestra.run.smithi204.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-18T23:50:50.142 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout:Install 4 Packages 2024-03-18T23:50:50.143 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:50:50.144 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 76 M 2024-03-18T23:50:50.144 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 123 M 2024-03-18T23:50:50.144 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-03-18T23:50:50.616 INFO:teuthology.orchestra.run.smithi204.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 108 kB/s | 44 kB 00:00 2024-03-18T23:50:51.256 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 4.6 MB/s | 27 MB 00:05 2024-03-18T23:50:51.594 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 5.1 MB/s | 33 MB 00:06 2024-03-18T23:50:51.594 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-18T23:50:51.595 INFO:teuthology.orchestra.run.smithi007.stdout:Total 11 MB/s | 76 MB 00:06 2024-03-18T23:50:51.889 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-18T23:50:52.059 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:01 ago on Mon 18 Mar 2024 11:50:51 PM UTC. 2024-03-18T23:50:52.285 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-18T23:50:52.720 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-18T23:50:52.720 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-18T23:50:53.774 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-18T23:50:53.774 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-18T23:50:54.096 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T23:50:54.096 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T23:50:54.200 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-03-18T23:50:54.208 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-03-18T23:50:54.208 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-03-18T23:50:54.208 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-03-18T23:50:54.208 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-03-18T23:50:54.208 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-03-18T23:50:54.209 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:50:54.210 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 76 M 2024-03-18T23:50:54.210 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 123 M 2024-03-18T23:50:54.210 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-03-18T23:50:54.607 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 450 kB/s | 44 kB 00:00 2024-03-18T23:50:54.991 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 31 MB/s | 15 MB 00:00 2024-03-18T23:50:55.399 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 35 MB/s | 27 MB 00:00 2024-03-18T23:50:55.717 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 28 MB/s | 33 MB 00:01 2024-03-18T23:50:55.717 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-03-18T23:50:55.717 INFO:teuthology.orchestra.run.smithi044.stdout:Total 50 MB/s | 76 MB 00:01 2024-03-18T23:50:56.379 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-18T23:50:56.413 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-03-18T23:50:56.829 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-03-18T23:50:56.830 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-03-18T23:50:57.021 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-18T23:50:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:50:57.892 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-03-18T23:50:57.893 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-03-18T23:50:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:00.487 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-03-18T23:51:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-18T23:51:01.217 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:03.415 INFO:teuthology.orchestra.run.smithi204.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 1.1 MB/s | 15 MB 00:13 2024-03-18T23:51:04.103 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:05.162 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:09.338 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:09.381 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:12.147 INFO:teuthology.orchestra.run.smithi204.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 1.3 MB/s | 27 MB 00:21 2024-03-18T23:51:14.503 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:18.922 INFO:teuthology.orchestra.run.smithi204.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 1.2 MB/s | 33 MB 00:28 2024-03-18T23:51:18.922 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-03-18T23:51:18.922 INFO:teuthology.orchestra.run.smithi204.stdout:Total 2.6 MB/s | 76 MB 00:28 2024-03-18T23:51:19.588 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-03-18T23:51:19.966 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-03-18T23:51:19.966 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-03-18T23:51:21.025 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-03-18T23:51:21.025 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-03-18T23:51:23.514 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-03-18T23:51:24.134 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-18T23:51:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:26.925 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:27.071 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:31.775 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:31.896 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:36.603 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:41.205 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:41.205 INFO:teuthology.orchestra.run.smithi007.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T23:51:41.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:51:41.250 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:41.662 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:41.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-18T23:51:41.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:41.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:42.126 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:42.126 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:51:42.126 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-18T23:51:42.126 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-18T23:51:42.126 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-18T23:51:42.126 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-18T23:51:42.127 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-18T23:51:42.127 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:51:42.127 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-18T23:51:42.435 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T23:51:42.542 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T23:51:42.542 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T23:51:42.542 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-18T23:51:42.543 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T23:51:42.543 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T23:51:42.543 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-18T23:51:42.543 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T23:51:42.543 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T23:51:42.626 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T23:51:42.648 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T23:51:42.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-03-18T23:51:42.695 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T23:51:42.696 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-18T23:51:42.697 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T23:51:42.698 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T23:51:42.699 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-430.el9.x86_64 2024-03-18T23:51:42.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T23:51:43.204 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-03-18T23:51:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T23:51:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-03-18T23:51:44.682 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-03-18T23:51:44.724 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.tMdEr4AHMU 2024-03-18T23:51:44.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tMdEr4AHMU 2024-03-18T23:51:44.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.tMdEr4AHMU 2024-03-18T23:51:44.954 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.tMdEr4AHMU is 6KB 2024-03-18T23:51:45.002 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.tMdEr4AHMU 2024-03-18T23:51:45.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T23:51:45.084 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T23:51:45.084 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T23:51:45.085 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T23:51:45.085 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-18T23:51:45.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-18T23:51:45.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-03-18T23:51:45.309 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:45.309 INFO:teuthology.orchestra.run.smithi044.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T23:51:45.309 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:51:45.355 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-18T23:51:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:51:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:51:46.067 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:51:46.067 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:51:46.067 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-03-18T23:51:46.067 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-18T23:51:46.067 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-18T23:51:46.067 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-18T23:51:46.068 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-18T23:51:46.068 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:51:46.068 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-03-18T23:51:46.396 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T23:51:46.512 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T23:51:46.512 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T23:51:46.512 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-18T23:51:46.512 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T23:51:46.512 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T23:51:46.512 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-18T23:51:46.512 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi044.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T23:51:46.513 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T23:51:46.595 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T23:51:46.595 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T23:51:46.596 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -qi grub2-tools 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Name : grub2-tools 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Epoch : 1 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Version : 2.06 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Release : 68.el9 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Architecture: x86_64 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Group : Unspecified 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Size : 8264254 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:License : GPLv3+ 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T23:51:46.636 INFO:teuthology.orchestra.run.smithi044.stdout:Packager : builder@centos.org 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:Vendor : CentOS 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:Summary : Support tools for GRUB. 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:Description : 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:hardware devices. 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-18T23:51:46.637 INFO:teuthology.orchestra.run.smithi044.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T23:51:46.638 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T23:51:46.638 INFO:teuthology.task.kernel:Updating grub on smithi044 to boot 5.14.0-430.el9.x86_64 2024-03-18T23:51:46.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T23:51:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-03-18T23:51:48.620 INFO:teuthology.orchestra.run.smithi044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T23:51:48.661 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-03-18T23:51:48.664 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-03-18T23:51:48.699 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.GAnnx2vBnA 2024-03-18T23:51:48.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GAnnx2vBnA 2024-03-18T23:51:48.771 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.GAnnx2vBnA 2024-03-18T23:51:48.906 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.GAnnx2vBnA is 6KB 2024-03-18T23:51:48.955 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.GAnnx2vBnA 2024-03-18T23:51:48.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T23:51:49.045 INFO:teuthology.orchestra.run.smithi044.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T23:51:49.045 INFO:teuthology.orchestra.run.smithi044.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T23:51:49.045 INFO:teuthology.orchestra.run.smithi044.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T23:51:49.046 INFO:teuthology.orchestra.run.smithi044.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-18T23:51:49.047 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-18T23:51:49.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo shutdown -r now 2024-03-18T23:52:05.717 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:52:05.717 INFO:teuthology.orchestra.run.smithi204.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T23:52:05.717 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:52:05.761 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:52:06.097 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:52:06.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-18T23:52:06.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T23:52:06.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T23:52:06.553 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:52:06.554 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-03-18T23:52:06.860 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T23:52:06.953 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T23:52:06.954 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T23:52:06.954 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-18T23:52:06.954 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T23:52:06.954 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T23:52:06.954 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-03-18T23:52:06.954 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T23:52:06.955 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T23:52:07.034 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T23:52:07.034 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T23:52:07.034 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T23:52:07.068 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-18T23:52:07.069 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T23:52:07.070 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T23:52:07.070 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 5.14.0-430.el9.x86_64 2024-03-18T23:52:07.070 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T23:52:07.664 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-03-18T23:52:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T23:52:09.164 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-03-18T23:52:09.166 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-03-18T23:52:09.181 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.DwDmjAWo5t 2024-03-18T23:52:09.182 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DwDmjAWo5t 2024-03-18T23:52:09.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.DwDmjAWo5t 2024-03-18T23:52:09.384 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.DwDmjAWo5t is 6KB 2024-03-18T23:52:09.403 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.DwDmjAWo5t 2024-03-18T23:52:09.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T23:52:09.482 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T23:52:09.482 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T23:52:09.482 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T23:52:09.482 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-18T23:52:09.483 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-18T23:52:09.639 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-03-18T23:52:15.242 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-18T23:52:15.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-18T23:52:15.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:52:19.196 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-18T23:52:19.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-18T23:52:19.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:52:33.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-03-18T23:52:37.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-03-18T23:52:39.643 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-18T23:52:39.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-18T23:52:39.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:52:42.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-18T23:52:42.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:52:46.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-18T23:52:46.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:52:58.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-03-18T23:53:07.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-18T23:53:07.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:53:10.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-03-18T23:53:22.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-18T23:53:22.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:53:42.575 DEBUG:teuthology.orchestra.remote:timed out 2024-03-18T23:53:46.670 DEBUG:teuthology.orchestra.remote:timed out 2024-03-18T23:53:54.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-18T23:53:54.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:53:54.886 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-18T23:53:55.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-18T23:53:55.289 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-18T23:53:55.289 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-18T23:53:55.304 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-430.el9.x86_64 2024-03-18T23:53:55.304 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-18T23:53:55.304 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T23:53:55.304 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T23:53:56.305 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-18T23:53:56.305 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T23:53:56.384 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-03-18T23:53:56.425 DEBUG:teuthology.parallel:result is None 2024-03-18T23:53:58.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-18T23:53:58.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:53:58.996 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-03-18T23:53:59.367 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-18T23:53:59.367 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-18T23:53:59.367 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-03-18T23:53:59.382 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-430.el9.x86_64 2024-03-18T23:53:59.382 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-18T23:53:59.382 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T23:53:59.382 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T23:54:00.383 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-18T23:54:00.383 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T23:54:00.463 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-03-18T23:54:00.500 DEBUG:teuthology.parallel:result is None 2024-03-18T23:54:22.253 DEBUG:teuthology.orchestra.remote:timed out 2024-03-18T23:54:37.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-18T23:54:37.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T23:54:37.757 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-03-18T23:54:38.160 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-18T23:54:38.160 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-430.el9.x86_64"... 2024-03-18T23:54:38.161 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-03-18T23:54:38.177 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-430.el9.x86_64 2024-03-18T23:54:38.177 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-18T23:54:38.177 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T23:54:38.177 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T23:54:39.178 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-18T23:54:39.178 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T23:54:39.263 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-03-18T23:54:39.299 DEBUG:teuthology.parallel:result is None 2024-03-18T23:54:39.300 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T23:54:39.310 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T23:54:39.310 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T23:54:39.313 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T23:54:39.315 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T23:54:39.332 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T23:54:39.352 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T23:54:39.428 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T23:54:39.428 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T23:54:39.431 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T23:54:39.433 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T23:54:39.472 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T23:54:39.484 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T23:54:39.484 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T23:54:39.497 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T23:54:39.500 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T23:54:39.533 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T23:54:39.543 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T23:54:39.544 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T23:54:39.547 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T23:54:39.558 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T23:54:39.558 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T23:54:39.560 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T23:54:39.572 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T23:54:39.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T23:54:39.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T23:54:39.594 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T23:54:39.630 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T23:54:39.643 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T23:54:39.644 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T23:54:39.662 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T23:54:39.665 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T23:54:39.686 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T23:54:39.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T23:54:39.813 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T23:54:39.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T23:54:39.939 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-18T23:54:39.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T23:54:40.007 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T23:54:40.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T23:54:40.106 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T23:54:40.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T23:54:40.236 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-18T23:54:40.236 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T23:54:40.306 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T23:54:40.352 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T23:54:40.405 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T23:54:40.490 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T23:54:40.535 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-18T23:54:40.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T23:54:40.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-03-18T23:54:40.606 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-03-18T23:54:40.609 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-03-18T23:54:40.648 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:54:40.655 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:54:40.687 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:54:41.141 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T23:54:41.150 INFO:teuthology.task.internal:Starting timer... 2024-03-18T23:54:41.150 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T23:54:41.179 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T23:54:41.211 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-18T23:54:41.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-03-18T23:54:41.259 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-03-18T23:54:41.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-03-18T23:54:41.323 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-03-18T23:54:41.325 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-03-18T23:54:41.376 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-03-18T23:54:41.378 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T23:54:41.378 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-03-18T23:54:41.396 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-03-18T23:54:41.396 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-03-18T23:54:41.417 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-03-18T23:54:41.417 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-03-18T23:54:41.440 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-03-18T23:54:41.440 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-03-18T23:54:41.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:54:41.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:54:41.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:54:41.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:54:41.497 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:54:41.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:54:41.525 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T23:54:41.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-03-18T23:54:41.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-03-18T23:54:41.594 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-03-18T23:54:41.633 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T23:54:41.645 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T23:54:41.661 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T23:54:41.663 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi044.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-19T00:07:41.391 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-03-19T00:07:41.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-19T00:07:41.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:07:41.482 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-19T00:07:41.561 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-19T00:07:41.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-19T00:07:41.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:07:41.648 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-03-19T00:07:41.729 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-19T00:07:41.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-19T00:07:41.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:07:41.823 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-03-19T00:07:41.903 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-19T00:07:41.903 INFO:teuthology.run_tasks:Running task clock... 2024-03-19T00:07:41.922 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-19T00:07:41.923 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T00:07:41.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:07:41.925 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T00:07:41.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:07:41.928 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T00:07:41.928 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:07:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T00:07:41.958 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T00:07:41.975 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T00:07:41.977 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T00:07:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T00:07:41.992 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T00:07:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-03-19T00:07:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-03-19T00:07:42.020 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-03-19T00:07:42.020 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-03-19T00:07:42.021 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-03-19T00:07:42.034 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-03-19T00:07:42.037 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T00:07:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T00:07:42.050 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T00:07:42.053 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T00:07:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T00:07:42.066 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T00:07:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:07:42.113 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:07:42.113 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-19T00:07:42.113 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.113 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.113 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.113 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.114 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:07:42.117 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:07:42.117 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-03-19T00:07:42.117 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.117 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.117 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.117 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.122 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:07:42.125 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:07:42.125 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-03-19T00:07:42.125 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.125 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.125 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.126 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:07:42.126 INFO:teuthology.run_tasks:Running task install... 2024-03-19T00:07:42.135 DEBUG:teuthology.task.install:project ceph 2024-03-19T00:07:42.135 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'}} 2024-03-19T00:07:42.135 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'} 2024-03-19T00:07:42.135 INFO:teuthology.task.install:Using flavor: default 2024-03-19T00:07:42.141 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-19T00:07:42.141 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-19T00:07:42.142 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T00:07:42.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:07:42.144 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T00:07:42.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:07:42.146 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T00:07:42.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:07:42.374 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T00:07:42.374 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T00:07:42.378 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T00:07:42.378 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T00:07:42.380 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T00:07:42.381 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T00:07:42.491 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T00:07:42.492 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:07:42.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T00:07:42.498 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T00:07:42.498 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:07:42.499 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T00:07:42.519 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T00:07:42.520 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T00:07:42.527 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T00:07:42.527 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T00:07:42.549 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T00:07:42.549 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:07:42.549 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T00:07:42.577 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T00:07:42.577 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T00:07:42.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T00:07:42.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T00:07:42.645 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T00:07:42.666 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T00:07:42.672 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T00:07:42.726 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T00:07:42.733 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-03-19T00:07:42.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-03-19T00:07:42.738 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-03-19T00:07:42.739 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-03-19T00:07:42.792 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-03-19T00:07:42.793 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-03-19T00:07:42.983 INFO:teuthology.orchestra.run.smithi044.stdout:74 files removed 2024-03-19T00:07:42.986 INFO:teuthology.orchestra.run.smithi007.stdout:74 files removed 2024-03-19T00:07:43.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T00:07:43.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T00:07:43.052 INFO:teuthology.orchestra.run.smithi204.stdout:74 files removed 2024-03-19T00:07:43.079 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T00:07:43.843 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-03-19T00:07:43.868 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 150 kB/s | 83 kB 00:00 2024-03-19T00:07:43.872 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 150 kB/s | 83 kB 00:00 2024-03-19T00:07:44.231 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 33 kB/s | 10 kB 00:00 2024-03-19T00:07:44.244 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-19T00:07:44.268 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-19T00:07:44.594 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-03-19T00:07:44.599 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-03-19T00:07:44.628 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-03-19T00:07:45.068 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-19T00:07:45.131 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-19T00:07:45.388 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-9 - AppStream 9.3 MB/s | 6.7 MB 00:00 2024-03-19T00:07:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-03-19T00:07:46.897 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-03-19T00:07:46.922 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-19T00:07:47.704 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-19T00:07:47.955 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-19T00:07:48.800 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-03-19T00:07:51.656 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-03-19T00:07:55.732 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-19T00:07:57.366 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-19T00:07:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-19T00:08:04.560 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-19T00:08:06.924 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-19T00:08:07.048 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:08:07.048 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:08:07.061 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T00:08:07.062 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T00:08:07.063 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T00:08:07.064 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T00:08:07.066 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T00:08:07.068 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T00:08:07.069 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T00:08:07.156 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T00:08:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T00:08:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T00:08:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T00:08:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-03-19T00:08:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T00:08:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T00:08:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T00:08:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T00:08:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T00:08:07.178 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T00:08:07.179 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T00:08:07.180 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T00:08:07.181 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T00:08:07.182 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T00:08:07.183 INFO:teuthology.orchestra.run.smithi044.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T00:08:07.184 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T00:08:07.185 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T00:08:07.186 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T00:08:07.187 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T00:08:07.188 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T00:08:07.189 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T00:08:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T00:08:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T00:08:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T00:08:07.190 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T00:08:07.191 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T00:08:07.192 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T00:08:07.193 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T00:08:07.194 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T00:08:07.195 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout:Install 150 Packages 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 24 Packages 2024-03-19T00:08:07.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:08:07.197 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 224 M 2024-03-19T00:08:07.197 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-03-19T00:08:07.861 INFO:teuthology.orchestra.run.smithi044.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-19T00:08:08.278 INFO:teuthology.orchestra.run.smithi044.stdout:(2/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 8.2 MB/s | 5.4 MB 00:00 2024-03-19T00:08:08.344 INFO:teuthology.orchestra.run.smithi044.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 2.1 MB/s | 145 kB 00:00 2024-03-19T00:08:08.595 INFO:teuthology.orchestra.run.smithi044.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 9.4 MB/s | 2.4 MB 00:00 2024-03-19T00:08:08.770 INFO:teuthology.orchestra.run.smithi044.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-19T00:08:08.804 INFO:teuthology.orchestra.run.smithi044.stdout:(6/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 945 kB/s | 891 kB 00:00 2024-03-19T00:08:09.163 INFO:teuthology.orchestra.run.smithi044.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 12 MB/s | 4.7 MB 00:00 2024-03-19T00:08:09.363 INFO:teuthology.orchestra.run.smithi044.stdout:(8/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 12 MB/s | 21 MB 00:01 2024-03-19T00:08:09.405 INFO:teuthology.orchestra.run.smithi044.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 610 kB/s | 25 kB 00:00 2024-03-19T00:08:09.963 INFO:teuthology.orchestra.run.smithi044.stdout:(10/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 12 MB/s | 9.8 MB 00:00 2024-03-19T00:08:10.030 INFO:teuthology.orchestra.run.smithi044.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 3.3 MB/s | 221 kB 00:00 2024-03-19T00:08:10.089 INFO:teuthology.orchestra.run.smithi044.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 465 kB/s | 27 kB 00:00 2024-03-19T00:08:10.131 INFO:teuthology.orchestra.run.smithi044.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 765 kB/s | 32 kB 00:00 2024-03-19T00:08:10.214 INFO:teuthology.orchestra.run.smithi044.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 8.8 MB/s | 746 kB 00:00 2024-03-19T00:08:10.281 INFO:teuthology.orchestra.run.smithi044.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.4 MB/s | 163 kB 00:00 2024-03-19T00:08:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 2.1 MB/s | 127 kB 00:00 2024-03-19T00:08:10.415 INFO:teuthology.orchestra.run.smithi044.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 6.5 MB/s | 501 kB 00:00 2024-03-19T00:08:10.865 INFO:teuthology.orchestra.run.smithi044.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-19T00:08:10.924 INFO:teuthology.orchestra.run.smithi044.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 778 kB/s | 45 kB 00:00 2024-03-19T00:08:10.999 INFO:teuthology.orchestra.run.smithi044.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.7 MB/s | 131 kB 00:00 2024-03-19T00:08:11.066 INFO:teuthology.orchestra.run.smithi044.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 2.4 MB/s | 162 kB 00:00 2024-03-19T00:08:11.142 INFO:teuthology.orchestra.run.smithi044.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 4.2 MB/s | 321 kB 00:00 2024-03-19T00:08:11.217 INFO:teuthology.orchestra.run.smithi044.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.9 MB/s | 298 kB 00:00 2024-03-19T00:08:11.284 INFO:teuthology.orchestra.run.smithi044.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 99 kB 00:00 2024-03-19T00:08:11.351 INFO:teuthology.orchestra.run.smithi044.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-19T00:08:11.626 INFO:teuthology.orchestra.run.smithi044.stdout:(26/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 11 MB/s | 3.1 MB 00:00 2024-03-19T00:08:11.674 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.1 MB 00:01 2024-03-19T00:08:11.693 INFO:teuthology.orchestra.run.smithi044.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-19T00:08:11.760 INFO:teuthology.orchestra.run.smithi044.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 510 kB/s | 34 kB 00:00 2024-03-19T00:08:11.819 INFO:teuthology.orchestra.run.smithi044.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 2.3 MB/s | 139 kB 00:00 2024-03-19T00:08:12.003 INFO:teuthology.orchestra.run.smithi044.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 9.8 MB/s | 1.8 MB 00:00 2024-03-19T00:08:12.953 INFO:teuthology.orchestra.run.smithi044.stdout:(31/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 13 MB/s | 48 MB 00:03 2024-03-19T00:08:13.045 INFO:teuthology.orchestra.run.smithi044.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-03-19T00:08:13.071 INFO:teuthology.orchestra.run.smithi044.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.1 MB/s | 248 kB 00:00 2024-03-19T00:08:13.087 INFO:teuthology.orchestra.run.smithi044.stdout:(34/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 1.2 MB/s | 49 kB 00:00 2024-03-19T00:08:13.104 INFO:teuthology.orchestra.run.smithi044.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2206.gb 495 kB/s | 16 kB 00:00 2024-03-19T00:08:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:(36/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 3.1 MB/s | 264 kB 00:00 2024-03-19T00:08:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:(37/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 7.9 MB/s | 749 kB 00:00 2024-03-19T00:08:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:(38/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 434 kB/s | 18 kB 00:00 2024-03-19T00:08:13.542 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-03-19T00:08:13.597 INFO:teuthology.orchestra.run.smithi044.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-19T00:08:13.639 INFO:teuthology.orchestra.run.smithi044.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 920 kB 00:00 2024-03-19T00:08:13.681 INFO:teuthology.orchestra.run.smithi044.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2024-03-19T00:08:13.706 INFO:teuthology.orchestra.run.smithi044.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-19T00:08:13.723 INFO:teuthology.orchestra.run.smithi044.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 767 kB/s | 32 kB 00:00 2024-03-19T00:08:13.790 INFO:teuthology.orchestra.run.smithi044.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 257 kB/s | 17 kB 00:00 2024-03-19T00:08:13.857 INFO:teuthology.orchestra.run.smithi044.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-19T00:08:13.874 INFO:teuthology.orchestra.run.smithi044.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 10 kB 00:00 2024-03-19T00:08:13.908 INFO:teuthology.orchestra.run.smithi044.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-19T00:08:13.933 INFO:teuthology.orchestra.run.smithi044.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-03-19T00:08:13.967 INFO:teuthology.orchestra.run.smithi044.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-19T00:08:14.000 INFO:teuthology.orchestra.run.smithi044.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-03-19T00:08:14.026 INFO:teuthology.orchestra.run.smithi044.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 791 kB/s | 46 kB 00:00 2024-03-19T00:08:14.059 INFO:teuthology.orchestra.run.smithi044.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-19T00:08:14.085 INFO:teuthology.orchestra.run.smithi044.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-03-19T00:08:14.118 INFO:teuthology.orchestra.run.smithi044.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-19T00:08:14.143 INFO:teuthology.orchestra.run.smithi044.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 584 kB/s | 34 kB 00:00 2024-03-19T00:08:14.344 INFO:teuthology.orchestra.run.smithi044.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-03-19T00:08:14.386 INFO:teuthology.orchestra.run.smithi044.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.5 MB 00:00 2024-03-19T00:08:14.419 INFO:teuthology.orchestra.run.smithi044.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-19T00:08:14.453 INFO:teuthology.orchestra.run.smithi044.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-03-19T00:08:14.737 INFO:teuthology.orchestra.run.smithi044.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2024-03-19T00:08:14.771 INFO:teuthology.orchestra.run.smithi044.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-03-19T00:08:14.796 INFO:teuthology.orchestra.run.smithi044.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-19T00:08:14.960 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-19T00:08:14.971 INFO:teuthology.orchestra.run.smithi044.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-19T00:08:14.988 INFO:teuthology.orchestra.run.smithi044.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 127 kB/s | 24 kB 00:00 2024-03-19T00:08:15.005 INFO:teuthology.orchestra.run.smithi044.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 543 kB/s | 18 kB 00:00 2024-03-19T00:08:15.030 INFO:teuthology.orchestra.run.smithi044.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-19T00:08:15.047 INFO:teuthology.orchestra.run.smithi044.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-03-19T00:08:15.214 INFO:teuthology.orchestra.run.smithi044.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-19T00:08:15.240 INFO:teuthology.orchestra.run.smithi044.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-19T00:08:15.265 INFO:teuthology.orchestra.run.smithi044.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-19T00:08:15.290 INFO:teuthology.orchestra.run.smithi044.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-19T00:08:15.307 INFO:teuthology.orchestra.run.smithi044.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 986 kB/s | 41 kB 00:00 2024-03-19T00:08:15.332 INFO:teuthology.orchestra.run.smithi044.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-19T00:08:15.358 INFO:teuthology.orchestra.run.smithi044.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-19T00:08:15.383 INFO:teuthology.orchestra.run.smithi044.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-19T00:08:15.400 INFO:teuthology.orchestra.run.smithi044.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-19T00:08:15.417 INFO:teuthology.orchestra.run.smithi044.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-19T00:08:15.576 INFO:teuthology.orchestra.run.smithi044.stdout:(78/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 2.5 MB/s | 17 MB 00:06 2024-03-19T00:08:15.709 INFO:teuthology.orchestra.run.smithi044.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 132 kB/s | 39 kB 00:00 2024-03-19T00:08:15.968 INFO:teuthology.orchestra.run.smithi044.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-03-19T00:08:15.993 INFO:teuthology.orchestra.run.smithi044.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 99 kB/s | 41 kB 00:00 2024-03-19T00:08:16.018 INFO:teuthology.orchestra.run.smithi044.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 978 kB/s | 300 kB 00:00 2024-03-19T00:08:16.043 INFO:teuthology.orchestra.run.smithi044.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 756 kB/s | 56 kB 00:00 2024-03-19T00:08:16.069 INFO:teuthology.orchestra.run.smithi044.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-19T00:08:16.094 INFO:teuthology.orchestra.run.smithi044.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-03-19T00:08:16.119 INFO:teuthology.orchestra.run.smithi044.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-19T00:08:16.144 INFO:teuthology.orchestra.run.smithi044.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-19T00:08:16.178 INFO:teuthology.orchestra.run.smithi044.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-19T00:08:16.204 INFO:teuthology.orchestra.run.smithi044.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-19T00:08:16.229 INFO:teuthology.orchestra.run.smithi044.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-19T00:08:16.254 INFO:teuthology.orchestra.run.smithi044.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-19T00:08:16.371 INFO:teuthology.orchestra.run.smithi044.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-19T00:08:16.396 INFO:teuthology.orchestra.run.smithi044.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-19T00:08:16.421 INFO:teuthology.orchestra.run.smithi044.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-19T00:08:16.447 INFO:teuthology.orchestra.run.smithi044.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-03-19T00:08:16.497 INFO:teuthology.orchestra.run.smithi044.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-03-19T00:08:16.523 INFO:teuthology.orchestra.run.smithi044.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-03-19T00:08:16.639 INFO:teuthology.orchestra.run.smithi044.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-03-19T00:08:16.689 INFO:teuthology.orchestra.run.smithi044.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-03-19T00:08:16.715 INFO:teuthology.orchestra.run.smithi044.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 216 kB 00:00 2024-03-19T00:08:16.749 INFO:teuthology.orchestra.run.smithi044.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 584 kB/s | 35 kB 00:00 2024-03-19T00:08:16.774 INFO:teuthology.orchestra.run.smithi044.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 760 kB/s | 44 kB 00:00 2024-03-19T00:08:16.808 INFO:teuthology.orchestra.run.smithi044.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-19T00:08:17.058 INFO:teuthology.orchestra.run.smithi044.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-03-19T00:08:17.257 INFO:teuthology.orchestra.run.smithi044.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.1 MB/s | 4.1 MB 00:00 2024-03-19T00:08:17.483 INFO:teuthology.orchestra.run.smithi044.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-03-19T00:08:17.550 INFO:teuthology.orchestra.run.smithi044.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-03-19T00:08:17.609 INFO:teuthology.orchestra.run.smithi044.stdout:(108/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-19T00:08:17.636 INFO:teuthology.orchestra.run.smithi044.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 800 kB/s | 309 kB 00:00 2024-03-19T00:08:17.711 INFO:teuthology.orchestra.run.smithi044.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 652 kB/s | 49 kB 00:00 2024-03-19T00:08:17.778 INFO:teuthology.orchestra.run.smithi044.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-19T00:08:17.846 INFO:teuthology.orchestra.run.smithi044.stdout:(112/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 110 kB/s | 26 kB 00:00 2024-03-19T00:08:17.913 INFO:teuthology.orchestra.run.smithi044.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-19T00:08:18.096 INFO:teuthology.orchestra.run.smithi044.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-03-19T00:08:18.205 INFO:teuthology.orchestra.run.smithi044.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-03-19T00:08:18.330 INFO:teuthology.orchestra.run.smithi044.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-03-19T00:08:18.355 INFO:teuthology.orchestra.run.smithi044.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 168 kB/s | 43 kB 00:00 2024-03-19T00:08:18.414 INFO:teuthology.orchestra.run.smithi044.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-19T00:08:18.556 INFO:teuthology.orchestra.run.smithi044.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 857 kB/s | 172 kB 00:00 2024-03-19T00:08:18.589 INFO:teuthology.orchestra.run.smithi044.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-03-19T00:08:18.799 INFO:teuthology.orchestra.run.smithi044.stdout:(121/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-03-19T00:08:18.874 INFO:teuthology.orchestra.run.smithi044.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 124 kB/s | 35 kB 00:00 2024-03-19T00:08:18.899 INFO:teuthology.orchestra.run.smithi044.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 884 kB/s | 303 kB 00:00 2024-03-19T00:08:18.924 INFO:teuthology.orchestra.run.smithi044.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-19T00:08:19.000 INFO:teuthology.orchestra.run.smithi044.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-03-19T00:08:19.067 INFO:teuthology.orchestra.run.smithi044.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-19T00:08:19.092 INFO:teuthology.orchestra.run.smithi044.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 751 kB/s | 144 kB 00:00 2024-03-19T00:08:19.134 INFO:teuthology.orchestra.run.smithi044.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-03-19T00:08:19.259 INFO:teuthology.orchestra.run.smithi044.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-03-19T00:08:19.426 INFO:teuthology.orchestra.run.smithi044.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-03-19T00:08:19.451 INFO:teuthology.orchestra.run.smithi044.stdout:(131/174): python3-jaraco-functools-3.5.0-2.el9 54 kB/s | 19 kB 00:00 2024-03-19T00:08:19.502 INFO:teuthology.orchestra.run.smithi044.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-03-19T00:08:19.527 INFO:teuthology.orchestra.run.smithi044.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-03-19T00:08:19.552 INFO:teuthology.orchestra.run.smithi044.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-03-19T00:08:19.686 INFO:teuthology.orchestra.run.smithi044.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-03-19T00:08:19.711 INFO:teuthology.orchestra.run.smithi044.stdout:(136/174): python3-natsort-7.1.1-5.el9.noarch.r 276 kB/s | 58 kB 00:00 2024-03-19T00:08:19.736 INFO:teuthology.orchestra.run.smithi044.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-03-19T00:08:19.778 INFO:teuthology.orchestra.run.smithi044.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 979 kB/s | 90 kB 00:00 2024-03-19T00:08:19.795 INFO:teuthology.orchestra.run.smithi044.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-03-19T00:08:19.812 INFO:teuthology.orchestra.run.smithi044.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-03-19T00:08:19.879 INFO:teuthology.orchestra.run.smithi044.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-19T00:08:19.896 INFO:teuthology.orchestra.run.smithi044.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-03-19T00:08:19.921 INFO:teuthology.orchestra.run.smithi044.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-03-19T00:08:19.938 INFO:teuthology.orchestra.run.smithi044.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 865 kB/s | 51 kB 00:00 2024-03-19T00:08:20.022 INFO:teuthology.orchestra.run.smithi044.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-19T00:08:20.056 INFO:teuthology.orchestra.run.smithi044.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-03-19T00:08:20.081 INFO:teuthology.orchestra.run.smithi044.stdout:(147/174): python3-websocket-client-1.2.3-2.el9 564 kB/s | 90 kB 00:00 2024-03-19T00:08:20.085 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-19T00:08:20.106 INFO:teuthology.orchestra.run.smithi044.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-03-19T00:08:20.140 INFO:teuthology.orchestra.run.smithi044.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-19T00:08:20.398 INFO:teuthology.orchestra.run.smithi044.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-03-19T00:08:20.573 INFO:teuthology.orchestra.run.smithi044.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.8 MB/s | 321 kB 00:00 2024-03-19T00:08:20.758 INFO:teuthology.orchestra.run.smithi044.stdout:(152/174): librados2-19.0.0-2206.gb6ae2491.el9. 5.2 MB/s | 3.4 MB 00:00 2024-03-19T00:08:20.808 INFO:teuthology.orchestra.run.smithi044.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-03-19T00:08:20.850 INFO:teuthology.orchestra.run.smithi044.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-19T00:08:20.892 INFO:teuthology.orchestra.run.smithi044.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-19T00:08:20.942 INFO:teuthology.orchestra.run.smithi044.stdout:(156/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-03-19T00:08:20.968 INFO:teuthology.orchestra.run.smithi044.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 949 kB/s | 71 kB 00:00 2024-03-19T00:08:21.010 INFO:teuthology.orchestra.run.smithi044.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-19T00:08:21.034 INFO:teuthology.orchestra.run.smithi044.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 395 kB/s | 36 kB 00:00 2024-03-19T00:08:21.059 INFO:teuthology.orchestra.run.smithi044.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-19T00:08:21.085 INFO:teuthology.orchestra.run.smithi044.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-03-19T00:08:21.110 INFO:teuthology.orchestra.run.smithi044.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-19T00:08:21.147 INFO:teuthology.orchestra.run.smithi044.stdout:(163/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.7 MB/s | 1.8 MB 00:00 2024-03-19T00:08:21.172 INFO:teuthology.orchestra.run.smithi044.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 155 kB 00:00 2024-03-19T00:08:21.197 INFO:teuthology.orchestra.run.smithi044.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 136 kB 00:00 2024-03-19T00:08:21.214 INFO:teuthology.orchestra.run.smithi044.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 684 kB/s | 29 kB 00:00 2024-03-19T00:08:21.240 INFO:teuthology.orchestra.run.smithi044.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 692 kB/s | 64 kB 00:00 2024-03-19T00:08:21.298 INFO:teuthology.orchestra.run.smithi044.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-03-19T00:08:21.340 INFO:teuthology.orchestra.run.smithi044.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 17 MB/s | 2.2 MB 00:00 2024-03-19T00:08:21.365 INFO:teuthology.orchestra.run.smithi044.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-19T00:08:21.391 INFO:teuthology.orchestra.run.smithi044.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 620 kB/s | 31 kB 00:00 2024-03-19T00:08:21.433 INFO:teuthology.orchestra.run.smithi044.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-19T00:08:21.475 INFO:teuthology.orchestra.run.smithi044.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.7 MB/s | 2.3 MB 00:00 2024-03-19T00:08:21.500 INFO:teuthology.orchestra.run.smithi044.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 684 kB/s | 74 kB 00:00 2024-03-19T00:08:21.507 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:08:21.507 INFO:teuthology.orchestra.run.smithi044.stdout:Total 16 MB/s | 224 MB 00:14 2024-03-19T00:08:21.989 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-03-19T00:08:22.288 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-03-19T00:08:22.298 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-03-19T00:08:22.533 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:08:22.534 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:08:22.550 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T00:08:22.550 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T00:08:22.551 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T00:08:22.552 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T00:08:22.554 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T00:08:22.557 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T00:08:22.558 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T00:08:22.644 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-19T00:08:22.670 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-19T00:08:22.670 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-19T00:08:22.670 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-19T00:08:22.670 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T00:08:22.671 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T00:08:22.672 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T00:08:22.673 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T00:08:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T00:08:22.675 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T00:08:22.676 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T00:08:22.677 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T00:08:22.678 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T00:08:22.679 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T00:08:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T00:08:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T00:08:22.682 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout:Install 150 Packages 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 24 Packages 2024-03-19T00:08:22.683 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:08:22.684 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 224 M 2024-03-19T00:08:22.684 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-19T00:08:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-19T00:08:24.003 INFO:teuthology.orchestra.run.smithi007.stdout:(2/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 7.8 MB/s | 5.4 MB 00:00 2024-03-19T00:08:24.062 INFO:teuthology.orchestra.run.smithi007.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 2.4 MB/s | 145 kB 00:00 2024-03-19T00:08:24.125 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-03-19T00:08:24.125 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-03-19T00:08:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-03-19T00:08:24.487 INFO:teuthology.orchestra.run.smithi007.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-03-19T00:08:24.537 INFO:teuthology.orchestra.run.smithi007.stdout:(6/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 905 kB/s | 891 kB 00:00 2024-03-19T00:08:24.971 INFO:teuthology.orchestra.run.smithi007.stdout:(7/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 12 MB/s | 21 MB 00:01 2024-03-19T00:08:25.046 INFO:teuthology.orchestra.run.smithi007.stdout:(8/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-03-19T00:08:25.130 INFO:teuthology.orchestra.run.smithi007.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 301 kB/s | 25 kB 00:00 2024-03-19T00:08:25.555 INFO:teuthology.orchestra.run.smithi007.stdout:(10/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 17 MB/s | 9.8 MB 00:00 2024-03-19T00:08:25.614 INFO:teuthology.orchestra.run.smithi007.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 3.7 MB/s | 221 kB 00:00 2024-03-19T00:08:25.673 INFO:teuthology.orchestra.run.smithi007.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 464 kB/s | 27 kB 00:00 2024-03-19T00:08:25.731 INFO:teuthology.orchestra.run.smithi007.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 547 kB/s | 32 kB 00:00 2024-03-19T00:08:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 8.0 MB/s | 746 kB 00:00 2024-03-19T00:08:25.890 INFO:teuthology.orchestra.run.smithi007.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.4 MB/s | 163 kB 00:00 2024-03-19T00:08:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.9 MB/s | 127 kB 00:00 2024-03-19T00:08:26.057 INFO:teuthology.orchestra.run.smithi007.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 4.9 MB/s | 501 kB 00:00 2024-03-19T00:08:26.449 INFO:teuthology.orchestra.run.smithi007.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 13 MB/s | 5.1 MB 00:00 2024-03-19T00:08:26.507 INFO:teuthology.orchestra.run.smithi007.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 779 kB/s | 45 kB 00:00 2024-03-19T00:08:26.575 INFO:teuthology.orchestra.run.smithi007.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.9 MB/s | 131 kB 00:00 2024-03-19T00:08:26.641 INFO:teuthology.orchestra.run.smithi007.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 2.4 MB/s | 162 kB 00:00 2024-03-19T00:08:26.725 INFO:teuthology.orchestra.run.smithi007.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.8 MB/s | 321 kB 00:00 2024-03-19T00:08:26.800 INFO:teuthology.orchestra.run.smithi007.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.9 MB/s | 298 kB 00:00 2024-03-19T00:08:26.867 INFO:teuthology.orchestra.run.smithi007.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 99 kB 00:00 2024-03-19T00:08:26.934 INFO:teuthology.orchestra.run.smithi007.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-19T00:08:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:(26/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 12 MB/s | 3.1 MB 00:00 2024-03-19T00:08:27.268 INFO:teuthology.orchestra.run.smithi007.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-19T00:08:27.326 INFO:teuthology.orchestra.run.smithi007.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 582 kB/s | 34 kB 00:00 2024-03-19T00:08:27.385 INFO:teuthology.orchestra.run.smithi007.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 2.3 MB/s | 139 kB 00:00 2024-03-19T00:08:27.552 INFO:teuthology.orchestra.run.smithi007.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 11 MB/s | 1.8 MB 00:00 2024-03-19T00:08:28.461 INFO:teuthology.orchestra.run.smithi007.stdout:(31/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 14 MB/s | 48 MB 00:03 2024-03-19T00:08:28.694 INFO:teuthology.orchestra.run.smithi007.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-03-19T00:08:28.719 INFO:teuthology.orchestra.run.smithi007.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2206.gb6 962 kB/s | 248 kB 00:00 2024-03-19T00:08:28.745 INFO:teuthology.orchestra.run.smithi007.stdout:(34/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 987 kB/s | 49 kB 00:00 2024-03-19T00:08:28.770 INFO:teuthology.orchestra.run.smithi007.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2206.gb 328 kB/s | 16 kB 00:00 2024-03-19T00:08:28.795 INFO:teuthology.orchestra.run.smithi007.stdout:(36/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 5.2 MB/s | 264 kB 00:00 2024-03-19T00:08:28.854 INFO:teuthology.orchestra.run.smithi007.stdout:(37/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 8.8 MB/s | 749 kB 00:00 2024-03-19T00:08:28.879 INFO:teuthology.orchestra.run.smithi007.stdout:(38/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 218 kB/s | 18 kB 00:00 2024-03-19T00:08:29.046 INFO:teuthology.orchestra.run.smithi007.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-19T00:08:29.080 INFO:teuthology.orchestra.run.smithi007.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.5 MB/s | 920 kB 00:00 2024-03-19T00:08:29.105 INFO:teuthology.orchestra.run.smithi007.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-19T00:08:29.131 INFO:teuthology.orchestra.run.smithi007.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-19T00:08:29.156 INFO:teuthology.orchestra.run.smithi007.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-19T00:08:29.239 INFO:teuthology.orchestra.run.smithi007.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-19T00:08:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-19T00:08:29.290 INFO:teuthology.orchestra.run.smithi007.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-03-19T00:08:29.316 INFO:teuthology.orchestra.run.smithi007.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-19T00:08:29.341 INFO:teuthology.orchestra.run.smithi007.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-19T00:08:29.366 INFO:teuthology.orchestra.run.smithi007.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-19T00:08:29.400 INFO:teuthology.orchestra.run.smithi007.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-19T00:08:29.425 INFO:teuthology.orchestra.run.smithi007.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 46 kB 00:00 2024-03-19T00:08:29.450 INFO:teuthology.orchestra.run.smithi007.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-19T00:08:29.475 INFO:teuthology.orchestra.run.smithi007.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-19T00:08:29.501 INFO:teuthology.orchestra.run.smithi007.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-19T00:08:29.518 INFO:teuthology.orchestra.run.smithi007.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 823 kB/s | 34 kB 00:00 2024-03-19T00:08:29.634 INFO:teuthology.orchestra.run.smithi007.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-19T00:08:29.676 INFO:teuthology.orchestra.run.smithi007.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.7 MB/s | 1.5 MB 00:00 2024-03-19T00:08:29.702 INFO:teuthology.orchestra.run.smithi007.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-19T00:08:29.727 INFO:teuthology.orchestra.run.smithi007.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-19T00:08:29.977 INFO:teuthology.orchestra.run.smithi007.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-19T00:08:30.011 INFO:teuthology.orchestra.run.smithi007.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-03-19T00:08:30.037 INFO:teuthology.orchestra.run.smithi007.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-19T00:08:30.170 INFO:teuthology.orchestra.run.smithi007.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-19T00:08:30.195 INFO:teuthology.orchestra.run.smithi007.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 155 kB/s | 24 kB 00:00 2024-03-19T00:08:30.220 INFO:teuthology.orchestra.run.smithi007.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-19T00:08:30.246 INFO:teuthology.orchestra.run.smithi007.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-19T00:08:30.271 INFO:teuthology.orchestra.run.smithi007.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-19T00:08:30.388 INFO:teuthology.orchestra.run.smithi007.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-19T00:08:30.413 INFO:teuthology.orchestra.run.smithi007.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-19T00:08:30.439 INFO:teuthology.orchestra.run.smithi007.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-19T00:08:30.455 INFO:teuthology.orchestra.run.smithi007.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-19T00:08:30.481 INFO:teuthology.orchestra.run.smithi007.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-03-19T00:08:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-19T00:08:30.531 INFO:teuthology.orchestra.run.smithi007.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-19T00:08:30.557 INFO:teuthology.orchestra.run.smithi007.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-19T00:08:30.573 INFO:teuthology.orchestra.run.smithi007.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-19T00:08:30.590 INFO:teuthology.orchestra.run.smithi007.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-19T00:08:30.766 INFO:teuthology.orchestra.run.smithi007.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 223 kB/s | 39 kB 00:00 2024-03-19T00:08:30.991 INFO:teuthology.orchestra.run.smithi007.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-19T00:08:31.008 INFO:teuthology.orchestra.run.smithi007.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 173 kB/s | 41 kB 00:00 2024-03-19T00:08:31.033 INFO:teuthology.orchestra.run.smithi007.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-19T00:08:31.050 INFO:teuthology.orchestra.run.smithi007.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-19T00:08:31.067 INFO:teuthology.orchestra.run.smithi007.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-19T00:08:31.092 INFO:teuthology.orchestra.run.smithi007.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-19T00:08:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-19T00:08:31.143 INFO:teuthology.orchestra.run.smithi007.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-19T00:08:31.185 INFO:teuthology.orchestra.run.smithi007.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-19T00:08:31.210 INFO:teuthology.orchestra.run.smithi007.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-03-19T00:08:31.252 INFO:teuthology.orchestra.run.smithi007.stdout:(89/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-03-19T00:08:31.277 INFO:teuthology.orchestra.run.smithi007.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-19T00:08:31.291 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-03-19T00:08:31.405 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T00:08:31.461 INFO:teuthology.orchestra.run.smithi007.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 899 kB/s | 187 kB 00:00 2024-03-19T00:08:31.538 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T00:08:31.561 INFO:teuthology.orchestra.run.smithi007.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-03-19T00:08:31.587 INFO:teuthology.orchestra.run.smithi007.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-03-19T00:08:31.621 INFO:teuthology.orchestra.run.smithi007.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-19T00:08:31.669 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:08:31.680 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:08:31.721 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T00:08:31.776 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T00:08:31.815 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T00:08:31.857 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T00:08:31.871 INFO:teuthology.orchestra.run.smithi007.stdout:(95/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 2.3 MB/s | 17 MB 00:07 2024-03-19T00:08:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T00:08:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T00:08:32.327 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:08:32.451 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:08:32.486 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:08:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:08:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:08:32.651 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T00:08:32.850 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T00:08:32.897 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T00:08:32.924 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:08:33.029 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:08:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:08:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:08:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T00:08:33.185 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:08:33.201 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:08:33.348 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:08:33.383 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:08:33.423 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T00:08:33.610 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T00:08:33.733 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:08:33.794 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:08:33.896 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:08:33.936 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:08:34.156 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:08:34.271 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T00:08:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T00:08:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T00:08:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T00:08:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T00:08:34.547 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T00:08:34.607 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T00:08:34.646 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T00:08:34.720 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T00:08:34.761 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T00:08:34.805 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T00:08:34.837 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T00:08:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T00:08:34.936 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T00:08:34.970 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T00:08:35.013 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T00:08:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T00:08:35.115 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T00:08:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T00:08:35.240 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T00:08:35.377 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T00:08:35.434 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T00:08:35.992 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T00:08:36.072 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T00:08:36.164 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T00:08:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:08:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:08:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T00:08:36.305 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T00:08:36.342 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T00:08:36.372 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T00:08:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T00:08:37.017 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T00:08:37.083 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T00:08:37.128 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T00:08:37.279 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T00:08:37.330 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T00:08:37.386 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T00:08:37.458 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T00:08:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T00:08:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T00:08:37.573 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T00:08:37.608 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T00:08:37.652 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T00:08:37.693 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T00:08:37.833 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T00:08:37.882 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T00:08:37.912 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T00:08:37.954 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T00:08:38.051 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T00:08:38.087 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T00:08:38.119 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T00:08:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T00:08:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T00:08:38.243 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T00:08:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T00:08:38.480 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T00:08:38.528 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T00:08:38.586 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T00:08:38.642 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T00:08:38.832 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T00:08:38.873 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T00:08:39.018 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T00:08:39.072 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T00:08:39.103 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T00:08:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T00:08:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T00:08:39.188 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T00:08:39.228 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T00:08:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T00:08:39.289 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T00:08:39.368 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T00:08:39.399 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T00:08:39.432 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T00:08:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T00:08:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T00:08:39.601 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T00:08:39.691 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T00:08:39.757 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T00:08:39.787 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T00:08:39.835 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T00:08:39.936 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T00:08:39.990 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T00:08:40.753 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T00:08:40.794 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T00:08:40.828 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T00:08:40.877 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T00:08:40.967 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T00:08:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T00:08:41.188 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T00:08:41.218 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T00:08:41.298 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T00:08:41.984 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T00:08:42.104 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T00:08:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T00:08:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T00:08:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T00:08:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T00:08:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T00:08:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T00:08:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:08:44.015 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:08:44.045 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T00:08:44.082 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T00:08:44.568 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T00:08:44.593 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:08:44.663 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:08:44.686 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T00:08:47.024 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:08:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:08:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:08:47.677 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:08:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:08:47.769 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T00:08:47.769 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T00:08:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:08:47.795 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-03-19T00:08:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:09:00.140 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:09:00.140 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:00.141 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T00:09:00.141 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:09:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:09:01.703 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:01.703 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T00:09:01.703 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:09:01.703 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:09:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:01.936 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:09:01.959 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:09:01.959 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:01.959 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T00:09:01.959 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:09:01.959 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:09:01.959 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:02.420 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:09:02.442 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:09:02.442 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:02.443 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T00:09:02.443 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:09:02.443 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:09:02.443 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:02.539 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T00:09:02.898 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:09:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:09:02.961 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T00:09:03.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T00:09:03.052 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:09:03.298 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:09:03.328 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:09:04.345 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:09:04.370 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:09:04.473 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:09:04.640 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T00:09:04.672 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:09:04.695 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:09:04.695 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:04.695 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T00:09:04.695 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:09:04.695 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:09:04.695 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:04.739 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:09:04.750 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:09:04.803 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T00:09:05.725 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T00:09:05.749 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:09:05.772 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:09:05.772 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:05.772 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T00:09:05.772 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:09:05.772 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:09:05.772 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:09:05.838 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:09:05.839 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:05.839 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T00:09:05.839 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:09:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:09:05.915 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:05.915 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T00:09:05.915 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:09:05.915 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:09:05.915 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:06.235 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:09:06.257 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:09:06.258 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:09:06.258 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T00:09:06.258 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:09:06.258 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:09:06.258 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:10.964 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:09:11.006 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T00:09:11.036 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T00:09:11.064 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T00:09:11.255 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T00:09:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T00:09:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T00:09:11.473 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T00:09:11.538 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T00:09:11.582 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T00:09:11.615 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T00:09:11.656 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T00:09:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T00:09:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T00:09:11.818 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T00:09:11.858 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T00:09:12.486 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:09:12.531 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:09:12.551 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:09:13.430 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T00:09:13.526 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T00:09:13.569 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T00:09:13.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T00:09:13.697 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T00:09:13.722 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T00:09:13.755 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T00:09:13.782 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T00:09:13.782 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:09:13.862 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:09:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T00:09:13.899 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:09:13.976 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:09:14.002 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T00:09:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T00:09:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T00:09:14.086 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T00:09:14.121 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T00:09:14.167 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T00:09:14.207 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T00:09:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T00:09:14.266 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T00:09:14.291 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:09:14.317 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T00:09:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:09:14.324 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:09:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:09:14.511 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:09:14.594 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:09:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:09:14.644 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T00:09:14.677 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T00:09:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T00:09:14.772 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:09:19.471 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:09:19.471 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T00:09:19.471 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T00:09:19.471 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T00:09:19.471 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T00:09:19.471 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T00:09:19.472 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T00:09:19.473 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T00:09:19.491 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T00:09:19.491 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T00:09:19.491 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T00:09:19.492 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T00:09:19.493 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T00:09:19.494 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T00:09:19.495 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T00:09:19.496 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T00:09:19.497 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T00:09:19.498 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T00:09:19.499 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T00:09:19.500 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T00:09:19.501 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T00:09:19.502 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T00:09:19.503 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T00:09:19.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T00:09:19.505 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T00:09:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T00:09:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:22.396 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-03-19T00:09:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T00:09:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T00:09:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T00:09:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T00:09:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T00:09:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: jq-1.6-16.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T00:09:22.403 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T00:09:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T00:09:22.405 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.406 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T00:09:22.407 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T00:09:22.408 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T00:09:22.409 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.410 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T00:09:22.411 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T00:09:22.412 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T00:09:22.413 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T00:09:22.413 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-19T00:09:22.413 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-03-19T00:09:22.863 DEBUG:teuthology.parallel:result is None 2024-03-19T00:10:43.698 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] elfutils-libelf-devel-0.190-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.190-2.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T00:10:43.699 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] libuuid-devel-2.37.4-18.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libuuid-devel-2.37.4-18.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T00:10:43.699 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] openssl-devel-3.0.7-27.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openssl-devel-3.0.7-27.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T00:10:43.971 INFO:teuthology.orchestra.run.smithi007.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 197 B/s | 25 kB 02:12 2024-03-19T00:10:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 166 B/s | 21 kB 02:12 2024-03-19T00:10:44.330 INFO:teuthology.orchestra.run.smithi007.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 684 kB/s | 245 kB 00:00 2024-03-19T00:10:44.464 INFO:teuthology.orchestra.run.smithi007.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 531 kB/s | 248 kB 00:00 2024-03-19T00:10:44.564 INFO:teuthology.orchestra.run.smithi007.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 927 kB/s | 216 kB 00:00 2024-03-19T00:10:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 278 kB/s | 35 kB 00:00 2024-03-19T00:10:44.673 INFO:teuthology.orchestra.run.smithi007.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 410 kB/s | 44 kB 00:00 2024-03-19T00:10:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-03-19T00:10:44.940 INFO:teuthology.orchestra.run.smithi007.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-03-19T00:10:45.299 INFO:teuthology.orchestra.run.smithi007.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-19T00:10:45.357 INFO:teuthology.orchestra.run.smithi007.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-03-19T00:10:45.416 INFO:teuthology.orchestra.run.smithi007.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-03-19T00:10:45.475 INFO:teuthology.orchestra.run.smithi007.stdout:(108/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-19T00:10:45.658 INFO:teuthology.orchestra.run.smithi007.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 142 kB/s | 26 kB 00:00 2024-03-19T00:10:45.733 INFO:teuthology.orchestra.run.smithi007.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-03-19T00:10:45.767 INFO:teuthology.orchestra.run.smithi007.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-03-19T00:10:45.784 INFO:teuthology.orchestra.run.smithi007.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-19T00:10:45.851 INFO:teuthology.orchestra.run.smithi007.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-03-19T00:10:45.876 INFO:teuthology.orchestra.run.smithi007.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-19T00:10:45.926 INFO:teuthology.orchestra.run.smithi007.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-03-19T00:10:45.952 INFO:teuthology.orchestra.run.smithi007.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-03-19T00:10:45.985 INFO:teuthology.orchestra.run.smithi007.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-03-19T00:10:46.011 INFO:teuthology.orchestra.run.smithi007.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-19T00:10:46.044 INFO:teuthology.orchestra.run.smithi007.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-19T00:10:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-19T00:10:46.137 INFO:teuthology.orchestra.run.smithi007.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-03-19T00:10:46.170 INFO:teuthology.orchestra.run.smithi007.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-03-19T00:10:46.229 INFO:teuthology.orchestra.run.smithi007.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-19T00:10:46.296 INFO:teuthology.orchestra.run.smithi007.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-19T00:10:46.346 INFO:teuthology.orchestra.run.smithi007.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-19T00:10:46.371 INFO:teuthology.orchestra.run.smithi007.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-19T00:10:46.397 INFO:teuthology.orchestra.run.smithi007.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-19T00:10:46.422 INFO:teuthology.orchestra.run.smithi007.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-03-19T00:10:46.447 INFO:teuthology.orchestra.run.smithi007.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-19T00:10:46.472 INFO:teuthology.orchestra.run.smithi007.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-03-19T00:10:46.523 INFO:teuthology.orchestra.run.smithi007.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-03-19T00:10:46.556 INFO:teuthology.orchestra.run.smithi007.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-03-19T00:10:46.582 INFO:teuthology.orchestra.run.smithi007.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-19T00:10:46.607 INFO:teuthology.orchestra.run.smithi007.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-19T00:10:46.657 INFO:teuthology.orchestra.run.smithi007.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-03-19T00:10:46.682 INFO:teuthology.orchestra.run.smithi007.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-19T00:10:46.707 INFO:teuthology.orchestra.run.smithi007.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-19T00:10:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-03-19T00:10:46.766 INFO:teuthology.orchestra.run.smithi007.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-03-19T00:10:46.800 INFO:teuthology.orchestra.run.smithi007.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-03-19T00:10:46.842 INFO:teuthology.orchestra.run.smithi007.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-19T00:10:46.868 INFO:teuthology.orchestra.run.smithi007.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-03-19T00:10:46.901 INFO:teuthology.orchestra.run.smithi007.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 870 kB/s | 51 kB 00:00 2024-03-19T00:10:46.959 INFO:teuthology.orchestra.run.smithi007.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-19T00:10:46.985 INFO:teuthology.orchestra.run.smithi007.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-19T00:10:47.043 INFO:teuthology.orchestra.run.smithi007.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 343 kB/s | 20 kB 00:00 2024-03-19T00:10:47.071 INFO:teuthology.orchestra.run.smithi007.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.7 MB/s | 427 kB 00:00 2024-03-19T00:10:47.105 INFO:teuthology.orchestra.run.smithi007.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.0 MB/s | 192 kB 00:00 2024-03-19T00:10:47.299 INFO:teuthology.orchestra.run.smithi007.stdout:(149/174): openssl-devel-3.0.7-27.el9.x86_64.rp 31 kB/s | 4.1 MB 02:15 2024-03-19T00:10:47.389 INFO:teuthology.orchestra.run.smithi007.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-03-19T00:10:47.514 INFO:teuthology.orchestra.run.smithi007.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-03-19T00:10:47.748 INFO:teuthology.orchestra.run.smithi007.stdout:(152/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-03-19T00:10:47.910 INFO:teuthology.orchestra.run.smithi007.stdout:(153/174): librados2-19.0.0-2206.gb6ae2491.el9. 4.2 MB/s | 3.4 MB 00:00 2024-03-19T00:10:48.024 INFO:teuthology.orchestra.run.smithi007.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 361 kB/s | 42 kB 00:00 2024-03-19T00:10:48.165 INFO:teuthology.orchestra.run.smithi007.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-03-19T00:10:48.241 INFO:teuthology.orchestra.run.smithi007.stdout:(156/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.2 MB/s | 2.3 MB 00:00 2024-03-19T00:10:48.266 INFO:teuthology.orchestra.run.smithi007.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 712 kB/s | 71 kB 00:00 2024-03-19T00:10:48.308 INFO:teuthology.orchestra.run.smithi007.stdout:(158/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-03-19T00:10:48.325 INFO:teuthology.orchestra.run.smithi007.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 434 kB/s | 36 kB 00:00 2024-03-19T00:10:48.358 INFO:teuthology.orchestra.run.smithi007.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-19T00:10:48.375 INFO:teuthology.orchestra.run.smithi007.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-03-19T00:10:48.409 INFO:teuthology.orchestra.run.smithi007.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-19T00:10:48.435 INFO:teuthology.orchestra.run.smithi007.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-19T00:10:48.460 INFO:teuthology.orchestra.run.smithi007.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-19T00:10:48.485 INFO:teuthology.orchestra.run.smithi007.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-19T00:10:48.510 INFO:teuthology.orchestra.run.smithi007.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 846 kB/s | 64 kB 00:00 2024-03-19T00:10:48.536 INFO:teuthology.orchestra.run.smithi007.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-03-19T00:10:48.769 INFO:teuthology.orchestra.run.smithi007.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.3 MB/s | 1.2 MB 00:00 2024-03-19T00:10:48.894 INFO:teuthology.orchestra.run.smithi007.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-19T00:10:48.970 INFO:teuthology.orchestra.run.smithi007.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 415 kB/s | 31 kB 00:00 2024-03-19T00:10:49.078 INFO:teuthology.orchestra.run.smithi007.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.8 MB/s | 2.2 MB 00:00 2024-03-19T00:10:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.3 MB/s | 2.3 MB 00:00 2024-03-19T00:10:49.354 INFO:teuthology.orchestra.run.smithi007.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.8 MB/s | 1.1 MB 00:00 2024-03-19T00:10:49.404 INFO:teuthology.orchestra.run.smithi007.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 228 kB/s | 74 kB 00:00 2024-03-19T00:10:49.410 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:10:49.410 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.5 MB/s | 224 MB 02:26 2024-03-19T00:10:49.864 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-19T00:10:50.161 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-19T00:10:50.161 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-19T00:10:51.997 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-19T00:10:51.998 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-19T00:10:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-19T00:10:59.217 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T00:10:59.251 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T00:10:59.289 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:10:59.298 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:10:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T00:10:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T00:10:59.443 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T00:10:59.468 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T00:10:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T00:10:59.865 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T00:10:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:11:00.026 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:11:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:11:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:11:00.184 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:11:00.228 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T00:11:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T00:11:00.474 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T00:11:00.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:11:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:11:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:11:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:11:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T00:11:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:11:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:11:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:11:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:11:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T00:11:01.258 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T00:11:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:11:01.446 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:11:01.549 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:11:01.597 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:11:01.815 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:11:01.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T00:11:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T00:11:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T00:11:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T00:11:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T00:11:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T00:11:02.277 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T00:11:02.315 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T00:11:02.390 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T00:11:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T00:11:02.474 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T00:11:02.507 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T00:11:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T00:11:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T00:11:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T00:11:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T00:11:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T00:11:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T00:11:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T00:11:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T00:11:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T00:11:03.096 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T00:11:03.650 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T00:11:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T00:11:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T00:11:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:11:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:11:03.893 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T00:11:03.934 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T00:11:03.971 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T00:11:04.001 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T00:11:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T00:11:04.465 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T00:11:04.538 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T00:11:04.582 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T00:11:04.714 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T00:11:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T00:11:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T00:11:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T00:11:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T00:11:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T00:11:05.002 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T00:11:05.038 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T00:11:05.082 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T00:11:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T00:11:05.264 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T00:11:05.312 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T00:11:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T00:11:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T00:11:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T00:11:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T00:11:05.566 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T00:11:05.591 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T00:11:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T00:11:05.697 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T00:11:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T00:11:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T00:11:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T00:11:06.045 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T00:11:06.102 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T00:11:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T00:11:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T00:11:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T00:11:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T00:11:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T00:11:06.585 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T00:11:06.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T00:11:06.648 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T00:11:06.679 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T00:11:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T00:11:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T00:11:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T00:11:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T00:11:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T00:11:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T00:11:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T00:11:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T00:11:07.150 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T00:11:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T00:11:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T00:11:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T00:11:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T00:11:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T00:11:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T00:11:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T00:11:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T00:11:08.330 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T00:11:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T00:11:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T00:11:08.642 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T00:11:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T00:11:08.751 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T00:11:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T00:11:09.538 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T00:11:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T00:11:10.888 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T00:11:10.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T00:11:10.951 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T00:11:10.978 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T00:11:11.306 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T00:11:11.333 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:11:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T00:11:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T00:11:11.960 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T00:11:11.986 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:11:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:11:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T00:11:14.432 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:11:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:11:15.036 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:11:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:11:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:11:15.172 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T00:11:15.173 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T00:11:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:11:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-03-19T00:11:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:28.155 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:11:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:11:28.170 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:28.170 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T00:11:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:11:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:11:29.727 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:29.727 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T00:11:29.727 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:11:29.727 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:11:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:11:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:11:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T00:11:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:11:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:11:29.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T00:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T00:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T00:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T00:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:11:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:11:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:11:32.483 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:11:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T00:11:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:11:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:11:32.703 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:32.703 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T00:11:32.703 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:11:32.703 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:11:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:32.748 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:11:32.758 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:11:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T00:11:33.731 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T00:11:33.758 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:11:33.780 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:11:33.780 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:33.780 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T00:11:33.780 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:11:33.780 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:11:33.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:33.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:11:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:11:33.847 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:33.847 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T00:11:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:11:33.923 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:11:33.923 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:33.923 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T00:11:33.923 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:11:33.923 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:11:33.924 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:34.252 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:11:34.274 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:11:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:11:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T00:11:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:11:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:11:34.274 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:11:39.040 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T00:11:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T00:11:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T00:11:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T00:11:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T00:11:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T00:11:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T00:11:39.554 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T00:11:39.589 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T00:11:39.622 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T00:11:39.671 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T00:11:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T00:11:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T00:11:39.829 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T00:11:39.866 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T00:11:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:11:40.547 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:11:40.566 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:11:41.452 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T00:11:41.535 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T00:11:41.577 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T00:11:41.666 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T00:11:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T00:11:41.731 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T00:11:41.756 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T00:11:41.783 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T00:11:41.783 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:11:41.871 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:11:41.908 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T00:11:41.909 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:11:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:11:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T00:11:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T00:11:42.087 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T00:11:42.112 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T00:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T00:11:42.202 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T00:11:42.233 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T00:11:42.259 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T00:11:42.284 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T00:11:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:11:42.335 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T00:11:42.341 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:11:42.342 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:11:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:11:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:11:42.646 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:11:42.670 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:11:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T00:11:42.729 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T00:11:42.755 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T00:11:42.824 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:11:47.652 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:11:47.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T00:11:47.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T00:11:47.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T00:11:47.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:11:47.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T00:11:47.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T00:11:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T00:11:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T00:11:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T00:11:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T00:11:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T00:11:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T00:11:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T00:11:47.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T00:11:47.679 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T00:11:47.680 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T00:11:47.681 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T00:11:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T00:11:47.683 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T00:11:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T00:11:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T00:11:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-16.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T00:11:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T00:11:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T00:11:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T00:11:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T00:11:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T00:11:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T00:11:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T00:11:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:11:50.754 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-19T00:11:51.217 DEBUG:teuthology.parallel:result is None 2024-03-19T00:20:57.161 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 24 kB/s | 19 MB 13:08 2024-03-19T00:21:02.035 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-19T00:21:03.503 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-03-19T00:25:27.473 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 80 kB/s | 21 MB 04:23 2024-03-19T00:25:32.372 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-19T00:25:34.743 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:25:34.744 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:25:34.758 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T00:25:34.759 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T00:25:34.760 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T00:25:34.761 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T00:25:34.763 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T00:25:34.766 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T00:25:34.767 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T00:25:34.847 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T00:25:34.874 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T00:25:34.875 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T00:25:34.876 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T00:25:34.877 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T00:25:34.878 INFO:teuthology.orchestra.run.smithi204.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T00:25:34.879 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T00:25:34.880 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T00:25:34.881 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T00:25:34.882 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T00:25:34.883 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T00:25:34.884 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T00:25:34.885 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T00:25:34.886 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T00:25:34.887 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout:Install 150 Packages 2024-03-19T00:25:34.888 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 24 Packages 2024-03-19T00:25:34.889 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:25:34.889 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 224 M 2024-03-19T00:25:34.889 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-03-19T00:25:35.982 INFO:teuthology.orchestra.run.smithi204.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-19T00:25:36.209 INFO:teuthology.orchestra.run.smithi204.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 3.8 MB/s | 891 kB 00:00 2024-03-19T00:25:36.274 INFO:teuthology.orchestra.run.smithi204.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 2.2 MB/s | 145 kB 00:00 2024-03-19T00:25:36.549 INFO:teuthology.orchestra.run.smithi204.stdout:(4/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 7.3 MB/s | 5.4 MB 00:00 2024-03-19T00:25:36.616 INFO:teuthology.orchestra.run.smithi204.stdout:(5/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-03-19T00:25:36.733 INFO:teuthology.orchestra.run.smithi204.stdout:(6/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 8.1 MB/s | 1.5 MB 00:00 2024-03-19T00:25:37.158 INFO:teuthology.orchestra.run.smithi204.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 8.7 MB/s | 4.7 MB 00:00 2024-03-19T00:25:38.150 INFO:teuthology.orchestra.run.smithi204.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 9.9 MB/s | 9.8 MB 00:00 2024-03-19T00:25:38.325 INFO:teuthology.orchestra.run.smithi204.stdout:(9/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x86 11 MB/s | 17 MB 00:01 2024-03-19T00:25:38.350 INFO:teuthology.orchestra.run.smithi204.stdout:(10/174): ceph-selinux-19.0.0-2206.gb6ae2491.el 126 kB/s | 25 kB 00:00 2024-03-19T00:25:38.434 INFO:teuthology.orchestra.run.smithi204.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.6 MB/s | 221 kB 00:00 2024-03-19T00:25:38.493 INFO:teuthology.orchestra.run.smithi204.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 466 kB/s | 27 kB 00:00 2024-03-19T00:25:38.560 INFO:teuthology.orchestra.run.smithi204.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 476 kB/s | 32 kB 00:00 2024-03-19T00:25:38.660 INFO:teuthology.orchestra.run.smithi204.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 7.3 MB/s | 746 kB 00:00 2024-03-19T00:25:38.735 INFO:teuthology.orchestra.run.smithi204.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.1 MB/s | 163 kB 00:00 2024-03-19T00:25:38.961 INFO:teuthology.orchestra.run.smithi204.stdout:(16/174): ceph-common-19.0.0-2206.gb6ae2491.el9 6.6 MB/s | 21 MB 00:03 2024-03-19T00:25:38.994 INFO:teuthology.orchestra.run.smithi204.stdout:(17/174): librados-devel-19.0.0-2206.gb6ae2491. 489 kB/s | 127 kB 00:00 2024-03-19T00:25:39.036 INFO:teuthology.orchestra.run.smithi204.stdout:(18/174): libradosstriper1-19.0.0-2206.gb6ae249 6.5 MB/s | 501 kB 00:00 2024-03-19T00:25:39.137 INFO:teuthology.orchestra.run.smithi204.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 452 kB/s | 45 kB 00:00 2024-03-19T00:25:39.237 INFO:teuthology.orchestra.run.smithi204.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.3 MB/s | 131 kB 00:00 2024-03-19T00:25:39.354 INFO:teuthology.orchestra.run.smithi204.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.4 MB/s | 162 kB 00:00 2024-03-19T00:25:39.446 INFO:teuthology.orchestra.run.smithi204.stdout:(22/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-19T00:25:39.471 INFO:teuthology.orchestra.run.smithi204.stdout:(23/174): python3-rados-19.0.0-2206.gb6ae2491.e 2.7 MB/s | 321 kB 00:00 2024-03-19T00:25:39.513 INFO:teuthology.orchestra.run.smithi204.stdout:(24/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 4.4 MB/s | 298 kB 00:00 2024-03-19T00:25:39.555 INFO:teuthology.orchestra.run.smithi204.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.2 MB/s | 99 kB 00:00 2024-03-19T00:25:39.580 INFO:teuthology.orchestra.run.smithi204.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-19T00:25:39.664 INFO:teuthology.orchestra.run.smithi204.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-19T00:25:39.731 INFO:teuthology.orchestra.run.smithi204.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 510 kB/s | 34 kB 00:00 2024-03-19T00:25:39.807 INFO:teuthology.orchestra.run.smithi204.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.8 MB/s | 139 kB 00:00 2024-03-19T00:25:39.990 INFO:teuthology.orchestra.run.smithi204.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 9.8 MB/s | 1.8 MB 00:00 2024-03-19T00:25:40.041 INFO:teuthology.orchestra.run.smithi204.stdout:(31/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 6.4 MB/s | 3.1 MB 00:00 2024-03-19T00:25:40.374 INFO:teuthology.orchestra.run.smithi204.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 746 kB/s | 248 kB 00:00 2024-03-19T00:25:40.499 INFO:teuthology.orchestra.run.smithi204.stdout:(33/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-03-19T00:25:40.524 INFO:teuthology.orchestra.run.smithi204.stdout:(34/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 328 kB/s | 49 kB 00:00 2024-03-19T00:25:40.558 INFO:teuthology.orchestra.run.smithi204.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2206.gb 281 kB/s | 16 kB 00:00 2024-03-19T00:25:40.650 INFO:teuthology.orchestra.run.smithi204.stdout:(36/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 8.0 MB/s | 749 kB 00:00 2024-03-19T00:25:40.709 INFO:teuthology.orchestra.run.smithi204.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 314 kB/s | 18 kB 00:00 2024-03-19T00:25:40.735 INFO:teuthology.orchestra.run.smithi204.stdout:(38/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 1.2 MB/s | 264 kB 00:00 2024-03-19T00:25:40.893 INFO:teuthology.orchestra.run.smithi204.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-03-19T00:25:40.926 INFO:teuthology.orchestra.run.smithi204.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 106 kB 00:00 2024-03-19T00:25:40.960 INFO:teuthology.orchestra.run.smithi204.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 920 kB 00:00 2024-03-19T00:25:40.985 INFO:teuthology.orchestra.run.smithi204.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-19T00:25:41.011 INFO:teuthology.orchestra.run.smithi204.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 634 kB/s | 32 kB 00:00 2024-03-19T00:25:41.086 INFO:teuthology.orchestra.run.smithi204.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-19T00:25:41.120 INFO:teuthology.orchestra.run.smithi204.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-19T00:25:41.145 INFO:teuthology.orchestra.run.smithi204.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 172 kB/s | 10 kB 00:00 2024-03-19T00:25:41.170 INFO:teuthology.orchestra.run.smithi204.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-19T00:25:41.195 INFO:teuthology.orchestra.run.smithi204.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-19T00:25:41.221 INFO:teuthology.orchestra.run.smithi204.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-19T00:25:41.246 INFO:teuthology.orchestra.run.smithi204.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-19T00:25:41.271 INFO:teuthology.orchestra.run.smithi204.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-03-19T00:25:41.297 INFO:teuthology.orchestra.run.smithi204.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-19T00:25:41.322 INFO:teuthology.orchestra.run.smithi204.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-19T00:25:41.347 INFO:teuthology.orchestra.run.smithi204.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-19T00:25:41.364 INFO:teuthology.orchestra.run.smithi204.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2024-03-19T00:25:41.815 INFO:teuthology.orchestra.run.smithi204.stdout:(56/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 14 MB/s | 48 MB 00:03 2024-03-19T00:25:41.890 INFO:teuthology.orchestra.run.smithi204.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 8.7 MB/s | 4.7 MB 00:00 2024-03-19T00:25:41.932 INFO:teuthology.orchestra.run.smithi204.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 2.7 MB/s | 1.5 MB 00:00 2024-03-19T00:25:41.965 INFO:teuthology.orchestra.run.smithi204.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.4 MB/s | 811 kB 00:00 2024-03-19T00:25:41.990 INFO:teuthology.orchestra.run.smithi204.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-19T00:25:42.032 INFO:teuthology.orchestra.run.smithi204.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-19T00:25:42.125 INFO:teuthology.orchestra.run.smithi204.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-19T00:25:42.149 INFO:teuthology.orchestra.run.smithi204.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 491 kB/s | 78 kB 00:00 2024-03-19T00:25:42.242 INFO:teuthology.orchestra.run.smithi204.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-19T00:25:42.267 INFO:teuthology.orchestra.run.smithi204.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-19T00:25:42.292 INFO:teuthology.orchestra.run.smithi204.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-03-19T00:25:42.319 INFO:teuthology.orchestra.run.smithi204.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 144 kB 00:00 2024-03-19T00:25:42.344 INFO:teuthology.orchestra.run.smithi204.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 610 kB/s | 32 kB 00:00 2024-03-19T00:25:42.420 INFO:teuthology.orchestra.run.smithi204.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-19T00:25:42.445 INFO:teuthology.orchestra.run.smithi204.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-19T00:25:42.470 INFO:teuthology.orchestra.run.smithi204.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-19T00:25:42.496 INFO:teuthology.orchestra.run.smithi204.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-19T00:25:42.521 INFO:teuthology.orchestra.run.smithi204.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 548 kB/s | 41 kB 00:00 2024-03-19T00:25:42.546 INFO:teuthology.orchestra.run.smithi204.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-19T00:25:42.572 INFO:teuthology.orchestra.run.smithi204.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-19T00:25:42.597 INFO:teuthology.orchestra.run.smithi204.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-19T00:25:42.622 INFO:teuthology.orchestra.run.smithi204.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 634 kB/s | 47 kB 00:00 2024-03-19T00:25:42.647 INFO:teuthology.orchestra.run.smithi204.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 862 kB/s | 43 kB 00:00 2024-03-19T00:25:42.764 INFO:teuthology.orchestra.run.smithi204.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 279 kB/s | 39 kB 00:00 2024-03-19T00:25:42.923 INFO:teuthology.orchestra.run.smithi204.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-03-19T00:25:42.948 INFO:teuthology.orchestra.run.smithi204.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 138 kB/s | 41 kB 00:00 2024-03-19T00:25:42.974 INFO:teuthology.orchestra.run.smithi204.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-19T00:25:42.999 INFO:teuthology.orchestra.run.smithi204.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-19T00:25:43.024 INFO:teuthology.orchestra.run.smithi204.stdout:(84/174): uuid-1.6.2-55.el9.x86_64.rpm 754 kB/s | 56 kB 00:00 2024-03-19T00:25:43.049 INFO:teuthology.orchestra.run.smithi204.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-19T00:25:43.074 INFO:teuthology.orchestra.run.smithi204.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-19T00:25:43.100 INFO:teuthology.orchestra.run.smithi204.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-19T00:25:43.134 INFO:teuthology.orchestra.run.smithi204.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-19T00:25:43.159 INFO:teuthology.orchestra.run.smithi204.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-19T00:25:43.184 INFO:teuthology.orchestra.run.smithi204.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-19T00:25:43.226 INFO:teuthology.orchestra.run.smithi204.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-19T00:25:43.726 INFO:teuthology.orchestra.run.smithi204.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 391 kB/s | 195 kB 00:00 2024-03-19T00:25:43.752 INFO:teuthology.orchestra.run.smithi204.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 316 kB/s | 187 kB 00:00 2024-03-19T00:25:43.827 INFO:teuthology.orchestra.run.smithi204.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 288 kB/s | 21 kB 00:00 2024-03-19T00:25:43.852 INFO:teuthology.orchestra.run.smithi204.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 994 kB/s | 125 kB 00:00 2024-03-19T00:25:43.877 INFO:teuthology.orchestra.run.smithi204.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-03-19T00:25:43.969 INFO:teuthology.orchestra.run.smithi204.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-03-19T00:25:44.019 INFO:teuthology.orchestra.run.smithi204.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-03-19T00:25:44.070 INFO:teuthology.orchestra.run.smithi204.stdout:(99/174): libgfortran-11.4.1-3.el9.x86_64.rpm 952 kB/s | 803 kB 00:00 2024-03-19T00:25:44.095 INFO:teuthology.orchestra.run.smithi204.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 216 kB 00:00 2024-03-19T00:25:44.162 INFO:teuthology.orchestra.run.smithi204.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-03-19T00:25:44.187 INFO:teuthology.orchestra.run.smithi204.stdout:(102/174): qatlib-service-23.11.0-1.el9.x86_64. 298 kB/s | 35 kB 00:00 2024-03-19T00:25:44.212 INFO:teuthology.orchestra.run.smithi204.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 380 kB/s | 44 kB 00:00 2024-03-19T00:25:44.237 INFO:teuthology.orchestra.run.smithi204.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-03-19T00:25:44.471 INFO:teuthology.orchestra.run.smithi204.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-03-19T00:25:44.505 INFO:teuthology.orchestra.run.smithi204.stdout:(106/174): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-03-19T00:25:44.530 INFO:teuthology.orchestra.run.smithi204.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.3 MB/s | 309 kB 00:00 2024-03-19T00:25:44.655 INFO:teuthology.orchestra.run.smithi204.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 129 kB/s | 19 kB 00:00 2024-03-19T00:25:44.689 INFO:teuthology.orchestra.run.smithi204.stdout:(109/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-03-19T00:25:44.714 INFO:teuthology.orchestra.run.smithi204.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-03-19T00:25:44.798 INFO:teuthology.orchestra.run.smithi204.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-03-19T00:25:44.823 INFO:teuthology.orchestra.run.smithi204.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 793 kB/s | 67 kB 00:00 2024-03-19T00:25:44.849 INFO:teuthology.orchestra.run.smithi204.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-03-19T00:25:44.874 INFO:teuthology.orchestra.run.smithi204.stdout:(114/174): liboath-2.6.7-2.el9.x86_64.rpm 263 kB/s | 49 kB 00:00 2024-03-19T00:25:44.907 INFO:teuthology.orchestra.run.smithi204.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 9.1 MB/s | 548 kB 00:00 2024-03-19T00:25:44.933 INFO:teuthology.orchestra.run.smithi204.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-03-19T00:25:44.958 INFO:teuthology.orchestra.run.smithi204.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-03-19T00:25:44.983 INFO:teuthology.orchestra.run.smithi204.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-03-19T00:25:45.008 INFO:teuthology.orchestra.run.smithi204.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-19T00:25:45.042 INFO:teuthology.orchestra.run.smithi204.stdout:(120/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.7 MB/s | 838 kB 00:00 2024-03-19T00:25:45.068 INFO:teuthology.orchestra.run.smithi204.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-03-19T00:25:45.093 INFO:teuthology.orchestra.run.smithi204.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-03-19T00:25:45.118 INFO:teuthology.orchestra.run.smithi204.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-19T00:25:45.143 INFO:teuthology.orchestra.run.smithi204.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-19T00:25:45.172 INFO:teuthology.orchestra.run.smithi204.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.6 MB/s | 144 kB 00:00 2024-03-19T00:25:45.197 INFO:teuthology.orchestra.run.smithi204.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 199 kB/s | 11 kB 00:00 2024-03-19T00:25:45.222 INFO:teuthology.orchestra.run.smithi204.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-19T00:25:45.248 INFO:teuthology.orchestra.run.smithi204.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-03-19T00:25:45.273 INFO:teuthology.orchestra.run.smithi204.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-19T00:25:45.298 INFO:teuthology.orchestra.run.smithi204.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-03-19T00:25:45.357 INFO:teuthology.orchestra.run.smithi204.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.7 MB/s | 2.0 MB 00:00 2024-03-19T00:25:45.382 INFO:teuthology.orchestra.run.smithi204.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-03-19T00:25:45.407 INFO:teuthology.orchestra.run.smithi204.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-19T00:25:45.435 INFO:teuthology.orchestra.run.smithi204.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-19T00:25:45.469 INFO:teuthology.orchestra.run.smithi204.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 5.2 MB/s | 1.0 MB 00:00 2024-03-19T00:25:45.494 INFO:teuthology.orchestra.run.smithi204.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.1 MB/s | 272 kB 00:00 2024-03-19T00:25:45.520 INFO:teuthology.orchestra.run.smithi204.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-19T00:25:45.544 INFO:teuthology.orchestra.run.smithi204.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-03-19T00:25:45.570 INFO:teuthology.orchestra.run.smithi204.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-03-19T00:25:45.599 INFO:teuthology.orchestra.run.smithi204.stdout:(140/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 713 kB/s | 90 kB 00:00 2024-03-19T00:25:45.620 INFO:teuthology.orchestra.run.smithi204.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-19T00:25:45.645 INFO:teuthology.orchestra.run.smithi204.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-03-19T00:25:45.671 INFO:teuthology.orchestra.run.smithi204.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-19T00:25:45.696 INFO:teuthology.orchestra.run.smithi204.stdout:(144/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-19T00:25:45.721 INFO:teuthology.orchestra.run.smithi204.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-03-19T00:25:45.747 INFO:teuthology.orchestra.run.smithi204.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-19T00:25:45.763 INFO:teuthology.orchestra.run.smithi204.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-03-19T00:25:45.789 INFO:teuthology.orchestra.run.smithi204.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-19T00:25:45.814 INFO:teuthology.orchestra.run.smithi204.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-19T00:25:45.856 INFO:teuthology.orchestra.run.smithi204.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-19T00:25:45.923 INFO:teuthology.orchestra.run.smithi204.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-19T00:25:46.090 INFO:teuthology.orchestra.run.smithi204.stdout:(152/174): librados2-19.0.0-2206.gb6ae2491.el9. 11 MB/s | 3.4 MB 00:00 2024-03-19T00:25:46.390 INFO:teuthology.orchestra.run.smithi204.stdout:(153/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-03-19T00:25:46.649 INFO:teuthology.orchestra.run.smithi204.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 163 kB/s | 42 kB 00:00 2024-03-19T00:25:46.782 INFO:teuthology.orchestra.run.smithi204.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.6 MB/s | 1.8 MB 00:00 2024-03-19T00:25:46.899 INFO:teuthology.orchestra.run.smithi204.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 611 kB/s | 71 kB 00:00 2024-03-19T00:25:46.925 INFO:teuthology.orchestra.run.smithi204.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 901 kB/s | 248 kB 00:00 2024-03-19T00:25:46.966 INFO:teuthology.orchestra.run.smithi204.stdout:(158/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-03-19T00:25:47.000 INFO:teuthology.orchestra.run.smithi204.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 363 kB/s | 36 kB 00:00 2024-03-19T00:25:47.034 INFO:teuthology.orchestra.run.smithi204.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-19T00:25:47.092 INFO:teuthology.orchestra.run.smithi204.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-19T00:25:47.142 INFO:teuthology.orchestra.run.smithi204.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 162 kB/s | 23 kB 00:00 2024-03-19T00:25:47.168 INFO:teuthology.orchestra.run.smithi204.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 810 kB/s | 108 kB 00:00 2024-03-19T00:25:47.193 INFO:teuthology.orchestra.run.smithi204.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-19T00:25:47.244 INFO:teuthology.orchestra.run.smithi204.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-19T00:25:47.269 INFO:teuthology.orchestra.run.smithi204.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 634 kB/s | 64 kB 00:00 2024-03-19T00:25:47.294 INFO:teuthology.orchestra.run.smithi204.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 285 kB/s | 29 kB 00:00 2024-03-19T00:25:47.370 INFO:teuthology.orchestra.run.smithi204.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 1.2 MB 00:00 2024-03-19T00:25:47.478 INFO:teuthology.orchestra.run.smithi204.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-03-19T00:25:47.536 INFO:teuthology.orchestra.run.smithi204.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.4 MB/s | 2.3 MB 00:00 2024-03-19T00:25:47.579 INFO:teuthology.orchestra.run.smithi204.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 310 kB/s | 31 kB 00:00 2024-03-19T00:25:47.662 INFO:teuthology.orchestra.run.smithi204.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.5 MB/s | 2.2 MB 00:00 2024-03-19T00:25:47.687 INFO:teuthology.orchestra.run.smithi204.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 686 kB/s | 74 kB 00:00 2024-03-19T00:25:47.721 INFO:teuthology.orchestra.run.smithi204.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-19T00:25:47.728 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:25:47.728 INFO:teuthology.orchestra.run.smithi204.stdout:Total 17 MB/s | 224 MB 00:12 2024-03-19T00:25:48.190 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-03-19T00:25:48.469 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-03-19T00:25:48.469 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-03-19T00:25:50.217 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-03-19T00:25:50.217 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-03-19T00:25:57.117 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-03-19T00:25:57.209 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T00:25:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T00:25:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:25:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:25:57.348 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T00:25:57.410 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T00:25:57.452 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T00:25:57.486 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T00:25:57.537 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T00:25:57.865 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T00:25:57.936 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:25:58.057 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:25:58.079 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:25:58.154 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:25:58.190 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:25:58.220 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T00:25:58.401 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T00:25:58.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T00:25:58.493 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:25:58.579 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:25:58.594 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:25:58.659 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:25:58.690 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T00:25:58.721 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:25:58.736 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:25:58.958 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:25:58.985 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:25:59.017 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T00:25:59.193 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T00:25:59.328 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:25:59.388 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:25:59.490 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:25:59.539 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:25:59.782 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:25:59.887 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T00:25:59.981 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T00:26:00.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T00:26:00.038 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T00:26:00.081 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T00:26:00.155 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T00:26:00.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T00:26:00.248 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T00:26:00.311 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T00:26:00.347 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T00:26:00.398 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T00:26:00.432 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T00:26:00.460 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T00:26:00.528 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T00:26:00.564 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T00:26:00.607 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T00:26:00.636 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T00:26:00.705 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T00:26:00.759 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T00:26:00.849 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T00:26:00.983 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T00:26:01.037 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T00:26:01.554 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T00:26:01.631 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T00:26:01.719 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T00:26:01.760 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:26:01.788 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:26:01.850 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T00:26:01.892 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T00:26:01.929 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T00:26:01.960 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T00:26:02.267 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T00:26:02.464 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T00:26:02.553 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T00:26:02.589 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T00:26:02.728 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T00:26:02.792 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T00:26:03.101 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T00:26:03.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T00:26:03.210 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T00:26:03.247 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T00:26:03.294 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T00:26:03.330 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T00:26:03.373 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T00:26:03.415 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T00:26:03.550 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T00:26:03.603 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T00:26:03.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T00:26:03.679 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T00:26:03.772 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T00:26:03.812 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T00:26:03.852 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T00:26:03.878 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T00:26:03.903 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T00:26:03.994 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T00:26:04.121 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T00:26:04.229 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T00:26:04.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T00:26:04.318 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T00:26:04.382 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T00:26:04.567 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T00:26:04.615 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T00:26:04.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T00:26:04.783 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T00:26:04.812 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T00:26:04.843 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T00:26:04.871 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T00:26:04.897 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T00:26:04.936 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T00:26:04.968 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T00:26:04.998 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T00:26:05.074 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T00:26:05.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T00:26:05.142 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T00:26:05.175 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T00:26:05.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T00:26:05.299 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T00:26:05.388 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T00:26:05.432 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T00:26:05.463 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T00:26:05.520 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T00:26:05.615 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T00:26:05.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T00:26:06.395 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T00:26:06.437 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T00:26:06.472 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T00:26:06.520 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T00:26:06.607 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T00:26:06.664 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T00:26:06.849 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T00:26:06.890 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T00:26:07.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T00:26:07.903 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T00:26:08.027 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T00:26:09.238 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T00:26:09.321 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T00:26:09.353 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T00:26:09.383 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T00:26:09.411 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T00:26:09.724 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T00:26:09.749 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:26:09.840 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:26:09.871 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T00:26:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T00:26:10.368 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T00:26:10.394 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:26:10.465 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:26:10.488 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T00:26:12.719 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:26:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:26:13.294 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:26:13.338 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:26:13.414 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:26:13.415 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T00:26:13.415 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T00:26:13.415 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:13.447 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:26:25.230 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:26:25.230 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-03-19T00:26:25.231 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:25.403 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:26:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:26:25.419 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:25.419 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T00:26:25.419 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:26.941 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:26:26.967 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:26:26.968 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:26.968 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T00:26:26.968 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:26:26.968 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:26:26.968 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:27.201 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:26:27.223 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:26:27.223 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:27.223 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T00:26:27.223 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:26:27.223 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:26:27.224 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:27.668 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:26:27.690 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:26:27.690 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:27.690 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T00:26:27.690 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:26:27.690 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:26:27.691 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:27.795 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T00:26:28.180 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:26:28.203 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:26:28.235 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T00:26:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T00:26:28.326 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:26:28.562 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:26:28.594 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:26:29.559 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:26:29.586 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:26:29.686 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:26:29.840 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T00:26:29.863 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:26:29.886 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:26:29.886 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:29.886 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T00:26:29.886 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:26:29.886 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:26:29.886 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:29.922 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:26:29.933 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:26:29.975 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T00:26:30.845 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T00:26:30.874 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:26:30.897 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:26:30.897 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:30.897 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T00:26:30.897 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:26:30.897 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:26:30.897 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:30.942 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:26:30.963 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:26:30.964 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:30.964 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T00:26:30.964 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:31.017 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:26:31.040 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:26:31.040 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:31.040 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T00:26:31.040 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:26:31.040 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:26:31.040 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:31.360 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:26:31.383 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:26:31.383 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:26:31.383 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T00:26:31.383 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:26:31.383 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:26:31.383 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:35.840 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:26:35.889 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T00:26:35.919 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T00:26:35.947 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T00:26:36.138 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T00:26:36.254 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T00:26:36.303 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T00:26:36.347 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T00:26:36.395 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T00:26:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T00:26:36.472 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T00:26:36.504 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T00:26:36.534 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T00:26:36.593 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T00:26:36.669 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T00:26:36.707 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T00:26:37.303 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:26:37.347 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:26:37.366 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:26:38.182 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T00:26:38.268 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T00:26:38.309 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T00:26:38.409 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T00:26:38.438 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T00:26:38.463 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T00:26:38.488 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T00:26:38.515 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T00:26:38.515 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:26:38.603 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:26:38.633 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T00:26:38.633 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:26:38.726 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:26:38.752 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T00:26:38.785 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T00:26:38.811 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T00:26:38.836 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T00:26:38.870 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T00:26:38.917 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T00:26:38.958 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T00:26:38.983 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T00:26:39.008 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T00:26:39.033 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:26:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T00:26:39.065 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:26:39.065 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:26:39.245 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:26:39.245 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:26:39.328 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:26:39.353 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:26:39.378 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T00:26:39.411 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T00:26:39.437 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T00:26:39.503 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T00:26:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T00:26:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T00:26:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T00:26:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T00:26:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T00:26:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T00:26:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T00:26:44.517 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T00:26:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T00:26:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T00:26:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T00:26:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T00:26:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T00:26:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T00:26:44.534 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T00:26:44.535 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T00:26:44.536 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T00:26:44.537 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T00:26:44.538 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T00:26:44.539 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T00:26:44.540 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T00:26:44.541 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T00:26:44.542 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T00:26:44.543 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T00:26:44.544 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T00:26:44.545 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T00:26:47.546 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T00:26:47.546 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:47.546 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-03-19T00:26:47.546 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T00:26:47.546 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T00:26:47.546 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.547 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T00:26:47.548 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.549 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.550 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-16.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T00:26:47.551 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.552 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T00:26:47.553 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.554 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T00:26:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T00:26:47.556 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T00:26:47.557 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T00:26:47.558 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T00:26:47.559 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T00:26:47.560 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T00:26:47.560 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T00:26:47.560 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T00:26:47.560 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:26:47.560 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-03-19T00:26:48.018 DEBUG:teuthology.parallel:result is None 2024-03-19T00:26:48.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:26:48.239 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T00:26:48.265 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T00:26:48.266 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T00:26:48.266 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T00:26:48.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:26:48.482 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T00:26:48.510 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T00:26:48.510 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T00:26:48.510 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T00:26:48.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:26:48.655 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T00:26:48.679 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T00:26:48.679 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T00:26:48.679 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T00:26:48.681 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-19T00:26:48.681 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:26:48.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T00:26:48.716 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:26:48.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T00:26:48.751 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:26:48.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T00:26:48.782 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-19T00:26:48.782 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:26:48.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T00:26:48.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T00:26:48.880 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:26:48.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T00:26:48.911 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T00:26:48.981 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:26:48.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T00:26:49.011 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T00:26:49.078 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-19T00:26:49.078 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:26:49.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T00:26:49.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T00:26:49.173 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:26:49.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T00:26:49.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T00:26:49.274 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:26:49.274 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T00:26:49.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T00:26:49.370 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-19T00:26:49.370 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:26:49.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T00:26:49.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T00:26:49.464 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:26:49.464 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T00:26:49.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T00:26:49.567 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:26:49.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T00:26:49.596 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T00:26:49.663 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-19T00:26:49.764 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'subvols': {'create': 2}} 2024-03-19T00:26:49.764 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:26:49.764 INFO:tasks.cephadm:Cluster fsid is 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:26:49.764 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-19T00:26:49.765 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.44', 'mon.c': '172.21.15.204'} 2024-03-19T00:26:49.765 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-03-19T00:26:49.765 INFO:tasks.cephadm:First mgr is x 2024-03-19T00:26:49.765 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-19T00:26:49.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-03-19T00:26:49.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-03-19T00:26:49.824 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-03-19T00:26:49.853 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-19T00:26:49.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:26:49.997 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77736/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2206-gb6ae2491', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2206.gb6ae2491'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-18 20:42:27.872142', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240318.181317', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-19T00:26:50.213 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240318.181317, sha1 b6ae24918b03b1b8a19c9263a50655bba80bb810 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:26:50.216 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm 2024-03-19T00:26:50.216 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:51.369 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 00:26 /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:51.370 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.227 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 00:26 /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.227 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.831 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 00:26 /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.831 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.849 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.868 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T00:26:52.901 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 on all hosts... 2024-03-19T00:26:52.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T00:26:52.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T00:26:52.911 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T00:26:53.765 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:26:53.794 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:26:53.848 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:27:23.118 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-19T00:27:23.147 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:27:23.147 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T00:27:23.147 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-03-19T00:27:23.148 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T00:27:23.148 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-03-19T00:27:23.148 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-03-19T00:27:45.330 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-19T00:27:49.752 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:27:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:27:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T00:27:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-03-19T00:27:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T00:27:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:27:49.753 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:27:49.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-03-19T00:27:49.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-03-19T00:27:49.853 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-03-19T00:27:49.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-03-19T00:27:49.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-03-19T00:27:49.948 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-03-19T00:27:49.978 INFO:tasks.cephadm:Writing seed config... 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-19T00:27:49.980 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-19T00:27:49.981 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-19T00:27:49.982 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-19T00:27:49.983 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-19T00:27:49.983 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-19T00:27:49.983 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-19T00:27:49.983 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-19T00:27:49.983 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-19T00:27:49.983 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-19T00:27:49.984 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:27:49.984 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-19T00:27:49.999 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 60839b00-e587-11ee-95c9-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-19T00:27:50.000 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a.service 2024-03-19T00:27:50.042 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x.service 2024-03-19T00:27:50.085 INFO:tasks.cephadm:Bootstrapping... 2024-03-19T00:27:50.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 -v bootstrap --fsid 60839b00-e587-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:27:50.281 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:27:50.281 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810', '-v', 'bootstrap', '--fsid', '60839b00-e587-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-03-19T00:27:50.306 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T00:27:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-19T00:27:50.306 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-03-19T00:27:50.330 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T00:27:50.330 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-03-19T00:27:50.331 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-03-19T00:27:50.337 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-19T00:27:50.338 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-19T00:27:50.344 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-19T00:27:50.344 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-19T00:27:50.349 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-19T00:27:50.356 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-19T00:27:50.356 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-19T00:27:50.356 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-03-19T00:27:50.378 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T00:27:50.379 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-19T00:27:50.379 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-03-19T00:27:50.379 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-03-19T00:27:50.385 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-19T00:27:50.385 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-19T00:27:50.391 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-19T00:27:50.391 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-19T00:27:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-19T00:27:50.403 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-19T00:27:50.403 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-19T00:27:50.403 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-03-19T00:27:50.403 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:27:50.403 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 140517133974976 on /run/cephadm/60839b00-e587-11ee-95c9-87774f69a715.lock 2024-03-19T00:27:50.403 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 140517133974976 acquired on /run/cephadm/60839b00-e587-11ee-95c9-87774f69a715.lock 2024-03-19T00:27:50.404 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-03-19T00:27:50.404 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-03-19T00:27:50.404 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-19T00:27:50.407 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-03-19T00:27:50.407 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-03-19T00:27:50.410 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-19T00:27:50.410 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-19T00:27:50.410 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-19T00:27:50.412 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-19T00:27:50.412 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-19T00:27:50.413 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-19T00:27:50.414 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-19T00:27:50.415 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:93b5a8d362aeee7e8b1f0bbf18238e1bb256c50573ad5b72aff9e382b1bb3ccb 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-19T00:27:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-19T00:27:51.739 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-19T00:27:51.739 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-19T00:27:51.739 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-03-19T00:27:52.825 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-03-19T00:27:52.826 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-03-19T00:27:53.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAJ3Phln5qWBxAAdKdKbfMeILIn6S28qKyF2w== 2024-03-19T00:27:54.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAK3PhlgM9wBBAAtWVKFh6uVP3u4YeCNLjrqA== 2024-03-19T00:27:59.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAM3PhliYecChAAeKb8MPpa/1MS2EptITcTIg== 2024-03-19T00:27:59.651 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-03-19T00:28:00.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-19T00:28:00.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-19T00:28:00.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:28:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-03-19T00:28:01.429 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-03-19T00:28:01.639 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-19T00:28:01.780 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-19T00:28:01.956 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-60839b00-e587-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-60839b00-e587-11ee-95c9-87774f69a715.target. 2024-03-19T00:28:01.956 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-60839b00-e587-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-60839b00-e587-11ee-95c9-87774f69a715.target. 2024-03-19T00:28:02.188 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a 2024-03-19T00:28:02.189 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-19T00:28:02.353 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-60839b00-e587-11ee-95c9-87774f69a715.target.wants/ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-60839b00-e587-11ee-95c9-87774f69a715@.service. 2024-03-19T00:28:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 podman[62455]: 2024-03-19 00:28:02.636853152 +0000 UTC m=+0.145821956 container create 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-19T00:28:02.894 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:28:02.895 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-19T00:28:02.901 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:28:02.901 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-19T00:28:02.902 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-19T00:28:02.902 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-19T00:28:02.902 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-03-19T00:28:02.902 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-03-19T00:28:03.026 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 podman[62455]: 2024-03-19 00:28:02.822467539 +0000 UTC m=+0.331436355 container init 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-19T00:28:03.027 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 podman[62455]: 2024-03-19 00:28:02.825593428 +0000 UTC m=+0.334562237 container start 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0) 2024-03-19T00:28:03.027 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 bash[62455]: 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd 2024-03-19T00:28:03.027 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 systemd[1]: Started Ceph mon.a for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:28:03.027 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 ceph-mon[62481]: mkfs 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:03.027 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:02 smithi007 ceph-mon[62481]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T00:28:04.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.548088s) 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-19T00:28:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.358+0000 7f163be46700 1 Processor -- start 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.358+0000 7f163be46700 1 -- start start 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.358+0000 7f163be46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f163410a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.358+0000 7f163be46700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f163407be30 con 0x7f1634108510 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.358+0000 7f1639be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f163410a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1639be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f163410a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51310/0 (socket says 172.21.15.7:51310) 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1639be2700 1 -- 172.21.15.7:0/1911881226 learned_addr learned my addr 172.21.15.7:0/1911881226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1639be2700 1 -- 172.21.15.7:0/1911881226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163410ae40 con 0x7f1634108510 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1639be2700 1 --2- 172.21.15.7:0/1911881226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f163410a900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f16240098a0 tx=0x7f1624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5144d2afc989d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:04.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1638be0700 1 -- 172.21.15.7:0/1911881226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1624004160 con 0x7f1634108510 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1638be0700 1 -- 172.21.15.7:0/1911881226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1624004300 con 0x7f1634108510 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.359+0000 7f1638be0700 1 -- 172.21.15.7:0/1911881226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16240036a0 con 0x7f1634108510 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 -- 172.21.15.7:0/1911881226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 msgr2=0x7f163410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 --2- 172.21.15.7:0/1911881226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f163410a900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f16240098a0 tx=0x7f1624009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 -- 172.21.15.7:0/1911881226 shutdown_connections 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 --2- 172.21.15.7:0/1911881226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f163410a900 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 -- 172.21.15.7:0/1911881226 >> 172.21.15.7:0/1911881226 conn(0x7f1634101ee0 msgr2=0x7f1634104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 -- 172.21.15.7:0/1911881226 shutdown_connections 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.360+0000 7f163be46700 1 -- 172.21.15.7:0/1911881226 wait complete. 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f163be46700 1 Processor -- start 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f163be46700 1 -- start start 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f163be46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f1634107d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f163be46700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f163410b2b0 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f1639be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f1634107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f1639be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f1634107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51312/0 (socket says 172.21.15.7:51312) 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f1639be2700 1 -- 172.21.15.7:0/1033214265 learned_addr learned my addr 172.21.15.7:0/1033214265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f1639be2700 1 -- 172.21.15.7:0/1033214265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1634108280 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f1639be2700 1 --2- 172.21.15.7:0/1033214265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f1634107d40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1624000c00 tx=0x7f1624004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.361+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1624003db0 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.362+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1624011630 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.362+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16341064c0 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.362+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16341069a0 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.362+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1624011950 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.362+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f162401a460 con 0x7f1634108510 2024-03-19T00:28:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.362+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f162401b780 con 0x7f1634108510 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.363+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1634063940 con 0x7f1634108510 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.365+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1624031020 con 0x7f1634108510 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.439+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1634194a20 con 0x7f1634108510 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.439+0000 7f162affd700 1 -- 172.21.15.7:0/1033214265 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f162401b9d0 con 0x7f1634108510 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.440+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 msgr2=0x7f1634107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.440+0000 7f163be46700 1 --2- 172.21.15.7:0/1033214265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f1634107d40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1624000c00 tx=0x7f1624004750 comp rx=0 tx=0).stop 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.440+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 shutdown_connections 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.440+0000 7f163be46700 1 --2- 172.21.15.7:0/1033214265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1634108510 0x7f1634107d40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.440+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 >> 172.21.15.7:0/1033214265 conn(0x7f1634101ee0 msgr2=0x7f16341041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.440+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 shutdown_connections 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:03.441+0000 7f163be46700 1 -- 172.21.15.7:0/1033214265 wait complete. 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-03-19T00:28:04.026 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-03-19T00:28:04.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: monmap epoch 1 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: last_changed 2024-03-19T00:28:00.008746+0000 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: min_mon_release 19 (squid) 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: election_strategy: 1 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: fsmap 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: osdmap e1: 0 total, 0 up, 0 in 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: mgrmap e1: no daemons active 2024-03-19T00:28:04.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:03 smithi007 ceph-mon[62481]: from='client.? 172.21.15.7:0/1033214265' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-19T00:28:05.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:04 smithi007 ceph-mon[62481]: from='client.? 172.21.15.7:0/432798081' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-19T00:28:05.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:04 smithi007 ceph-mon[62481]: from='client.? 172.21.15.7:0/432798081' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-19T00:28:05.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:05.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-19T00:28:05.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f0851a8e700 1 Processor -- start 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f0851a8e700 1 -- start start 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f0851a8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f0851a8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f084c109b40 con 0x7f084c107180 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51328/0 (socket says 172.21.15.7:51328) 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.451+0000 7f084b7fe700 1 -- 172.21.15.7:0/3863754041 learned_addr learned my addr 172.21.15.7:0/3863754041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f084b7fe700 1 -- 172.21.15.7:0/3863754041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c109cc0 con 0x7f084c107180 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f084b7fe700 1 --2- 172.21.15.7:0/3863754041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c109570 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f083c0098a0 tx=0x7f083c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a3274e8847e7a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f084a7fc700 1 -- 172.21.15.7:0/3863754041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f083c004160 con 0x7f084c107180 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f084a7fc700 1 -- 172.21.15.7:0/3863754041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c004300 con 0x7f084c107180 2024-03-19T00:28:05.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f084a7fc700 1 -- 172.21.15.7:0/3863754041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f083c0036a0 con 0x7f084c107180 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f0851a8e700 1 -- 172.21.15.7:0/3863754041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 msgr2=0x7f084c109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.452+0000 7f0851a8e700 1 --2- 172.21.15.7:0/3863754041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c109570 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f083c0098a0 tx=0x7f083c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 -- 172.21.15.7:0/3863754041 shutdown_connections 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 --2- 172.21.15.7:0/3863754041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c109570 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 -- 172.21.15.7:0/3863754041 >> 172.21.15.7:0/3863754041 conn(0x7f084c101240 msgr2=0x7f084c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 -- 172.21.15.7:0/3863754041 shutdown_connections 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 -- 172.21.15.7:0/3863754041 wait complete. 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 Processor -- start 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 -- start start 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.453+0000 7f0851a8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10a130 con 0x7f084c107180 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51330/0 (socket says 172.21.15.7:51330) 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f084b7fe700 1 -- 172.21.15.7:0/432798081 learned_addr learned my addr 172.21.15.7:0/432798081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f084b7fe700 1 -- 172.21.15.7:0/432798081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19b6a0 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f084b7fe700 1 --2- 172.21.15.7:0/432798081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c19b160 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f083c000c00 tx=0x7f083c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f083c003db0 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c011630 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f083c011950 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19b990 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c19e620 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f083c01a460 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.454+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f083c01b780 con 0x7f084c107180 2024-03-19T00:28:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.455+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c063940 con 0x7f084c107180 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.457+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f083c01b980 con 0x7f084c107180 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.527+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f084c02ddc0 con 0x7f084c107180 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.530+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7f083c01bb60 con 0x7f084c107180 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.530+0000 7f0848ff9700 1 -- 172.21.15.7:0/432798081 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f083c01ad70 con 0x7f084c107180 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 msgr2=0x7f084c19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 --2- 172.21.15.7:0/432798081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c19b160 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f083c000c00 tx=0x7f083c004750 comp rx=0 tx=0).stop 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 shutdown_connections 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 --2- 172.21.15.7:0/432798081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f084c107180 0x7f084c19b160 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 >> 172.21.15.7:0/432798081 conn(0x7f084c101240 msgr2=0x7f084c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 shutdown_connections 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:04.531+0000 7f0851a8e700 1 -- 172.21.15.7:0/432798081 wait complete. 2024-03-19T00:28:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-03-19T00:28:06.175 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:05 smithi007 ceph-mon[62481]: from='client.? 172.21.15.7:0/379550497' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:28:06.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.843+0000 7f059f3b5700 1 Processor -- start 2024-03-19T00:28:06.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.843+0000 7f059f3b5700 1 -- start start 2024-03-19T00:28:06.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.843+0000 7f059f3b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f0598109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:06.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.843+0000 7f059f3b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0598109a50 con 0x7f0598109060 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.843+0000 7f059d151700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f0598109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f059d151700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f0598109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44868/0 (socket says 172.21.15.7:44868) 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f059d151700 1 -- 172.21.15.7:0/1061790706 learned_addr learned my addr 172.21.15.7:0/1061790706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f059d151700 1 -- 172.21.15.7:0/1061790706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0598109bd0 con 0x7f0598109060 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f059d151700 1 --2- 172.21.15.7:0/1061790706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f0598109480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05940098a0 tx=0x7f0594009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53da87b1109c28c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f058ffff700 1 -- 172.21.15.7:0/1061790706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0594004160 con 0x7f0598109060 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f058ffff700 1 -- 172.21.15.7:0/1061790706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0594004300 con 0x7f0598109060 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.844+0000 7f058ffff700 1 -- 172.21.15.7:0/1061790706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0594011a80 con 0x7f0598109060 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 -- 172.21.15.7:0/1061790706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 msgr2=0x7f0598109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 --2- 172.21.15.7:0/1061790706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f0598109480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05940098a0 tx=0x7f0594009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 -- 172.21.15.7:0/1061790706 shutdown_connections 2024-03-19T00:28:06.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 --2- 172.21.15.7:0/1061790706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f0598109480 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 -- 172.21.15.7:0/1061790706 >> 172.21.15.7:0/1061790706 conn(0x7f0598101660 msgr2=0x7f0598103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 -- 172.21.15.7:0/1061790706 shutdown_connections 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 -- 172.21.15.7:0/1061790706 wait complete. 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 Processor -- start 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.845+0000 7f059f3b5700 1 -- start start 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059f3b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f059819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059f3b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0598106860 con 0x7f0598109060 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059d151700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f059819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059d151700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f059819b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44874/0 (socket says 172.21.15.7:44874) 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059d151700 1 -- 172.21.15.7:0/379550497 learned_addr learned my addr 172.21.15.7:0/379550497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059d151700 1 -- 172.21.15.7:0/379550497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819bf10 con 0x7f0598109060 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059d151700 1 --2- 172.21.15.7:0/379550497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f059819b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0594006af0 tx=0x7f0594011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0594003930 con 0x7f0598109060 2024-03-19T00:28:06.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0594003ad0 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059819c200 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0594019590 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059819ee90 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.846+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0594019bf0 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.847+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0594024730 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.847+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059810c030 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.850+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0594024930 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.920+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0598063940 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.920+0000 7f058e7fc700 1 -- 172.21.15.7:0/379550497 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f0594024b10 con 0x7f0598109060 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 msgr2=0x7f059819b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 --2- 172.21.15.7:0/379550497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f059819b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0594006af0 tx=0x7f0594011b70 comp rx=0 tx=0).stop 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 shutdown_connections 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 --2- 172.21.15.7:0/379550497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0598109060 0x7f059819b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 >> 172.21.15.7:0/379550497 conn(0x7f0598101660 msgr2=0x7f0598102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:06.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 shutdown_connections 2024-03-19T00:28:06.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:05.921+0000 7f059f3b5700 1 -- 172.21.15.7:0/379550497 wait complete. 2024-03-19T00:28:06.453 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-03-19T00:28:06.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:06 smithi007 systemd[1]: Stopping Ceph mon.a for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:28:06.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a[62477]: 2024-03-19T00:28:06.711+0000 7f0adbb5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:28:06.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a[62477]: 2024-03-19T00:28:06.711+0000 7f0adbb5d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-19T00:28:06.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:06 smithi007 podman[62729]: 2024-03-19 00:28:06.876014957 +0000 UTC m=+0.266891705 container died 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-19T00:28:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 podman[62729]: 2024-03-19 00:28:07.253331935 +0000 UTC m=+0.644208694 container cleanup 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True) 2024-03-19T00:28:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 bash[62729]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a 2024-03-19T00:28:07.871 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 podman[62743]: 2024-03-19 00:28:07.512520015 +0000 UTC m=+0.628079733 container remove 9d5984843cb965c6a069d44db4f16d59e1adc3b5f35085ff0ce788c6aea8accd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:28:07.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-19T00:28:07.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 systemd[1]: Stopped Ceph mon.a for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:28:07.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 systemd[1]: Starting Ceph mon.a for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:28:08.124 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 podman[62836]: 2024-03-19 00:28:07.871658299 +0000 UTC m=+0.104712762 container create ffb69919e0d5cba101777691e5299ff97cd96cca7ce2dd2b96f8edeb9a8aec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:28:08.124 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:07 smithi007 podman[62836]: 2024-03-19 00:28:07.784231851 +0000 UTC m=+0.017286301 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 podman[62836]: 2024-03-19 00:28:08.065676127 +0000 UTC m=+0.298730566 container init ffb69919e0d5cba101777691e5299ff97cd96cca7ce2dd2b96f8edeb9a8aec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 podman[62836]: 2024-03-19 00:28:08.068438695 +0000 UTC m=+0.301493125 container start ffb69919e0d5cba101777691e5299ff97cd96cca7ce2dd2b96f8edeb9a8aec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: set uid:gid to 167:167 (ceph:ceph) 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: pidfile_write: ignore empty --pid-file 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: load: jerasure load: lrc 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: RocksDB version: 7.9.2 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Git sha 0 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: DB SUMMARY 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: DB Session ID: MON6ONUDRBRAZM6K4TDS 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: CURRENT file: CURRENT 2024-03-19T00:28:08.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: IDENTITY file: IDENTITY 2024-03-19T00:28:08.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-19T00:28:08.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-19T00:28:08.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103772 ; 2024-03-19T00:28:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.error_if_exists: 0 2024-03-19T00:28:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.create_if_missing: 0 2024-03-19T00:28:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.paranoid_checks: 1 2024-03-19T00:28:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-19T00:28:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-19T00:28:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.env: 0x559465352c60 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.fs: PosixFileSystem 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.info_log: 0x559467571ae0 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_file_opening_threads: 16 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.statistics: (nil) 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.use_fsync: 0 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_log_file_size: 0 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.keep_log_file_num: 1000 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.recycle_log_file_num: 0 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.allow_fallocate: 1 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.allow_mmap_reads: 0 2024-03-19T00:28:08.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.allow_mmap_writes: 0 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.use_direct_reads: 0 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.create_missing_column_families: 0 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.db_log_dir: 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.wal_dir: 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.advise_random_on_open: 1 2024-03-19T00:28:08.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.db_write_buffer_size: 0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.write_buffer_manager: 0x5594675441e0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.rate_limiter: (nil) 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.wal_recovery_mode: 2 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.enable_thread_tracking: 0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.enable_pipelined_write: 0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.unordered_write: 0 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-19T00:28:08.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.row_cache: None 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.wal_filter: None 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.allow_ingest_behind: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.two_write_queues: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.manual_wal_flush: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.wal_compression: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.atomic_flush: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.log_readahead_size: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.best_efforts_recovery: 0 2024-03-19T00:28:08.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.allow_data_in_errors: 0 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.db_host_id: __hostname__ 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_background_jobs: 2 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_background_compactions: -1 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_subcompactions: 1 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_total_wal_size: 0 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-19T00:28:08.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_open_files: -1 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bytes_per_sync: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_readahead_size: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_background_flushes: -1 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Compression algorithms supported: 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kZSTD supported: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kXpressCompression supported: 0 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kLZ4HCCompression supported: 1 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kZlibCompression supported: 1 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kSnappyCompression supported: 1 2024-03-19T00:28:08.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kLZ4Compression supported: 1 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: kBZip2Compression supported: 0 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.merge_operator: 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_filter: None 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_filter_factory: None 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.sst_partitioner_factory: None 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559467571be0) 2024-03-19T00:28:08.134 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x559467577090 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-03-19T00:28:08.135 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-03-19T00:28:08.136 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.write_buffer_size: 33554432 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_write_buffer_number: 2 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression: NoCompression 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression: Disabled 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.prefix_extractor: nullptr 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.num_levels: 7 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-19T00:28:08.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.level: 32767 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.strategy: 0 2024-03-19T00:28:08.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.enabled: false 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.target_file_size_base: 67108864 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-19T00:28:08.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.arena_block_size: 1048576 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.disable_auto_compactions: 0 2024-03-19T00:28:08.140 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.table_properties_collectors: 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.inplace_update_support: 0 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-19T00:28:08.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.bloom_locality: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.max_successive_merges: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.paranoid_file_checks: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.force_consistency_checks: 1 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.report_bg_io_stats: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.ttl: 2592000 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.enable_blob_files: false 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.min_blob_size: 0 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.blob_file_size: 268435456 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-19T00:28:08.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.blob_file_starting_level: 0 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2138b59e-2d70-4c0c-9a2d-c839ba20d94e 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710808088096304, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710808088097324, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98983, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96962, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13842, "raw_average_key_size": 49, "raw_value_size": 89405, "raw_average_value_size": 320, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710808088, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2138b59e-2d70-4c0c-9a2d-c839ba20d94e", "db_session_id": "MON6ONUDRBRAZM6K4TDS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710808088097406, "job": 1, "event": "recovery_finished"} 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-19T00:28:08.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55946766a000 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: DB pointer 0x559467654000 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-19T00:28:08.144 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 98.50 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 98.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-19T00:28:08.145 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 31.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 31.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x559467577090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-19T00:28:08.146 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???) e1 preinit fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).mds e1 new map 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).mds e1 print_map 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:28:08.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:28:08.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:28:08.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-19T00:28:08.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 bash[62836]: ffb69919e0d5cba101777691e5299ff97cd96cca7ce2dd2b96f8edeb9a8aec2e 2024-03-19T00:28:08.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 systemd[1]: Started Ceph mon.a for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:28:08.148 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-19T00:28:08.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T00:28:08.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: monmap epoch 1 2024-03-19T00:28:08.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: last_changed 2024-03-19T00:28:00.008746+0000 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: min_mon_release 19 (squid) 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: election_strategy: 1 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: fsmap 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: osdmap e1: 0 total, 0 up, 0 in 2024-03-19T00:28:08.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:08 smithi007 ceph-mon[62854]: mgrmap e1: no daemons active 2024-03-19T00:28:09.162 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.512+0000 7f5bd1f72700 1 Processor -- start 2024-03-19T00:28:09.162 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.512+0000 7f5bd1f72700 1 -- start start 2024-03-19T00:28:09.162 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.512+0000 7f5bd1f72700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:09.162 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.512+0000 7f5bd1f72700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc108480 con 0x7f5bcc107a90 2024-03-19T00:28:09.162 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.513+0000 7f5bcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.513+0000 7f5bcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44880/0 (socket says 172.21.15.7:44880) 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.513+0000 7f5bcb7fe700 1 -- 172.21.15.7:0/1894092933 learned_addr learned my addr 172.21.15.7:0/1894092933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.513+0000 7f5bcb7fe700 1 -- 172.21.15.7:0/1894092933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc108c90 con 0x7f5bcc107a90 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.513+0000 7f5bcb7fe700 1 --2- 172.21.15.7:0/1894092933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc107eb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0098a0 tx=0x7f5bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742e44d34299df0e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bca7fc700 1 -- 172.21.15.7:0/1894092933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bbc004160 con 0x7f5bcc107a90 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bca7fc700 1 -- 172.21.15.7:0/1894092933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5bbc004300 con 0x7f5bcc107a90 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bca7fc700 1 -- 172.21.15.7:0/1894092933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bbc011a80 con 0x7f5bcc107a90 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 -- 172.21.15.7:0/1894092933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 msgr2=0x7f5bcc107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 --2- 172.21.15.7:0/1894092933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc107eb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0098a0 tx=0x7f5bbc009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 -- 172.21.15.7:0/1894092933 shutdown_connections 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 --2- 172.21.15.7:0/1894092933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc107eb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 -- 172.21.15.7:0/1894092933 >> 172.21.15.7:0/1894092933 conn(0x7f5bcc102ff0 msgr2=0x7f5bcc105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 -- 172.21.15.7:0/1894092933 shutdown_connections 2024-03-19T00:28:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.514+0000 7f5bd1f72700 1 -- 172.21.15.7:0/1894092933 wait complete. 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bd1f72700 1 Processor -- start 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bd1f72700 1 -- start start 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bd1f72700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bd1f72700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc10c040 con 0x7f5bcc107a90 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44892/0 (socket says 172.21.15.7:44892) 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bcb7fe700 1 -- 172.21.15.7:0/2948857016 learned_addr learned my addr 172.21.15.7:0/2948857016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.515+0000 7f5bcb7fe700 1 -- 172.21.15.7:0/2948857016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc19bfb0 con 0x7f5bcc107a90 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bcb7fe700 1 --2- 172.21.15.7:0/2948857016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc19ba70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006af0 tx=0x7f5bbc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bbc003930 con 0x7f5bcc107a90 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5bbc003ad0 con 0x7f5bcc107a90 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc19c240 con 0x7f5bcc107a90 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bbc0195e0 con 0x7f5bcc107a90 2024-03-19T00:28:09.164 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc19ef30 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5bbc019c20 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.516+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5bbc024730 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.517+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bcc063940 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.519+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5bbc043dd0 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.589+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f5bcc19f4f0 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.591+0000 7f5bc8ff9700 1 -- 172.21.15.7:0/2948857016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5bbc032030 con 0x7f5bcc107a90 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 msgr2=0x7f5bcc19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 --2- 172.21.15.7:0/2948857016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc19ba70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006af0 tx=0x7f5bbc011b70 comp rx=0 tx=0).stop 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 shutdown_connections 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 --2- 172.21.15.7:0/2948857016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bcc107a90 0x7f5bcc19ba70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 >> 172.21.15.7:0/2948857016 conn(0x7f5bcc102ff0 msgr2=0x7f5bcc103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 shutdown_connections 2024-03-19T00:28:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:08.592+0000 7f5bd1f72700 1 -- 172.21.15.7:0/2948857016 wait complete. 2024-03-19T00:28:09.166 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-19T00:28:09.180 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:28:09.180 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-03-19T00:28:09.180 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-19T00:28:09.181 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-19T00:28:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x 2024-03-19T00:28:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-19T00:28:09.576 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-60839b00-e587-11ee-95c9-87774f69a715.target.wants/ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-60839b00-e587-11ee-95c9-87774f69a715@.service. 2024-03-19T00:28:09.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:09 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2948857016' entity='client.admin' 2024-03-19T00:28:09.715 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:09 smithi007 systemd[1]: Starting Ceph mgr.x for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:28:10.052 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:09 smithi007 podman[63073]: 2024-03-19 00:28:09.79086109 +0000 UTC m=+0.098783547 container create d6d7ca8e28f62c602a66a70dab552d7668b6127aaadbd0cd1687ba161a7f4b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-19T00:28:10.052 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:09 smithi007 podman[63073]: 2024-03-19 00:28:09.710043366 +0000 UTC m=+0.017965831 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:28:10.052 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:09 smithi007 podman[63073]: 2024-03-19 00:28:09.976468949 +0000 UTC m=+0.284391413 container init d6d7ca8e28f62c602a66a70dab552d7668b6127aaadbd0cd1687ba161a7f4b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-03-19T00:28:10.053 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:09 smithi007 podman[63073]: 2024-03-19 00:28:09.978867432 +0000 UTC m=+0.286789895 container start d6d7ca8e28f62c602a66a70dab552d7668b6127aaadbd0cd1687ba161a7f4b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-19T00:28:10.053 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:10 smithi007 bash[63073]: d6d7ca8e28f62c602a66a70dab552d7668b6127aaadbd0cd1687ba161a7f4b68 2024-03-19T00:28:10.065 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:28:10.065 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-19T00:28:10.072 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:28:10.072 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-19T00:28:10.073 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-19T00:28:10.073 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-19T00:28:10.080 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:28:10.081 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-19T00:28:10.088 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:28:10.088 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-19T00:28:10.088 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-19T00:28:10.088 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-19T00:28:10.088 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-03-19T00:28:10.088 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-03-19T00:28:10.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:10 smithi007 systemd[1]: Started Ceph mgr.x for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:28:10.805 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:10.539+0000 7f9f27bcd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:28:10.805 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:10.636+0000 7f9f27bcd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:28:11.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3079157309' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:28:11.551 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:11.312+0000 7f9f27bcd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:28:11.551 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:11.407+0000 7f9f27bcd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:28:11.654 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:11.654 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-19T00:28:11.655 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:28:11.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:28:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:28:02.905343+0000", 2024-03-19T00:28:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.829+0000 7f982fab2700 1 Processor -- start 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982fab2700 1 -- start start 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982fab2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f982810a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982fab2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f982807be70 con 0x7f98281084f0 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982d84e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f982810a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982d84e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f982810a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44916/0 (socket says 172.21.15.7:44916) 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982d84e700 1 -- 172.21.15.7:0/3916283362 learned_addr learned my addr 172.21.15.7:0/3916283362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.830+0000 7f982d84e700 1 -- 172.21.15.7:0/3916283362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982810ae20 con 0x7f98281084f0 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.831+0000 7f982d84e700 1 --2- 172.21.15.7:0/3916283362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f982810a8e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f98240098a0 tx=0x7f9824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4cbf4e3e17ebae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.831+0000 7f982c84c700 1 -- 172.21.15.7:0/3916283362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9824004160 con 0x7f98281084f0 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.831+0000 7f982c84c700 1 -- 172.21.15.7:0/3916283362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9824004300 con 0x7f98281084f0 2024-03-19T00:28:11.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.831+0000 7f982c84c700 1 -- 172.21.15.7:0/3916283362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9824011a80 con 0x7f98281084f0 2024-03-19T00:28:11.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.831+0000 7f982fab2700 1 -- 172.21.15.7:0/3916283362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 msgr2=0x7f982810a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.831+0000 7f982fab2700 1 --2- 172.21.15.7:0/3916283362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f982810a8e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f98240098a0 tx=0x7f9824009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 -- 172.21.15.7:0/3916283362 shutdown_connections 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 --2- 172.21.15.7:0/3916283362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f982810a8e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 -- 172.21.15.7:0/3916283362 >> 172.21.15.7:0/3916283362 conn(0x7f9828101ee0 msgr2=0x7f9828104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 -- 172.21.15.7:0/3916283362 shutdown_connections 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 -- 172.21.15.7:0/3916283362 wait complete. 2024-03-19T00:28:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 Processor -- start 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 -- start start 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f9828107d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982fab2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f982810bde0 con 0x7f98281084f0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.832+0000 7f982d84e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f9828107d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f982d84e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f9828107d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44926/0 (socket says 172.21.15.7:44926) 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f982d84e700 1 -- 172.21.15.7:0/3079157309 learned_addr learned my addr 172.21.15.7:0/3079157309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f982d84e700 1 -- 172.21.15.7:0/3079157309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98281082b0 con 0x7f98281084f0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f982d84e700 1 --2- 172.21.15.7:0/3079157309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f9828107d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9824006af0 tx=0x7f9824011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98240039e0 con 0x7f98281084f0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9824003b80 con 0x7f98281084f0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98281064a0 con 0x7f98281084f0 2024-03-19T00:28:11.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98240196a0 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.833+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98281069e0 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.834+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9824019950 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.834+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98240243f0 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.834+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9828063940 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.837+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9824019dd0 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.910+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9828106f10 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.910+0000 7f981effd700 1 -- 172.21.15.7:0/3079157309 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f98240042a0 con 0x7f98281084f0 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 msgr2=0x7f9828107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 --2- 172.21.15.7:0/3079157309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f9828107d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9824006af0 tx=0x7f9824011c40 comp rx=0 tx=0).stop 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 shutdown_connections 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 --2- 172.21.15.7:0/3079157309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98281084f0 0x7f9828107d70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 >> 172.21.15.7:0/3079157309 conn(0x7f9828101ee0 msgr2=0x7f9828102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 shutdown_connections 2024-03-19T00:28:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:10.911+0000 7f982fab2700 1 -- 172.21.15.7:0/3079157309 wait complete. 2024-03-19T00:28:11.664 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-03-19T00:28:11.964 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:11.550+0000 7f9f27bcd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:28:11.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:11.648+0000 7f9f27bcd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:28:11.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:11.733+0000 7f9f27bcd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:28:12.299 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:12.001+0000 7f9f27bcd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:28:12.299 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:12.113+0000 7f9f27bcd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:28:12.300 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:12.201+0000 7f9f27bcd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:28:12.300 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:12.298+0000 7f9f27bcd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:28:12.714 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:12.476+0000 7f9f27bcd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:28:13.750 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:13.471+0000 7f9f27bcd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:28:13.751 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:13.604+0000 7f9f27bcd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:28:14.027 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:13.950+0000 7f9f27bcd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:28:14.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2204453702' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:28:14.422 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:14.043+0000 7f9f27bcd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:28:14.422 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:14.253+0000 7f9f27bcd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:28:14.714 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:14.445+0000 7f9f27bcd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:28:14.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:14.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:14.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:28:14.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:28:14.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:28:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:28:14.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:28:14.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:28:14.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:28:14.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:14.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:28:02.905343+0000", 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f1628634700 1 Processor -- start 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f1628634700 1 -- start start 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f1628634700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620106150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:14.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f1628634700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1620106720 con 0x7f1620105d30 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f16263d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f16263d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44934/0 (socket says 172.21.15.7:44934) 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f16263d0700 1 -- 172.21.15.7:0/40613206 learned_addr learned my addr 172.21.15.7:0/40613206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.035+0000 7f16263d0700 1 -- 172.21.15.7:0/40613206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1620108cc0 con 0x7f1620105d30 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.036+0000 7f16263d0700 1 --2- 172.21.15.7:0/40613206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620106150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f161c0098a0 tx=0x7f161c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f8a1454865993e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.036+0000 7f16253ce700 1 -- 172.21.15.7:0/40613206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c004160 con 0x7f1620105d30 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.036+0000 7f16253ce700 1 -- 172.21.15.7:0/40613206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f161c004300 con 0x7f1620105d30 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.036+0000 7f16253ce700 1 -- 172.21.15.7:0/40613206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c011a80 con 0x7f1620105d30 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.036+0000 7f1628634700 1 -- 172.21.15.7:0/40613206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 msgr2=0x7f1620106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:14.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.036+0000 7f1628634700 1 --2- 172.21.15.7:0/40613206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620106150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f161c0098a0 tx=0x7f161c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 -- 172.21.15.7:0/40613206 shutdown_connections 2024-03-19T00:28:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 --2- 172.21.15.7:0/40613206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620106150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 -- 172.21.15.7:0/40613206 >> 172.21.15.7:0/40613206 conn(0x7f1620101660 msgr2=0x7f1620103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 -- 172.21.15.7:0/40613206 shutdown_connections 2024-03-19T00:28:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 -- 172.21.15.7:0/40613206 wait complete. 2024-03-19T00:28:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 Processor -- start 2024-03-19T00:28:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.037+0000 7f1628634700 1 -- start start 2024-03-19T00:28:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f1628634700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620108540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f1628634700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f162010c070 con 0x7f1620105d30 2024-03-19T00:28:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16263d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16263d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44946/0 (socket says 172.21.15.7:44946) 2024-03-19T00:28:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16263d0700 1 -- 172.21.15.7:0/2204453702 learned_addr learned my addr 172.21.15.7:0/2204453702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:14.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16263d0700 1 -- 172.21.15.7:0/2204453702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1620108a80 con 0x7f1620105d30 2024-03-19T00:28:14.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16263d0700 1 --2- 172.21.15.7:0/2204453702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620108540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c00 tx=0x7f161c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:14.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c0039c0 con 0x7f1620105d30 2024-03-19T00:28:14.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f161c003b60 con 0x7f1620105d30 2024-03-19T00:28:14.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1620106c70 con 0x7f1620105d30 2024-03-19T00:28:14.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c01a6f0 con 0x7f1620105d30 2024-03-19T00:28:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.038+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16201071b0 con 0x7f1620105d30 2024-03-19T00:28:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.039+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f161c02d3f0 con 0x7f1620105d30 2024-03-19T00:28:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.039+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f161c019700 con 0x7f1620105d30 2024-03-19T00:28:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.039+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1620063940 con 0x7f1620105d30 2024-03-19T00:28:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.042+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f161c024a40 con 0x7f1620105d30 2024-03-19T00:28:14.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.117+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1620107460 con 0x7f1620105d30 2024-03-19T00:28:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.117+0000 7f16177fe700 1 -- 172.21.15.7:0/2204453702 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f161c032030 con 0x7f1620105d30 2024-03-19T00:28:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 msgr2=0x7f1620108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 --2- 172.21.15.7:0/2204453702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620108540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c00 tx=0x7f161c004290 comp rx=0 tx=0).stop 2024-03-19T00:28:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 shutdown_connections 2024-03-19T00:28:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 --2- 172.21.15.7:0/2204453702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1620105d30 0x7f1620108540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 >> 172.21.15.7:0/2204453702 conn(0x7f1620101660 msgr2=0x7f1620102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:14.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 shutdown_connections 2024-03-19T00:28:14.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:14.118+0000 7f1628634700 1 -- 172.21.15.7:0/2204453702 wait complete. 2024-03-19T00:28:14.771 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-03-19T00:28:14.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:14.734+0000 7f9f27bcd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:28:14.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:14.995+0000 7f9f27bcd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:28:15.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:15.236+0000 7f9f27bcd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:28:16.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:16.061+0000 7f9f27bcd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:28:16.953 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:16.672+0000 7f9f27bcd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:28:16.953 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:16.774+0000 7f9f27bcd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:28:17.207 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:16.997+0000 7f9f27bcd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: Activating manager daemon x 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: mgrmap e2: x(active, starting, since 0.00287174s) 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: Manager daemon x is now available 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T00:28:17.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' 2024-03-19T00:28:17.209 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:17 smithi007 ceph-mon[62854]: from='mgr.14100 172.21.15.7:0/3093472135' entity='mgr.x' 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:28:17.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:28:17.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:17.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:28:17.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:28:17.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:28:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:28:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:28:02.905343+0000", 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1f51b700 1 Processor -- start 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1f51b700 1 -- start start 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1f51b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a18107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1f51b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181084a0 con 0x7f7a18107ab0 2024-03-19T00:28:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1d2b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a18107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1d2b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a18107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60832/0 (socket says 172.21.15.7:60832) 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1d2b7700 1 -- 172.21.15.7:0/29518129 learned_addr learned my addr 172.21.15.7:0/29518129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1d2b7700 1 -- 172.21.15.7:0/29518129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a18108c60 con 0x7f7a18107ab0 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.127+0000 7f7a1d2b7700 1 --2- 172.21.15.7:0/29518129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a18107ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009e80 tx=0x7f7a0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba706e00eccd1bb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a17fff700 1 -- 172.21.15.7:0/29518129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a08017070 con 0x7f7a18107ab0 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a17fff700 1 -- 172.21.15.7:0/29518129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7a0800b8c0 con 0x7f7a18107ab0 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a17fff700 1 -- 172.21.15.7:0/29518129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a08004520 con 0x7f7a18107ab0 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 -- 172.21.15.7:0/29518129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 msgr2=0x7f7a18107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 --2- 172.21.15.7:0/29518129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a18107ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009e80 tx=0x7f7a0800b330 comp rx=0 tx=0).stop 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 -- 172.21.15.7:0/29518129 shutdown_connections 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 --2- 172.21.15.7:0/29518129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a18107ed0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 -- 172.21.15.7:0/29518129 >> 172.21.15.7:0/29518129 conn(0x7f7a18103050 msgr2=0x7f7a18105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 -- 172.21.15.7:0/29518129 shutdown_connections 2024-03-19T00:28:17.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.128+0000 7f7a1f51b700 1 -- 172.21.15.7:0/29518129 wait complete. 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1f51b700 1 Processor -- start 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1f51b700 1 -- start start 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1f51b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a1807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1f51b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1810c010 con 0x7f7a18107ab0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1d2b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a1807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1d2b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a1807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60836/0 (socket says 172.21.15.7:60836) 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.129+0000 7f7a1d2b7700 1 -- 172.21.15.7:0/229401908 learned_addr learned my addr 172.21.15.7:0/229401908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a1d2b7700 1 -- 172.21.15.7:0/229401908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1807e0b0 con 0x7f7a18107ab0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a1d2b7700 1 --2- 172.21.15.7:0/229401908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a1807db70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009e50 tx=0x7f7a08004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a08020020 con 0x7f7a18107ab0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7a0800bba0 con 0x7f7a18107ab0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a08017040 con 0x7f7a18107ab0 2024-03-19T00:28:17.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1807c2a0 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.130+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1807c7e0 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.131+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 2) ==== 45862+0+0 (secure 0 0 0) 0x7f7a08023020 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.131+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a0800f050 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.131+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a18063940 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.133+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7a0802a020 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.206+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7a1807ca00 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.207+0000 7f7a167fc700 1 -- 172.21.15.7:0/229401908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7a0800bdc0 con 0x7f7a18107ab0 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 msgr2=0x7f7a1807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 --2- 172.21.15.7:0/229401908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a1807db70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009e50 tx=0x7f7a08004670 comp rx=0 tx=0).stop 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 shutdown_connections 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 --2- 172.21.15.7:0/229401908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a18107ab0 0x7f7a1807db70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 >> 172.21.15.7:0/229401908 conn(0x7f7a18103050 msgr2=0x7f7a18103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:17.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 shutdown_connections 2024-03-19T00:28:17.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:17.208+0000 7f7a1f51b700 1 -- 172.21.15.7:0/229401908 wait complete. 2024-03-19T00:28:17.692 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-03-19T00:28:18.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/229401908' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:28:18.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:18 smithi007 ceph-mon[62854]: mgrmap e3: x(active, since 1.00768s) 2024-03-19T00:28:20.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:20 smithi007 ceph-mon[62854]: mgrmap e4: x(active, since 2s) 2024-03-19T00:28:20.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:20.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:28:20.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T00:28:20.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:28:02.905343+0000", 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:28:20.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.107+0000 7f5531712700 1 Processor -- start 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.107+0000 7f5531712700 1 -- start start 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f5531712700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c10a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f5531712700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f552c07beb0 con 0x7f552c1084b0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f552affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c10a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f552affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c10a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60838/0 (socket says 172.21.15.7:60838) 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f552affd700 1 -- 172.21.15.7:0/715919409 learned_addr learned my addr 172.21.15.7:0/715919409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f552affd700 1 -- 172.21.15.7:0/715919409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c10ade0 con 0x7f552c1084b0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f552affd700 1 --2- 172.21.15.7:0/715919409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c10a8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f551c0098a0 tx=0x7f551c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e844a80e4838f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f5529ffb700 1 -- 172.21.15.7:0/715919409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f551c004160 con 0x7f552c1084b0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f5529ffb700 1 -- 172.21.15.7:0/715919409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f551c004300 con 0x7f552c1084b0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.108+0000 7f5529ffb700 1 -- 172.21.15.7:0/715919409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f551c011a80 con 0x7f552c1084b0 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 -- 172.21.15.7:0/715919409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 msgr2=0x7f552c10a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:20.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 --2- 172.21.15.7:0/715919409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c10a8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f551c0098a0 tx=0x7f551c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 -- 172.21.15.7:0/715919409 shutdown_connections 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 --2- 172.21.15.7:0/715919409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c10a8a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 -- 172.21.15.7:0/715919409 >> 172.21.15.7:0/715919409 conn(0x7f552c101ee0 msgr2=0x7f552c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 -- 172.21.15.7:0/715919409 shutdown_connections 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.109+0000 7f5531712700 1 -- 172.21.15.7:0/715919409 wait complete. 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f5531712700 1 Processor -- start 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f5531712700 1 -- start start 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f5531712700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f5531712700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f552c10bdb0 con 0x7f552c1084b0 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f552affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f552affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60842/0 (socket says 172.21.15.7:60842) 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f552affd700 1 -- 172.21.15.7:0/4037813832 learned_addr learned my addr 172.21.15.7:0/4037813832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:20.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.110+0000 7f552affd700 1 -- 172.21.15.7:0/4037813832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c19be40 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.111+0000 7f552affd700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c19b900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f551c011a50 tx=0x7f551c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.111+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f551c003a80 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.111+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f551c003c20 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.111+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c19c0d0 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.111+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f551c01a650 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.111+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c19edc0 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.112+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f551c024ad0 con 0x7f552c1084b0 2024-03-19T00:28:20.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.112+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c063940 con 0x7f552c1084b0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.112+0000 7f5513fff700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f5514038f20 0x7f551403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.112+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f551c04d210 con 0x7f552c1084b0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.112+0000 7f552a7fc700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f5514038f20 0x7f551403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.112+0000 7f552a7fc700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f5514038f20 0x7f551403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5520006fd0 tx=0x7f5520006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.117+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f551c02d4b0 con 0x7f552c1084b0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.405+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f552c19ef90 con 0x7f552c1084b0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.405+0000 7f5513fff700 1 -- 172.21.15.7:0/4037813832 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f552c19ef90 con 0x7f552c1084b0 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f5514038f20 msgr2=0x7f551403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f5514038f20 0x7f551403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5520006fd0 tx=0x7f5520006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 msgr2=0x7f552c19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c19b900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f551c011a50 tx=0x7f551c011b30 comp rx=0 tx=0).stop 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 shutdown_connections 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f5514038f20 0x7f551403b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 --2- 172.21.15.7:0/4037813832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c1084b0 0x7f552c19b900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.408+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 >> 172.21.15.7:0/4037813832 conn(0x7f552c101ee0 msgr2=0x7f552c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.409+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 shutdown_connections 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:20.409+0000 7f5531712700 1 -- 172.21.15.7:0/4037813832 wait complete. 2024-03-19T00:28:20.987 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-03-19T00:28:21.380 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:21 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4037813832' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:28:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-19T00:28:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-19T00:28:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-19T00:28:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.386+0000 7f793a04c700 1 Processor -- start 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.386+0000 7f793a04c700 1 -- start start 2024-03-19T00:28:22.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.386+0000 7f793a04c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f79341066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.386+0000 7f793a04c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f793407beb0 con 0x7f7934106280 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f79341066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f79341066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60856/0 (socket says 172.21.15.7:60856) 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79337fe700 1 -- 172.21.15.7:0/1277421537 learned_addr learned my addr 172.21.15.7:0/1277421537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79337fe700 1 -- 172.21.15.7:0/1277421537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7934106be0 con 0x7f7934106280 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79337fe700 1 --2- 172.21.15.7:0/1277421537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f79341066a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7924009e80 tx=0x7f792400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f7dc5df76d8f4d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79327fc700 1 -- 172.21.15.7:0/1277421537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7924017070 con 0x7f7934106280 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79327fc700 1 -- 172.21.15.7:0/1277421537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f792400b8c0 con 0x7f7934106280 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.387+0000 7f79327fc700 1 -- 172.21.15.7:0/1277421537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7924004520 con 0x7f7934106280 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 -- 172.21.15.7:0/1277421537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 msgr2=0x7f79341066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 --2- 172.21.15.7:0/1277421537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f79341066a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7924009e80 tx=0x7f792400b330 comp rx=0 tx=0).stop 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 -- 172.21.15.7:0/1277421537 shutdown_connections 2024-03-19T00:28:22.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 --2- 172.21.15.7:0/1277421537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f79341066a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 -- 172.21.15.7:0/1277421537 >> 172.21.15.7:0/1277421537 conn(0x7f7934101ee0 msgr2=0x7f7934104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 -- 172.21.15.7:0/1277421537 shutdown_connections 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 -- 172.21.15.7:0/1277421537 wait complete. 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.388+0000 7f793a04c700 1 Processor -- start 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f793a04c700 1 -- start start 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f793a04c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f793419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f793a04c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7934107c10 con 0x7f7934106280 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f79337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f793419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f79337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f793419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60866/0 (socket says 172.21.15.7:60866) 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f79337fe700 1 -- 172.21.15.7:0/2766290174 learned_addr learned my addr 172.21.15.7:0/2766290174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f79337fe700 1 -- 172.21.15.7:0/2766290174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793419e0b0 con 0x7f7934106280 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f79337fe700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f793419db70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f792400b7b0 tx=0x7f7924004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7924020020 con 0x7f7934106280 2024-03-19T00:28:22.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f792400bba0 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793419e340 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7924017040 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.389+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79341a1030 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.390+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f7924023020 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.390+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7934063940 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.390+0000 7f7930ff9700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f7920038e80 0x7f792003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.391+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f792400f050 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.391+0000 7f7932ffd700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f7920038e80 0x7f792003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.391+0000 7f7932ffd700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f7920038e80 0x7f792003b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f791c006fd0 tx=0x7f791c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.395+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f792402a020 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.598+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f793402df20 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.599+0000 7f7930ff9700 1 -- 172.21.15.7:0/2766290174 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f7924007310 con 0x7f7934106280 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f7920038e80 msgr2=0x7f792003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:22.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f7920038e80 0x7f792003b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f791c006fd0 tx=0x7f791c006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 msgr2=0x7f793419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f793419db70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f792400b7b0 tx=0x7f7924004670 comp rx=0 tx=0).stop 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 shutdown_connections 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f7920038e80 0x7f792003b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 --2- 172.21.15.7:0/2766290174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7934106280 0x7f793419db70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 >> 172.21.15.7:0/2766290174 conn(0x7f7934101ee0 msgr2=0x7f79341947e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 shutdown_connections 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:21.601+0000 7f793a04c700 1 -- 172.21.15.7:0/2766290174 wait complete. 2024-03-19T00:28:22.298 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-03-19T00:28:22.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:22 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2766290174' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-19T00:28:23.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4233633093' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-19T00:28:23.364 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: ignoring --setuser ceph since I am not root 2024-03-19T00:28:23.364 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: ignoring --setgroup ceph since I am not root 2024-03-19T00:28:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.740+0000 7f3f32d7c700 1 Processor -- start 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.740+0000 7f3f32d7c700 1 -- start start 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f32d7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c10a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f32d7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c07be50 con 0x7f3f2c108500 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f30b18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c10a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f30b18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c10a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60874/0 (socket says 172.21.15.7:60874) 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f30b18700 1 -- 172.21.15.7:0/683956049 learned_addr learned my addr 172.21.15.7:0/683956049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f30b18700 1 -- 172.21.15.7:0/683956049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c10ae30 con 0x7f3f2c108500 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.741+0000 7f3f30b18700 1 --2- 172.21.15.7:0/683956049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c10a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f200098a0 tx=0x7f3f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa9e65f45ffe385 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f2b7fe700 1 -- 172.21.15.7:0/683956049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f20004160 con 0x7f3f2c108500 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f2b7fe700 1 -- 172.21.15.7:0/683956049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3f20004300 con 0x7f3f2c108500 2024-03-19T00:28:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f2b7fe700 1 -- 172.21.15.7:0/683956049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f20011a80 con 0x7f3f2c108500 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 -- 172.21.15.7:0/683956049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 msgr2=0x7f3f2c10a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 --2- 172.21.15.7:0/683956049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c10a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f200098a0 tx=0x7f3f20009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 -- 172.21.15.7:0/683956049 shutdown_connections 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 --2- 172.21.15.7:0/683956049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c10a8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 -- 172.21.15.7:0/683956049 >> 172.21.15.7:0/683956049 conn(0x7f3f2c101ef0 msgr2=0x7f3f2c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 -- 172.21.15.7:0/683956049 shutdown_connections 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.742+0000 7f3f32d7c700 1 -- 172.21.15.7:0/683956049 wait complete. 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f32d7c700 1 Processor -- start 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f32d7c700 1 -- start start 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f32d7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f32d7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c10bda0 con 0x7f3f2c108500 2024-03-19T00:28:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f30b18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f30b18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60888/0 (socket says 172.21.15.7:60888) 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f30b18700 1 -- 172.21.15.7:0/4233633093 learned_addr learned my addr 172.21.15.7:0/4233633093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f30b18700 1 -- 172.21.15.7:0/4233633093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c19bee0 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.743+0000 7f3f30b18700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c19b9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f20011a50 tx=0x7f3f20004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.744+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f200039f0 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.744+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3f20003b90 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.744+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c19c170 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.744+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f2001a600 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.744+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c19ee60 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.745+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f3f2001a900 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.745+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c063940 con 0x7f3f2c108500 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.745+0000 7f3f29ffb700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f3f1c038ed0 0x7f3f1c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.745+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f2004d430 con 0x7f3f2c108500 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.745+0000 7f3f2bfff700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f3f1c038ed0 0x7f3f1c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.745+0000 7f3f2bfff700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f3f1c038ed0 0x7f3f1c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f18006fd0 tx=0x7f3f18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:22.750+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f2000bc90 con 0x7f3f2c108500 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.010+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f3f2c19f030 con 0x7f3f2c108500 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.066+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f3f2c19f030 con 0x7f3f2c108500 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.068+0000 7f3f29ffb700 1 -- 172.21.15.7:0/4233633093 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f3f20024a40 con 0x7f3f2c108500 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f3f1c038ed0 msgr2=0x7f3f1c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f3f1c038ed0 0x7f3f1c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f18006fd0 tx=0x7f3f18006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 msgr2=0x7f3f2c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c19b9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f20011a50 tx=0x7f3f20004290 comp rx=0 tx=0).stop 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 shutdown_connections 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7f3f1c038ed0 0x7f3f1c03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 --2- 172.21.15.7:0/4233633093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f2c108500 0x7f3f2c19b9a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 >> 172.21.15.7:0/4233633093 conn(0x7f3f2c101ef0 msgr2=0x7f3f2c102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 shutdown_connections 2024-03-19T00:28:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:23.069+0000 7f3f32d7c700 1 -- 172.21.15.7:0/4233633093 wait complete. 2024-03-19T00:28:23.910 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:23.649+0000 7f2e88cc7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:28:23.910 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:23.747+0000 7f2e88cc7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:28:24.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4233633093' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-19T00:28:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-mon[62854]: mgrmap e5: x(active, since 6s) 2024-03-19T00:28:24.522 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:24.421+0000 7f2e88cc7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:28:24.523 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:24.522+0000 7f2e88cc7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:28:24.779 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:24.668+0000 7f2e88cc7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:28:24.779 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:24.767+0000 7f2e88cc7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:28:24.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:24.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-19T00:28:24.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T00:28:24.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.008+0000 7efc2e1e1700 1 Processor -- start 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.008+0000 7efc2e1e1700 1 -- start start 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.008+0000 7efc2e1e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2810a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.008+0000 7efc2e1e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc2807be70 con 0x7efc281084f0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2810a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2810a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60910/0 (socket says 172.21.15.7:60910) 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc277fe700 1 -- 172.21.15.7:0/1921753627 learned_addr learned my addr 172.21.15.7:0/1921753627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc277fe700 1 -- 172.21.15.7:0/1921753627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2810ae20 con 0x7efc281084f0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc277fe700 1 --2- 172.21.15.7:0/1921753627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2810a8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0098a0 tx=0x7efc1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bebc5e0386a7948 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc267fc700 1 -- 172.21.15.7:0/1921753627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc1c004160 con 0x7efc281084f0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc267fc700 1 -- 172.21.15.7:0/1921753627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efc1c004300 con 0x7efc281084f0 2024-03-19T00:28:24.789 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.009+0000 7efc267fc700 1 -- 172.21.15.7:0/1921753627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc1c011a80 con 0x7efc281084f0 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 -- 172.21.15.7:0/1921753627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 msgr2=0x7efc2810a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 --2- 172.21.15.7:0/1921753627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2810a8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0098a0 tx=0x7efc1c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 -- 172.21.15.7:0/1921753627 shutdown_connections 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 --2- 172.21.15.7:0/1921753627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2810a8e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 -- 172.21.15.7:0/1921753627 >> 172.21.15.7:0/1921753627 conn(0x7efc28101ee0 msgr2=0x7efc28104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 -- 172.21.15.7:0/1921753627 shutdown_connections 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.010+0000 7efc2e1e1700 1 -- 172.21.15.7:0/1921753627 wait complete. 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc2e1e1700 1 Processor -- start 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc2e1e1700 1 -- start start 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc2e1e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc2e1e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc2810bde0 con 0x7efc281084f0 2024-03-19T00:28:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60926/0 (socket says 172.21.15.7:60926) 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.011+0000 7efc277fe700 1 -- 172.21.15.7:0/519675464 learned_addr learned my addr 172.21.15.7:0/519675464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc277fe700 1 -- 172.21.15.7:0/519675464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2819be40 con 0x7efc281084f0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc277fe700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2819b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efc1c006af0 tx=0x7efc1c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc1c0039e0 con 0x7efc281084f0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efc1c003b80 con 0x7efc281084f0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2819c0d0 con 0x7efc281084f0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc1c0196a0 con 0x7efc281084f0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.012+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2819edc0 con 0x7efc281084f0 2024-03-19T00:28:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.013+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7efc1c019950 con 0x7efc281084f0 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.013+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc28063940 con 0x7efc281084f0 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.013+0000 7efc24ff9700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 0x7efc1003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.013+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efc1c04d580 con 0x7efc281084f0 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.013+0000 7efc26ffd700 1 -- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 msgr2=0x7efc1003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.013+0000 7efc26ffd700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 0x7efc1003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.018+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc1c04cfa0 con 0x7efc281084f0 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.214+0000 7efc26ffd700 1 -- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 msgr2=0x7efc1003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.214+0000 7efc26ffd700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 0x7efc1003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:28:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.282+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7efc28106770 con 0x7efc281084f0 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.282+0000 7efc24ff9700 1 -- 172.21.15.7:0/519675464 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7efc1c019d40 con 0x7efc281084f0 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 msgr2=0x7efc1003b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 0x7efc1003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 msgr2=0x7efc2819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2819b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efc1c006af0 tx=0x7efc1c011c40 comp rx=0 tx=0).stop 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 shutdown_connections 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7efc10038f20 0x7efc1003b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 --2- 172.21.15.7:0/519675464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc281084f0 0x7efc2819b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 >> 172.21.15.7:0/519675464 conn(0x7efc28101ee0 msgr2=0x7efc28102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 shutdown_connections 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:24.285+0000 7efc2e1e1700 1 -- 172.21.15.7:0/519675464 wait complete. 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-19T00:28:24.793 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-03-19T00:28:25.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:24.858+0000 7f2e88cc7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:28:25.337 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/519675464' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-19T00:28:25.338 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:25.131+0000 7f2e88cc7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:28:25.338 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:25.248+0000 7f2e88cc7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:28:25.338 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:25.337+0000 7f2e88cc7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:28:25.613 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:25.434+0000 7f2e88cc7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:28:25.613 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:25.612+0000 7f2e88cc7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:28:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:26.605+0000 7f2e88cc7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:28:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:26.740+0000 7f2e88cc7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:28:27.373 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:27.079+0000 7f2e88cc7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:28:27.373 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:27.170+0000 7f2e88cc7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:28:27.373 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:27.372+0000 7f2e88cc7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:28:27.714 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:27.563+0000 7f2e88cc7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:28:28.115 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:27.854+0000 7f2e88cc7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:28:28.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:28.114+0000 7f2e88cc7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:28:28.465 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:28.358+0000 7f2e88cc7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:28:29.465 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:29.188+0000 7f2e88cc7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:28:30.093 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:29.770+0000 7f2e88cc7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:28:30.094 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:29.870+0000 7f2e88cc7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:28:30.094 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:30.092+0000 7f2e88cc7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: Active manager daemon x restarted 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: Activating manager daemon x 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: osdmap e2: 0 total, 0 up, 0 in 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: mgrmap e6: x(active, starting, since 0.00331035s) 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T00:28:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: Manager daemon x is now available 2024-03-19T00:28:30.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:30.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:30.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:30.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:30 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:31.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: Found migration_current of "None". Setting to last migration. 2024-03-19T00:28:31.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: mgrmap e7: x(active, since 1.00514s) 2024-03-19T00:28:31.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:31 smithi007 ceph-mon[62854]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-19T00:28:31.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:31.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa0869e9700 1 Processor -- start 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa0869e9700 1 -- start start 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa0869e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa0869e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa080107310 con 0x7fa080106920 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa07ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa07ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43824/0 (socket says 172.21.15.7:43824) 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa07ffff700 1 -- 172.21.15.7:0/853564156 learned_addr learned my addr 172.21.15.7:0/853564156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.232+0000 7fa07ffff700 1 -- 172.21.15.7:0/853564156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa080107b20 con 0x7fa080106920 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa07ffff700 1 --2- 172.21.15.7:0/853564156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080106d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0700098a0 tx=0x7fa070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe96c49cbe7355 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa07effd700 1 -- 172.21.15.7:0/853564156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa070004160 con 0x7fa080106920 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa07effd700 1 -- 172.21.15.7:0/853564156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa070004300 con 0x7fa080106920 2024-03-19T00:28:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa07effd700 1 -- 172.21.15.7:0/853564156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa070011a80 con 0x7fa080106920 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa0869e9700 1 -- 172.21.15.7:0/853564156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 msgr2=0x7fa080106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa0869e9700 1 --2- 172.21.15.7:0/853564156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080106d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0700098a0 tx=0x7fa070009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa0869e9700 1 -- 172.21.15.7:0/853564156 shutdown_connections 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa0869e9700 1 --2- 172.21.15.7:0/853564156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080106d40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.233+0000 7fa0869e9700 1 -- 172.21.15.7:0/853564156 >> 172.21.15.7:0/853564156 conn(0x7fa080101ee0 msgr2=0x7fa080104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.234+0000 7fa0869e9700 1 -- 172.21.15.7:0/853564156 shutdown_connections 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.234+0000 7fa0869e9700 1 -- 172.21.15.7:0/853564156 wait complete. 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.234+0000 7fa0869e9700 1 Processor -- start 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.234+0000 7fa0869e9700 1 -- start start 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa0869e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080191a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa0869e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010aed0 con 0x7fa080106920 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080191a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:31.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080191a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43826/0 (socket says 172.21.15.7:43826) 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07ffff700 1 -- 172.21.15.7:0/3285729647 learned_addr learned my addr 172.21.15.7:0/3285729647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07ffff700 1 -- 172.21.15.7:0/3285729647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa080191fd0 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07ffff700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080191a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa070000c00 tx=0x7fa070004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0700039c0 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa070003b60 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0801901c0 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa080190700 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.235+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa07001a6f0 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.236+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fa07001a890 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.237+0000 7fa07d7fa700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.237+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 --> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa06803bb40 con 0x7fa068038f70 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.237+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa07004d020 con 0x7fa080106920 2024-03-19T00:28:31.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.237+0000 7fa07f7fe700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 msgr2=0x7fa06803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.237+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.438+0000 7fa07f7fe700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 msgr2=0x7fa06803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.438+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.839+0000 7fa07f7fe700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 msgr2=0x7fa06803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:25.839+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:26.640+0000 7fa07f7fe700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 msgr2=0x7fa06803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:26.640+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:28.241+0000 7fa07f7fe700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 msgr2=0x7fa06803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1237634613 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:28.241+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:30.108+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7fa07001ab80 con 0x7fa080106920 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:30.108+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 msgr2=0x7fa06803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:30.108+0000 7fa07d7fa700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.109+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7fa07004eaa0 con 0x7fa080106920 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.109+0000 7fa07d7fa700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa06803c860 0x7fa06803ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.109+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa06803bb40 con 0x7fa06803c860 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.110+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa06803c860 0x7fa06803ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.110+0000 7fa07f7fe700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa06803c860 0x7fa06803ec50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa074003a10 tx=0x7fa0740092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.111+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa06803bb40 con 0x7fa06803c860 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.117+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa06c002630 con 0x7fa06803c860 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.117+0000 7fa07d7fa700 1 -- 172.21.15.7:0/3285729647 <== mgr.14120 v2:172.21.15.7:6800/501265234 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fa06c002630 con 0x7fa06803c860 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa06803c860 msgr2=0x7fa06803ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa06803c860 0x7fa06803ec50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa074003a10 tx=0x7fa0740092b0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 msgr2=0x7fa080191a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080191a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa070000c00 tx=0x7fa070004290 comp rx=0 tx=0).stop 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 shutdown_connections 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa06803c860 0x7fa06803ec50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080106920 0x7fa080191a90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 --2- 172.21.15.7:0/3285729647 >> [v2:172.21.15.7:6800/1237634613,v1:172.21.15.7:6801/1237634613] conn(0x7fa068038f70 0x7fa06803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 >> 172.21.15.7:0/3285729647 conn(0x7fa080101ee0 msgr2=0x7fa080102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 shutdown_connections 2024-03-19T00:28:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:31.118+0000 7fa0869e9700 1 -- 172.21.15.7:0/3285729647 wait complete. 2024-03-19T00:28:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-03-19T00:28:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-03-19T00:28:32.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:32 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:31] ENGINE Bus STARTING 2024-03-19T00:28:32.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:32 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:31] ENGINE Serving on http://172.21.15.7:8765 2024-03-19T00:28:32.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:32 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:31] ENGINE Serving on https://172.21.15.7:7150 2024-03-19T00:28:32.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:32 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:31] ENGINE Bus STARTED 2024-03-19T00:28:32.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:32 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:32.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d8939700 1 Processor -- start 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d8939700 1 -- start start 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d8939700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d410a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d8939700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02d407be50 con 0x7f02d4108570 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d410a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d410a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43890/0 (socket says 172.21.15.7:43890) 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.148+0000 7f02d259c700 1 -- 172.21.15.7:0/3453482576 learned_addr learned my addr 172.21.15.7:0/3453482576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d259c700 1 -- 172.21.15.7:0/3453482576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d410aea0 con 0x7f02d4108570 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d259c700 1 --2- 172.21.15.7:0/3453482576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d410a960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f02c40098a0 tx=0x7f02c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbba50489b272ae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:32.900 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d159a700 1 -- 172.21.15.7:0/3453482576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c4004160 con 0x7f02d4108570 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d159a700 1 -- 172.21.15.7:0/3453482576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f02c4004300 con 0x7f02d4108570 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d159a700 1 -- 172.21.15.7:0/3453482576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c4011a80 con 0x7f02d4108570 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d8939700 1 -- 172.21.15.7:0/3453482576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 msgr2=0x7f02d410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.149+0000 7f02d8939700 1 --2- 172.21.15.7:0/3453482576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d410a960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f02c40098a0 tx=0x7f02c4009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 -- 172.21.15.7:0/3453482576 shutdown_connections 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 --2- 172.21.15.7:0/3453482576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d410a960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 -- 172.21.15.7:0/3453482576 >> 172.21.15.7:0/3453482576 conn(0x7f02d4101f40 msgr2=0x7f02d4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 -- 172.21.15.7:0/3453482576 shutdown_connections 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 -- 172.21.15.7:0/3453482576 wait complete. 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 Processor -- start 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.150+0000 7f02d8939700 1 -- start start 2024-03-19T00:28:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d8939700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d8939700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02d410be10 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d419b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43892/0 (socket says 172.21.15.7:43892) 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d259c700 1 -- 172.21.15.7:0/4246348498 learned_addr learned my addr 172.21.15.7:0/4246348498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d259c700 1 -- 172.21.15.7:0/4246348498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d419bf30 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d259c700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d419b9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f02c4006af0 tx=0x7f02c4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c4003930 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f02c4003ad0 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d419c1c0 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c40195e0 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.151+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d419eeb0 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.152+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f02c40198b0 con 0x7f02d4108570 2024-03-19T00:28:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.152+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d4063940 con 0x7f02d4108570 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.152+0000 7f02c37fe700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f02bc038df0 0x7f02bc03b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.152+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f02c404d3b0 con 0x7f02d4108570 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.152+0000 7f02d1d9b700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f02bc038df0 0x7f02bc03b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.153+0000 7f02d1d9b700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f02bc038df0 0x7f02bc03b2b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02c8006fd0 tx=0x7f02c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.158+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f02c402d960 con 0x7f02d4108570 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.162+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f02c402d440 con 0x7f02d4108570 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.380+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f02d4103190 con 0x7f02bc038df0 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.386+0000 7f02c37fe700 1 -- 172.21.15.7:0/4246348498 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02d4103190 con 0x7f02bc038df0 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f02bc038df0 msgr2=0x7f02bc03b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f02bc038df0 0x7f02bc03b2b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02c8006fd0 tx=0x7f02c8006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 msgr2=0x7f02d419b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d419b9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f02c4006af0 tx=0x7f02c4011b70 comp rx=0 tx=0).stop 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 shutdown_connections 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f02bc038df0 0x7f02bc03b2b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:32.903 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 --2- 172.21.15.7:0/4246348498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4108570 0x7f02d419b9f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:32.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 >> 172.21.15.7:0/4246348498 conn(0x7f02d4101f40 msgr2=0x7f02d4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:32.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 shutdown_connections 2024-03-19T00:28:32.904 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:32.388+0000 7f02d8939700 1 -- 172.21.15.7:0/4246348498 wait complete. 2024-03-19T00:28:33.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:33 smithi007 ceph-mon[62854]: mgrmap e8: x(active, since 2s) 2024-03-19T00:28:33.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:33 smithi007 ceph-mon[62854]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:33.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:33 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:33.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:33 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.320+0000 7f86bc460700 1 Processor -- start 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.320+0000 7f86bc460700 1 -- start start 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86bc460700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b410a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86bc460700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86b407be50 con 0x7f86b41084f0 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86ba1fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b410a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86ba1fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b410a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43908/0 (socket says 172.21.15.7:43908) 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86ba1fc700 1 -- 172.21.15.7:0/1875736590 learned_addr learned my addr 172.21.15.7:0/1875736590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86ba1fc700 1 -- 172.21.15.7:0/1875736590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b410ae20 con 0x7f86b41084f0 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86ba1fc700 1 --2- 172.21.15.7:0/1875736590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b410a8e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0098a0 tx=0x7f86ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e584505d8d62d117 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.321+0000 7f86b91fa700 1 -- 172.21.15.7:0/1875736590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ac004160 con 0x7f86b41084f0 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86b91fa700 1 -- 172.21.15.7:0/1875736590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f86ac004300 con 0x7f86b41084f0 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86b91fa700 1 -- 172.21.15.7:0/1875736590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ac011a80 con 0x7f86b41084f0 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 -- 172.21.15.7:0/1875736590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 msgr2=0x7f86b410a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 --2- 172.21.15.7:0/1875736590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b410a8e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0098a0 tx=0x7f86ac009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 -- 172.21.15.7:0/1875736590 shutdown_connections 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 --2- 172.21.15.7:0/1875736590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b410a8e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 -- 172.21.15.7:0/1875736590 >> 172.21.15.7:0/1875736590 conn(0x7f86b4101ee0 msgr2=0x7f86b4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 -- 172.21.15.7:0/1875736590 shutdown_connections 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.322+0000 7f86bc460700 1 -- 172.21.15.7:0/1875736590 wait complete. 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86bc460700 1 Processor -- start 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86bc460700 1 -- start start 2024-03-19T00:28:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86bc460700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86bc460700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86b410bd90 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86ba1fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86ba1fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43914/0 (socket says 172.21.15.7:43914) 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86ba1fc700 1 -- 172.21.15.7:0/3997515973 learned_addr learned my addr 172.21.15.7:0/3997515973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.323+0000 7f86ba1fc700 1 -- 172.21.15.7:0/3997515973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b419bed0 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.324+0000 7f86ba1fc700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b419b990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86ac006af0 tx=0x7f86ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.324+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ac003930 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.324+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f86ac003ad0 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.324+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b419c160 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.324+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ac0195e0 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.324+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b419ee50 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.325+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f86ac019900 con 0x7f86b41084f0 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.325+0000 7f86b37fe700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f869c038ed0 0x7f869c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.325+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b4063940 con 0x7f86b41084f0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.325+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f86ac04d600 con 0x7f86b41084f0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.325+0000 7f86b99fb700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f869c038ed0 0x7f869c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.325+0000 7f86b99fb700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f869c038ed0 0x7f869c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86a8006fd0 tx=0x7f86a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.330+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86ac02d960 con 0x7f86b41084f0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.539+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f86b4103130 con 0x7f869c038ed0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.540+0000 7f86b37fe700 1 -- 172.21.15.7:0/3997515973 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f86b4103130 con 0x7f869c038ed0 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f869c038ed0 msgr2=0x7f869c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f869c038ed0 0x7f869c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86a8006fd0 tx=0x7f86a8006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 msgr2=0x7f86b419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b419b990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86ac006af0 tx=0x7f86ac011b70 comp rx=0 tx=0).stop 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 shutdown_connections 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f869c038ed0 0x7f869c03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 --2- 172.21.15.7:0/3997515973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b41084f0 0x7f86b419b990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:34.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 >> 172.21.15.7:0/3997515973 conn(0x7f86b4101ee0 msgr2=0x7f86b4102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:34.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 shutdown_connections 2024-03-19T00:28:34.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:33.542+0000 7f86bc460700 1 -- 172.21.15.7:0/3997515973 wait complete. 2024-03-19T00:28:34.060 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-03-19T00:28:34.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-mon[62854]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: Generating public/private rsa key pair. 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: Your identification has been saved in /tmp/tmp8m0cv2pv/key. 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: Your public key has been saved in /tmp/tmp8m0cv2pv/key.pub. 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: The key fingerprint is: 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: SHA256:hoh+YrmaJnfaynNvt6YMjuNWfm6qyq2KcuWR9qViDTs ceph-60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: The key's randomart image is: 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: +---[RSA 3072]----+ 2024-03-19T00:28:35.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | . . . | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | . ... S | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | . .B .. | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: | =*o* o | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: |==BBEoB o | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: |@XX@+X=+.. | 2024-03-19T00:28:35.216 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: +----[SHA256]-----+ 2024-03-19T00:28:35.504 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.476+0000 7f27e7b54700 1 Processor -- start 2024-03-19T00:28:35.504 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e7b54700 1 -- start start 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e7b54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e7b54700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0107830 con 0x7f27e0106e40 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e58f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e58f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43918/0 (socket says 172.21.15.7:43918) 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e58f0700 1 -- 172.21.15.7:0/2183002237 learned_addr learned my addr 172.21.15.7:0/2183002237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e58f0700 1 -- 172.21.15.7:0/2183002237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e0107b60 con 0x7f27e0106e40 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.477+0000 7f27e58f0700 1 --2- 172.21.15.7:0/2183002237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0107260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009e80 tx=0x7f27d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76efb4fa8ae84227 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e48ee700 1 -- 172.21.15.7:0/2183002237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0017070 con 0x7f27e0106e40 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e48ee700 1 -- 172.21.15.7:0/2183002237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f27d000b8c0 con 0x7f27e0106e40 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e48ee700 1 -- 172.21.15.7:0/2183002237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0004520 con 0x7f27e0106e40 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 -- 172.21.15.7:0/2183002237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 msgr2=0x7f27e0107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 --2- 172.21.15.7:0/2183002237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0107260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009e80 tx=0x7f27d000b330 comp rx=0 tx=0).stop 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 -- 172.21.15.7:0/2183002237 shutdown_connections 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 --2- 172.21.15.7:0/2183002237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0107260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 -- 172.21.15.7:0/2183002237 >> 172.21.15.7:0/2183002237 conn(0x7f27e0101660 msgr2=0x7f27e0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 -- 172.21.15.7:0/2183002237 shutdown_connections 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.478+0000 7f27e7b54700 1 -- 172.21.15.7:0/2183002237 wait complete. 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e7b54700 1 Processor -- start 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e7b54700 1 -- start start 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e7b54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0106420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e7b54700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27e010af10 con 0x7f27e0106e40 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e58f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0106420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e58f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0106420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43930/0 (socket says 172.21.15.7:43930) 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e58f0700 1 -- 172.21.15.7:0/2281462534 learned_addr learned my addr 172.21.15.7:0/2281462534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.479+0000 7f27e58f0700 1 -- 172.21.15.7:0/2281462534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e019e6f0 con 0x7f27e0106e40 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.480+0000 7f27e58f0700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0106420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27d000b7b0 tx=0x7f27d0004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.480+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0020020 con 0x7f27e0106e40 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.480+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f27d000bba0 con 0x7f27e0106e40 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.480+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e019e8c0 con 0x7f27e0106e40 2024-03-19T00:28:35.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.480+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0017040 con 0x7f27e0106e40 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.480+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e019ece0 con 0x7f27e0106e40 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.481+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f27d0023020 con 0x7f27e0106e40 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.481+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e0063940 con 0x7f27e0106e40 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.481+0000 7f27deffd700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f27c8038ed0 0x7f27c803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.481+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27d000f050 con 0x7f27e0106e40 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.481+0000 7f27e50ef700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f27c8038ed0 0x7f27c803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.481+0000 7f27e50ef700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f27c8038ed0 0x7f27c803b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f27d4006fd0 tx=0x7f27d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.486+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f27d002a020 con 0x7f27e0106e40 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.689+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f27e0102870 con 0x7f27c8038ed0 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.915+0000 7f27deffd700 1 -- 172.21.15.7:0/2281462534 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27e0102870 con 0x7f27c8038ed0 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f27c8038ed0 msgr2=0x7f27c803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f27c8038ed0 0x7f27c803b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f27d4006fd0 tx=0x7f27d4006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 msgr2=0x7f27e0106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0106420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27d000b7b0 tx=0x7f27d0004670 comp rx=0 tx=0).stop 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 shutdown_connections 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f27c8038ed0 0x7f27c803b390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 --2- 172.21.15.7:0/2281462534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e0106e40 0x7f27e0106420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 >> 172.21.15.7:0/2281462534 conn(0x7f27e0101660 msgr2=0x7f27e0102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 shutdown_connections 2024-03-19T00:28:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:34.917+0000 7f27e7b54700 1 -- 172.21.15.7:0/2281462534 wait complete. 2024-03-19T00:28:36.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:35 smithi007 ceph-mon[62854]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:36.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:35 smithi007 ceph-mon[62854]: Generating ssh key... 2024-03-19T00:28:36.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:35 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:36.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:35 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:36.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a73c70700 1 Processor -- start 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a73c70700 1 -- start start 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a73c70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a73c70700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c107330 con 0x7f0a6c106940 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a71a0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a71a0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60502/0 (socket says 172.21.15.7:60502) 2024-03-19T00:28:36.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a71a0c700 1 -- 172.21.15.7:0/464362991 learned_addr learned my addr 172.21.15.7:0/464362991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.897+0000 7f0a71a0c700 1 -- 172.21.15.7:0/464362991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c107b40 con 0x7f0a6c106940 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.898+0000 7f0a71a0c700 1 --2- 172.21.15.7:0/464362991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c106d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098a0 tx=0x7f0a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d911632ea54a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.898+0000 7f0a70a0a700 1 -- 172.21.15.7:0/464362991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68004160 con 0x7f0a6c106940 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.898+0000 7f0a70a0a700 1 -- 172.21.15.7:0/464362991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0a68004300 con 0x7f0a6c106940 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.898+0000 7f0a70a0a700 1 -- 172.21.15.7:0/464362991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a68011a80 con 0x7f0a6c106940 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.898+0000 7f0a73c70700 1 -- 172.21.15.7:0/464362991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 msgr2=0x7f0a6c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.898+0000 7f0a73c70700 1 --2- 172.21.15.7:0/464362991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c106d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098a0 tx=0x7f0a68009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 -- 172.21.15.7:0/464362991 shutdown_connections 2024-03-19T00:28:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 --2- 172.21.15.7:0/464362991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c106d60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 -- 172.21.15.7:0/464362991 >> 172.21.15.7:0/464362991 conn(0x7f0a6c101ee0 msgr2=0x7f0a6c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 -- 172.21.15.7:0/464362991 shutdown_connections 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 -- 172.21.15.7:0/464362991 wait complete. 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 Processor -- start 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 -- start start 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.899+0000 7f0a73c70700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c10aef0 con 0x7f0a6c106940 2024-03-19T00:28:36.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a71a0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a71a0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60512/0 (socket says 172.21.15.7:60512) 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a71a0c700 1 -- 172.21.15.7:0/2353479910 learned_addr learned my addr 172.21.15.7:0/2353479910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a71a0c700 1 -- 172.21.15.7:0/2353479910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c19bed0 con 0x7f0a6c106940 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a71a0c700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c19b990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a68006af0 tx=0x7f0a68011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a680039e0 con 0x7f0a6c106940 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0a68003b80 con 0x7f0a6c106940 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c19c160 con 0x7f0a6c106940 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a680196a0 con 0x7f0a6c106940 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.900+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6c19ee50 con 0x7f0a6c106940 2024-03-19T00:28:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.901+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f0a6801a430 con 0x7f0a6c106940 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.901+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6c063940 con 0x7f0a6c106940 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.901+0000 7f0a5effd700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f0a58038f20 0x7f0a5803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.901+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a6804d440 con 0x7f0a6c106940 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.901+0000 7f0a7120b700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f0a58038f20 0x7f0a5803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.902+0000 7f0a7120b700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f0a58038f20 0x7f0a5803b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:35.906+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0a6804cd90 con 0x7f0a6c106940 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.111+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f0a6c0008d0 con 0x7f0a58038f20 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.112+0000 7f0a5effd700 1 -- 172.21.15.7:0/2353479910 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f0a6c0008d0 con 0x7f0a58038f20 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f0a58038f20 msgr2=0x7f0a5803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f0a58038f20 0x7f0a5803b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 msgr2=0x7f0a6c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:36.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c19b990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a68006af0 tx=0x7f0a68011c40 comp rx=0 tx=0).stop 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 shutdown_connections 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f0a58038f20 0x7f0a5803b3e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 --2- 172.21.15.7:0/2353479910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a6c106940 0x7f0a6c19b990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 >> 172.21.15.7:0/2353479910 conn(0x7f0a6c101ee0 msgr2=0x7f0a6c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 shutdown_connections 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:36.114+0000 7f0a73c70700 1 -- 172.21.15.7:0/2353479910 wait complete. 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-03-19T00:28:36.700 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-03-19T00:28:37.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:36 smithi007 ceph-mon[62854]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:38.432 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:38 smithi007 ceph-mon[62854]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:38.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:38 smithi007 ceph-mon[62854]: Deploying cephadm binary to smithi007 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.129+0000 7fbc2a795700 1 Processor -- start 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc2a795700 1 -- start start 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc2a795700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc24107e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc2a795700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24108430 con 0x7fbc24107a40 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc24107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:42.334 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc24107e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60528/0 (socket says 172.21.15.7:60528) 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc23fff700 1 -- 172.21.15.7:0/3065380922 learned_addr learned my addr 172.21.15.7:0/3065380922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc23fff700 1 -- 172.21.15.7:0/3065380922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc24108c40 con 0x7fbc24107a40 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.130+0000 7fbc23fff700 1 --2- 172.21.15.7:0/3065380922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc24107e60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc140098a0 tx=0x7fbc14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2926222ae8828ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc22ffd700 1 -- 172.21.15.7:0/3065380922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc14004160 con 0x7fbc24107a40 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc22ffd700 1 -- 172.21.15.7:0/3065380922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbc14004300 con 0x7fbc24107a40 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc22ffd700 1 -- 172.21.15.7:0/3065380922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc14011a80 con 0x7fbc24107a40 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 -- 172.21.15.7:0/3065380922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 msgr2=0x7fbc24107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 --2- 172.21.15.7:0/3065380922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc24107e60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc140098a0 tx=0x7fbc14009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 -- 172.21.15.7:0/3065380922 shutdown_connections 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 --2- 172.21.15.7:0/3065380922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc24107e60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 -- 172.21.15.7:0/3065380922 >> 172.21.15.7:0/3065380922 conn(0x7fbc24103040 msgr2=0x7fbc24105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:42.335 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 -- 172.21.15.7:0/3065380922 shutdown_connections 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.131+0000 7fbc2a795700 1 -- 172.21.15.7:0/3065380922 wait complete. 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc2a795700 1 Processor -- start 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc2a795700 1 -- start start 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc2a795700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc2419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc2a795700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2410bff0 con 0x7fbc24107a40 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc2419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc2419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60536/0 (socket says 172.21.15.7:60536) 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.132+0000 7fbc23fff700 1 -- 172.21.15.7:0/276236975 learned_addr learned my addr 172.21.15.7:0/276236975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc23fff700 1 -- 172.21.15.7:0/276236975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2419bef0 con 0x7fbc24107a40 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc23fff700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc2419b9b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc14011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc140039f0 con 0x7fbc24107a40 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbc14003b90 con 0x7fbc24107a40 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2419c180 con 0x7fbc24107a40 2024-03-19T00:28:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1401a740 con 0x7fbc24107a40 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.133+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2419ee70 con 0x7fbc24107a40 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.134+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fbc1401a9f0 con 0x7fbc24107a40 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.134+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc24063940 con 0x7fbc24107a40 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.134+0000 7fbc217fa700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fbc0c038f70 0x7fbc0c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.134+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc1404d370 con 0x7fbc24107a40 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.134+0000 7fbc237fe700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fbc0c038f70 0x7fbc0c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.135+0000 7fbc237fe700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fbc0c038f70 0x7fbc0c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc18006fd0 tx=0x7fbc18006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.139+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc1401ad90 con 0x7fbc24107a40 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:37.350+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) -- 0x7fbc24104280 con 0x7fbc0c038f70 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.727+0000 7fbc217fa700 1 -- 172.21.15.7:0/276236975 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fbc24104280 con 0x7fbc0c038f70 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fbc0c038f70 msgr2=0x7fbc0c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fbc0c038f70 0x7fbc0c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc18006fd0 tx=0x7fbc18006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:42.337 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 msgr2=0x7fbc2419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:42.338 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc2419b9b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc14011b00 comp rx=0 tx=0).stop 2024-03-19T00:28:42.338 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 shutdown_connections 2024-03-19T00:28:42.338 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fbc0c038f70 0x7fbc0c03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:42.338 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 --2- 172.21.15.7:0/276236975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24107a40 0x7fbc2419b9b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:42.339 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 >> 172.21.15.7:0/276236975 conn(0x7fbc24103040 msgr2=0x7fbc24103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:42.339 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 shutdown_connections 2024-03-19T00:28:42.339 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:41.730+0000 7fbc2a795700 1 -- 172.21.15.7:0/276236975 wait complete. 2024-03-19T00:28:42.339 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-03-19T00:28:42.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:42 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:42 smithi007 ceph-mon[62854]: Added host smithi007 2024-03-19T00:28:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:42 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.766+0000 7fcc30e96700 1 Processor -- start 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.766+0000 7fcc30e96700 1 -- start start 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.766+0000 7fcc30e96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c10a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc30e96700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c07be70 con 0x7fcc2c1084e0 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c10a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c10a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60538/0 (socket says 172.21.15.7:60538) 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2a59c700 1 -- 172.21.15.7:0/2059754613 learned_addr learned my addr 172.21.15.7:0/2059754613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2a59c700 1 -- 172.21.15.7:0/2059754613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c10ae10 con 0x7fcc2c1084e0 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2a59c700 1 --2- 172.21.15.7:0/2059754613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c10a8d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009e80 tx=0x7fcc1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18001955fcfeaf21 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2959a700 1 -- 172.21.15.7:0/2059754613 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c017070 con 0x7fcc2c1084e0 2024-03-19T00:28:43.751 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2959a700 1 -- 172.21.15.7:0/2059754613 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcc1c00b8c0 con 0x7fcc2c1084e0 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.767+0000 7fcc2959a700 1 -- 172.21.15.7:0/2059754613 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c004520 con 0x7fcc2c1084e0 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 -- 172.21.15.7:0/2059754613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 msgr2=0x7fcc2c10a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 --2- 172.21.15.7:0/2059754613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c10a8d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009e80 tx=0x7fcc1c00b330 comp rx=0 tx=0).stop 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 -- 172.21.15.7:0/2059754613 shutdown_connections 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 --2- 172.21.15.7:0/2059754613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c10a8d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 -- 172.21.15.7:0/2059754613 >> 172.21.15.7:0/2059754613 conn(0x7fcc2c101ef0 msgr2=0x7fcc2c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 -- 172.21.15.7:0/2059754613 shutdown_connections 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.768+0000 7fcc30e96700 1 -- 172.21.15.7:0/2059754613 wait complete. 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc30e96700 1 Processor -- start 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc30e96700 1 -- start start 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc30e96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c107d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc30e96700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c10bdd0 con 0x7fcc2c1084e0 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc2a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc2a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60548/0 (socket says 172.21.15.7:60548) 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc2a59c700 1 -- 172.21.15.7:0/3574467524 learned_addr learned my addr 172.21.15.7:0/3574467524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.769+0000 7fcc2a59c700 1 -- 172.21.15.7:0/3574467524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c1082a0 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.770+0000 7fcc2a59c700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c107d60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b7b0 tx=0x7fcc1c004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.770+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c020020 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.770+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c106490 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.770+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcc1c00bba0 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.770+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c017040 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.770+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1069d0 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.771+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fcc1c023020 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.771+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2c063940 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.771+0000 7fcc237fe700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fcc18038ed0 0x7fcc1803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.771+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc1c00f050 con 0x7fcc2c1084e0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.771+0000 7fcc29d9b700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fcc18038ed0 0x7fcc1803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.771+0000 7fcc29d9b700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fcc18038ed0 0x7fcc1803b390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:43.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:42.776+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcc1c02a020 con 0x7fcc2c1084e0 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.016+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcc2c107230 con 0x7fcc18038ed0 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.020+0000 7fcc237fe700 1 -- 172.21.15.7:0/3574467524 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcc2c107230 con 0x7fcc18038ed0 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fcc18038ed0 msgr2=0x7fcc1803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fcc18038ed0 0x7fcc1803b390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 msgr2=0x7fcc2c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c107d60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b7b0 tx=0x7fcc1c004670 comp rx=0 tx=0).stop 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 shutdown_connections 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fcc18038ed0 0x7fcc1803b390 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.022+0000 7fcc30e96700 1 --2- 172.21.15.7:0/3574467524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc2c1084e0 0x7fcc2c107d60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.023+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 >> 172.21.15.7:0/3574467524 conn(0x7fcc2c101ef0 msgr2=0x7fcc2c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.023+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 shutdown_connections 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:43.023+0000 7fcc30e96700 1 -- 172.21.15.7:0/3574467524 wait complete. 2024-03-19T00:28:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-03-19T00:28:44.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:44 smithi007 ceph-mon[62854]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:44.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:44 smithi007 ceph-mon[62854]: Saving service mon spec with placement count:5 2024-03-19T00:28:44.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:44 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:45.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-19T00:28:45.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f7985a700 1 Processor -- start 2024-03-19T00:28:45.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f7985a700 1 -- start start 2024-03-19T00:28:45.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f7985a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f74107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f7985a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74108450 con 0x7f7f74107a60 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f74107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f74107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60562/0 (socket says 172.21.15.7:60562) 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.199+0000 7f7f72ffd700 1 -- 172.21.15.7:0/1351034397 learned_addr learned my addr 172.21.15.7:0/1351034397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.200+0000 7f7f72ffd700 1 -- 172.21.15.7:0/1351034397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f74108c60 con 0x7f7f74107a60 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.200+0000 7f7f72ffd700 1 --2- 172.21.15.7:0/1351034397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f74107e80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7f640098a0 tx=0x7f7f64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde5401db416b2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.200+0000 7f7f727fc700 1 -- 172.21.15.7:0/1351034397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f64004160 con 0x7f7f74107a60 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.200+0000 7f7f727fc700 1 -- 172.21.15.7:0/1351034397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7f64004300 con 0x7f7f74107a60 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.200+0000 7f7f727fc700 1 -- 172.21.15.7:0/1351034397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f64011a80 con 0x7f7f74107a60 2024-03-19T00:28:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 -- 172.21.15.7:0/1351034397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 msgr2=0x7f7f74107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 --2- 172.21.15.7:0/1351034397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f74107e80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7f640098a0 tx=0x7f7f64009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 -- 172.21.15.7:0/1351034397 shutdown_connections 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 --2- 172.21.15.7:0/1351034397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f74107e80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 -- 172.21.15.7:0/1351034397 >> 172.21.15.7:0/1351034397 conn(0x7f7f74103080 msgr2=0x7f7f741054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 -- 172.21.15.7:0/1351034397 shutdown_connections 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 -- 172.21.15.7:0/1351034397 wait complete. 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 Processor -- start 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 -- start start 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.201+0000 7f7f7985a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f7419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f7985a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7410c010 con 0x7f7f74107a60 2024-03-19T00:28:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f7419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f7419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60570/0 (socket says 172.21.15.7:60570) 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f72ffd700 1 -- 172.21.15.7:0/2452216859 learned_addr learned my addr 172.21.15.7:0/2452216859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f72ffd700 1 -- 172.21.15.7:0/2452216859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7419bef0 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f72ffd700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f7419b9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006af0 tx=0x7f7f64011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f640039e0 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7f64003b80 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f640196a0 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7419c180 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.202+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7419ee70 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.203+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f7f640199a0 con 0x7f7f74107a60 2024-03-19T00:28:45.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.203+0000 7f7f78858700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f7f60038ed0 0x7f7f6003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.203+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f74063940 con 0x7f7f74107a60 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.203+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f6404d3b0 con 0x7f7f74107a60 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.204+0000 7f7f6bfff700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f7f60038ed0 0x7f7f6003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.204+0000 7f7f6bfff700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f7f60038ed0 0x7f7f6003b390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006fd0 tx=0x7f7f5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.209+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f6404a400 con 0x7f7f74107a60 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.430+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7f74104260 con 0x7f7f60038ed0 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.435+0000 7f7f78858700 1 -- 172.21.15.7:0/2452216859 <== mgr.14120 v2:172.21.15.7:6800/501265234 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7f74104260 con 0x7f7f60038ed0 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f7f60038ed0 msgr2=0x7f7f6003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f7f60038ed0 0x7f7f6003b390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006fd0 tx=0x7f7f5c006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 msgr2=0x7f7f7419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f7419b9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006af0 tx=0x7f7f64011c40 comp rx=0 tx=0).stop 2024-03-19T00:28:45.013 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 shutdown_connections 2024-03-19T00:28:45.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f7f60038ed0 0x7f7f6003b390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:45.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 --2- 172.21.15.7:0/2452216859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f74107a60 0x7f7f7419b9b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:45.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 >> 172.21.15.7:0/2452216859 conn(0x7f7f74103080 msgr2=0x7f7f74103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:45.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 shutdown_connections 2024-03-19T00:28:45.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:44.437+0000 7f7f7985a700 1 -- 172.21.15.7:0/2452216859 wait complete. 2024-03-19T00:28:45.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:45 smithi007 ceph-mon[62854]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:45.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:45 smithi007 ceph-mon[62854]: Saving service mgr spec with placement count:2 2024-03-19T00:28:45.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:45 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.543+0000 7fa3af274700 1 Processor -- start 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.543+0000 7fa3af274700 1 -- start start 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.543+0000 7fa3af274700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a8107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3af274700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8108470 con 0x7fa3a8107a80 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3ad010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a8107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3ad010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a8107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47416/0 (socket says 172.21.15.7:47416) 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3ad010700 1 -- 172.21.15.7:0/487288798 learned_addr learned my addr 172.21.15.7:0/487288798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3ad010700 1 -- 172.21.15.7:0/487288798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a8108c80 con 0x7fa3a8107a80 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3ad010700 1 --2- 172.21.15.7:0/487288798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a8107ea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009e80 tx=0x7fa3a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152f284cc14164a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3a7fff700 1 -- 172.21.15.7:0/487288798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0017070 con 0x7fa3a8107a80 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3a7fff700 1 -- 172.21.15.7:0/487288798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa3a000b8c0 con 0x7fa3a8107a80 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.544+0000 7fa3a7fff700 1 -- 172.21.15.7:0/487288798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0004520 con 0x7fa3a8107a80 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 -- 172.21.15.7:0/487288798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 msgr2=0x7fa3a8107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 --2- 172.21.15.7:0/487288798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a8107ea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009e80 tx=0x7fa3a000b330 comp rx=0 tx=0).stop 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 -- 172.21.15.7:0/487288798 shutdown_connections 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 --2- 172.21.15.7:0/487288798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a8107ea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 -- 172.21.15.7:0/487288798 >> 172.21.15.7:0/487288798 conn(0x7fa3a8103040 msgr2=0x7fa3a8105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 -- 172.21.15.7:0/487288798 shutdown_connections 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.545+0000 7fa3af274700 1 -- 172.21.15.7:0/487288798 wait complete. 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3af274700 1 Processor -- start 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3af274700 1 -- start start 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3af274700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3af274700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810c030 con 0x7fa3a8107a80 2024-03-19T00:28:46.740 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3ad010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3ad010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47428/0 (socket says 172.21.15.7:47428) 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3ad010700 1 -- 172.21.15.7:0/1366542537 learned_addr learned my addr 172.21.15.7:0/1366542537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3ad010700 1 -- 172.21.15.7:0/1366542537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a807e0b0 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3ad010700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a807db70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b7b0 tx=0x7fa3a0004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0020020 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa3a000bba0 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a807c2a0 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.546+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0017040 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.547+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a807c7e0 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.547+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fa3a0023020 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.547+0000 7fa3a67fc700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa390038ed0 0x7fa39003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.547+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa3a000f050 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.548+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a8063940 con 0x7fa3a8107a80 2024-03-19T00:28:46.741 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.548+0000 7fa3ac80f700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa390038ed0 0x7fa39003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.548+0000 7fa3ac80f700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa390038ed0 0x7fa39003b390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa39c006fd0 tx=0x7fa39c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.553+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa3a002a020 con 0x7fa3a8107a80 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.754+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa3a807d060 con 0x7fa3a8107a80 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.757+0000 7fa3a67fc700 1 -- 172.21.15.7:0/1366542537 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fa3a0007310 con 0x7fa3a8107a80 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa390038ed0 msgr2=0x7fa39003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa390038ed0 0x7fa39003b390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa39c006fd0 tx=0x7fa39c006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 msgr2=0x7fa3a807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a807db70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b7b0 tx=0x7fa3a0004670 comp rx=0 tx=0).stop 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 shutdown_connections 2024-03-19T00:28:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fa390038ed0 0x7fa39003b390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:46.743 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 --2- 172.21.15.7:0/1366542537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a8107a80 0x7fa3a807db70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:46.743 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 >> 172.21.15.7:0/1366542537 conn(0x7fa3a8103040 msgr2=0x7fa3a8103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:46.743 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 shutdown_connections 2024-03-19T00:28:46.743 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:45.759+0000 7fa3af274700 1 -- 172.21.15.7:0/1366542537 wait complete. 2024-03-19T00:28:47.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:46 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1366542537' entity='client.admin' 2024-03-19T00:28:47.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:46 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:47.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:47 smithi007 ceph-mon[62854]: from='mgr.14120 172.21.15.7:0/2214057252' entity='mgr.x' 2024-03-19T00:28:47.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:47 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4067931284' entity='client.admin' 2024-03-19T00:28:47.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.179+0000 7fba21e81700 1 Processor -- start 2024-03-19T00:28:47.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.179+0000 7fba21e81700 1 -- start start 2024-03-19T00:28:47.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.179+0000 7fba21e81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c10a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.179+0000 7fba21e81700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c07be70 con 0x7fba1c1084f0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.180+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c10a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.180+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c10a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47432/0 (socket says 172.21.15.7:47432) 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.180+0000 7fba1b7fe700 1 -- 172.21.15.7:0/3935177085 learned_addr learned my addr 172.21.15.7:0/3935177085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.180+0000 7fba1b7fe700 1 -- 172.21.15.7:0/3935177085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c10ae20 con 0x7fba1c1084f0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.180+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/3935177085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c10a8e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba100098a0 tx=0x7fba10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b471e2da712425 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.180+0000 7fba1a7fc700 1 -- 172.21.15.7:0/3935177085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba10004160 con 0x7fba1c1084f0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba1a7fc700 1 -- 172.21.15.7:0/3935177085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba10004300 con 0x7fba1c1084f0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba1a7fc700 1 -- 172.21.15.7:0/3935177085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba10011a50 con 0x7fba1c1084f0 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 -- 172.21.15.7:0/3935177085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 msgr2=0x7fba1c10a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 --2- 172.21.15.7:0/3935177085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c10a8e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba100098a0 tx=0x7fba10009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 -- 172.21.15.7:0/3935177085 shutdown_connections 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 --2- 172.21.15.7:0/3935177085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c10a8e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 -- 172.21.15.7:0/3935177085 >> 172.21.15.7:0/3935177085 conn(0x7fba1c101ee0 msgr2=0x7fba1c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 -- 172.21.15.7:0/3935177085 shutdown_connections 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.181+0000 7fba21e81700 1 -- 172.21.15.7:0/3935177085 wait complete. 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba21e81700 1 Processor -- start 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba21e81700 1 -- start start 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba21e81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c107d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba21e81700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c10be20 con 0x7fba1c1084f0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c107d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c107d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47440/0 (socket says 172.21.15.7:47440) 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.182+0000 7fba1b7fe700 1 -- 172.21.15.7:0/4067931284 learned_addr learned my addr 172.21.15.7:0/4067931284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba1b7fe700 1 -- 172.21.15.7:0/4067931284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c1082b0 con 0x7fba1c1084f0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c107d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fba10011a20 tx=0x7fba10011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba100039c0 con 0x7fba1c1084f0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c1064a0 con 0x7fba1c1084f0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1069e0 con 0x7fba1c1084f0 2024-03-19T00:28:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba10003b60 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.183+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba1001a6b0 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.184+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fba1001a9e0 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.184+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba08005360 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.184+0000 7fba18ff9700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fba04038f20 0x7fba0403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.184+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba1004d420 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.184+0000 7fba1affd700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fba04038f20 0x7fba0403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.185+0000 7fba1affd700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fba04038f20 0x7fba0403b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba0c006fd0 tx=0x7fba0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.189+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba10024a70 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.414+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fba08005650 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.417+0000 7fba18ff9700 1 -- 172.21.15.7:0/4067931284 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fba1002d910 con 0x7fba1c1084f0 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fba04038f20 msgr2=0x7fba0403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fba04038f20 0x7fba0403b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba0c006fd0 tx=0x7fba0c006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 msgr2=0x7fba1c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:47.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c107d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fba10011a20 tx=0x7fba10011b00 comp rx=0 tx=0).stop 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 shutdown_connections 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7fba04038f20 0x7fba0403b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 --2- 172.21.15.7:0/4067931284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c1084f0 0x7fba1c107d70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 >> 172.21.15.7:0/4067931284 conn(0x7fba1c101ee0 msgr2=0x7fba1c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 shutdown_connections 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:47.420+0000 7fba21e81700 1 -- 172.21.15.7:0/4067931284 wait complete. 2024-03-19T00:28:47.950 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-03-19T00:28:48.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:48 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1956588948' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-19T00:28:49.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:48 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: ignoring --setuser ceph since I am not root 2024-03-19T00:28:49.164 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:48 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: ignoring --setgroup ceph since I am not root 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.423+0000 7f15322d3700 1 Processor -- start 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.423+0000 7f15322d3700 1 -- start start 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.423+0000 7f15322d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.423+0000 7f15322d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f152c07c8d0 con 0x7f152c07dea0 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47444/0 (socket says 172.21.15.7:47444) 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152bfff700 1 -- 172.21.15.7:0/2731239187 learned_addr learned my addr 172.21.15.7:0/2731239187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:49.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152bfff700 1 -- 172.21.15.7:0/2731239187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c07ca50 con 0x7f152c07dea0 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152bfff700 1 --2- 172.21.15.7:0/2731239187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c07c300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f151c0098a0 tx=0x7f151c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7422e5bb703690 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152affd700 1 -- 172.21.15.7:0/2731239187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c004160 con 0x7f152c07dea0 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152affd700 1 -- 172.21.15.7:0/2731239187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c004300 con 0x7f152c07dea0 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.424+0000 7f152affd700 1 -- 172.21.15.7:0/2731239187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c011a50 con 0x7f152c07dea0 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 -- 172.21.15.7:0/2731239187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 msgr2=0x7f152c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 --2- 172.21.15.7:0/2731239187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c07c300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f151c0098a0 tx=0x7f151c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 -- 172.21.15.7:0/2731239187 shutdown_connections 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 --2- 172.21.15.7:0/2731239187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c07c300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 -- 172.21.15.7:0/2731239187 >> 172.21.15.7:0/2731239187 conn(0x7f152c102ff0 msgr2=0x7f152c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 -- 172.21.15.7:0/2731239187 shutdown_connections 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.425+0000 7f15322d3700 1 -- 172.21.15.7:0/2731239187 wait complete. 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f15322d3700 1 Processor -- start 2024-03-19T00:28:49.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f15322d3700 1 -- start start 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f15322d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f15322d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f152c10e070 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f152bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f152bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c19dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47456/0 (socket says 172.21.15.7:47456) 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.426+0000 7f152bfff700 1 -- 172.21.15.7:0/1956588948 learned_addr learned my addr 172.21.15.7:0/1956588948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f152bfff700 1 -- 172.21.15.7:0/1956588948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c19e110 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f152bfff700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c19dbd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f151c011710 tx=0x7f151c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c0039c0 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c003b60 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c19e3a0 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f151c01a690 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.427+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c1a1090 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.428+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f151c01a990 con 0x7f152c07dea0 2024-03-19T00:28:49.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.428+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152c063940 con 0x7f152c07dea0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.428+0000 7f15297fa700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f1518038ed0 0x7f151803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.428+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f151c04d740 con 0x7f152c07dea0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.428+0000 7f152b7fe700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f1518038ed0 0x7f151803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.429+0000 7f152b7fe700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f1518038ed0 0x7f151803b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1514006fd0 tx=0x7f1514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.433+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f151c019c70 con 0x7f152c07dea0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.697+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f152c1a1640 con 0x7f152c07dea0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.852+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f151c00f2e0 con 0x7f152c07dea0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.854+0000 7f15297fa700 1 -- 172.21.15.7:0/1956588948 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f151c02d910 con 0x7f152c07dea0 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f1518038ed0 msgr2=0x7f151803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f1518038ed0 0x7f151803b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1514006fd0 tx=0x7f1514006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 msgr2=0x7f152c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c19dbd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f151c011710 tx=0x7f151c011ad0 comp rx=0 tx=0).stop 2024-03-19T00:28:49.677 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 shutdown_connections 2024-03-19T00:28:49.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f1518038ed0 0x7f151803b390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:49.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 --2- 172.21.15.7:0/1956588948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152c07dea0 0x7f152c19dbd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:49.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.855+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 >> 172.21.15.7:0/1956588948 conn(0x7f152c102ff0 msgr2=0x7f152c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:49.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.856+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 shutdown_connections 2024-03-19T00:28:49.678 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:48.856+0000 7f15322d3700 1 -- 172.21.15.7:0/1956588948 wait complete. 2024-03-19T00:28:49.714 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:49 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:49.441+0000 7fc9059a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:28:49.714 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:49 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:49.537+0000 7fc9059a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:28:50.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:49 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1956588948' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-19T00:28:50.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:49 smithi007 ceph-mon[62854]: mgrmap e9: x(active, since 18s) 2024-03-19T00:28:50.382 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:50.209+0000 7fc9059a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:28:50.382 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:50.304+0000 7fc9059a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:28:50.636 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:50.451+0000 7fc9059a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:28:50.636 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:50.548+0000 7fc9059a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:28:50.636 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:50.634+0000 7fc9059a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:28:50.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86d53cb700 1 Processor -- start 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86d53cb700 1 -- start start 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86d53cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d0107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86d53cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0108450 con 0x7f86d0107a60 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d0107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d0107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47484/0 (socket says 172.21.15.7:47484) 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86ceffd700 1 -- 172.21.15.7:0/3117776263 learned_addr learned my addr 172.21.15.7:0/3117776263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.102+0000 7f86ceffd700 1 -- 172.21.15.7:0/3117776263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d0108c60 con 0x7f86d0107a60 2024-03-19T00:28:50.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86ceffd700 1 --2- 172.21.15.7:0/3117776263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d0107e80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86c00098a0 tx=0x7f86c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20247f7d77ca7566 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86cdffb700 1 -- 172.21.15.7:0/3117776263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c0004160 con 0x7f86d0107a60 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86cdffb700 1 -- 172.21.15.7:0/3117776263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86c0004300 con 0x7f86d0107a60 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86cdffb700 1 -- 172.21.15.7:0/3117776263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c0011a50 con 0x7f86d0107a60 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86d53cb700 1 -- 172.21.15.7:0/3117776263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 msgr2=0x7f86d0107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86d53cb700 1 --2- 172.21.15.7:0/3117776263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d0107e80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86c00098a0 tx=0x7f86c0009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86d53cb700 1 -- 172.21.15.7:0/3117776263 shutdown_connections 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86d53cb700 1 --2- 172.21.15.7:0/3117776263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d0107e80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86d53cb700 1 -- 172.21.15.7:0/3117776263 >> 172.21.15.7:0/3117776263 conn(0x7f86d0103040 msgr2=0x7f86d0105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.103+0000 7f86d53cb700 1 -- 172.21.15.7:0/3117776263 shutdown_connections 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.104+0000 7f86d53cb700 1 -- 172.21.15.7:0/3117776263 wait complete. 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.104+0000 7f86d53cb700 1 Processor -- start 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.104+0000 7f86d53cb700 1 -- start start 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.104+0000 7f86d53cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:50.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.104+0000 7f86d53cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010c010 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.104+0000 7f86ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47490/0 (socket says 172.21.15.7:47490) 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86ceffd700 1 -- 172.21.15.7:0/2252721990 learned_addr learned my addr 172.21.15.7:0/2252721990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86ceffd700 1 -- 172.21.15.7:0/2252721990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d007e0b0 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86ceffd700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d007db70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86c0006b20 tx=0x7f86c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c00039e0 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86c001a8e0 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d007c2a0 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c00247e0 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.105+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d007c7e0 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.106+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f86c0022020 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.106+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d0063940 con 0x7f86d0107a60 2024-03-19T00:28:50.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.106+0000 7f86b7fff700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 0x7f86b803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.106+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f86c004d2d0 con 0x7f86d0107a60 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.106+0000 7f86ce7fc700 1 -- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 msgr2=0x7f86b803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.106+0000 7f86ce7fc700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 0x7f86b803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.115+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86c00172b0 con 0x7f86d0107a60 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.307+0000 7f86ce7fc700 1 -- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 msgr2=0x7f86b803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.307+0000 7f86ce7fc700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 0x7f86b803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.380+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f86d007d080 con 0x7f86d0107a60 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.380+0000 7f86b7fff700 1 -- 172.21.15.7:0/2252721990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f86c001aa90 con 0x7f86d0107a60 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 msgr2=0x7f86b803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 0x7f86b803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 msgr2=0x7f86d007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d007db70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86c0006b20 tx=0x7f86c0011ad0 comp rx=0 tx=0).stop 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 shutdown_connections 2024-03-19T00:28:50.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7f86b8038f70 0x7f86b803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:50.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 --2- 172.21.15.7:0/2252721990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0107a60 0x7f86d007db70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:50.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 >> 172.21.15.7:0/2252721990 conn(0x7f86d0103040 msgr2=0x7f86d0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:50.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 shutdown_connections 2024-03-19T00:28:50.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:50.383+0000 7f86d53cb700 1 -- 172.21.15.7:0/2252721990 wait complete. 2024-03-19T00:28:50.869 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-19T00:28:50.869 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-03-19T00:28:50.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2252721990' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-19T00:28:51.213 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:50.910+0000 7fc9059a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:28:51.213 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:51 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:51.023+0000 7fc9059a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:28:51.213 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:51 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:51.111+0000 7fc9059a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:28:51.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:51 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:51.211+0000 7fc9059a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:28:51.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:51 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:51.394+0000 7fc9059a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:28:52.714 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:52 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:52.380+0000 7fc9059a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:28:52.715 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:52 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:52.515+0000 7fc9059a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:28:53.161 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:52 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:52.862+0000 7fc9059a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:28:53.161 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:52 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:52.955+0000 7fc9059a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:28:53.161 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:53.159+0000 7fc9059a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:28:53.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:53.350+0000 7fc9059a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:28:53.909 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:53.647+0000 7fc9059a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:28:53.910 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:53.907+0000 7fc9059a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:28:54.214 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:54 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:54.152+0000 7fc9059a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:28:55.464 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:54 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:54.978+0000 7fc9059a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:28:55.903 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:55.575+0000 7fc9059a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:28:55.903 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:55.676+0000 7fc9059a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:28:55.903 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:28:55.901+0000 7fc9059a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: Active manager daemon x restarted 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: Activating manager daemon x 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: osdmap e3: 0 total, 0 up, 0 in 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: mgrmap e10: x(active, starting, since 0.00326882s) 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: Manager daemon x is now available 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:28:57.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T00:28:57.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T00:28:57.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:56 smithi007 ceph-mon[62854]: mgrmap e11: x(active, since 1.00581s) 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e91cc700 1 Processor -- start 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e91cc700 1 -- start start 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e91cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e4106120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e91cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41066f0 con 0x7ff1e4105d00 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e4106120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e4106120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47496/0 (socket says 172.21.15.7:47496) 2024-03-19T00:28:57.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.318+0000 7ff1e2d9d700 1 -- 172.21.15.7:0/1514291932 learned_addr learned my addr 172.21.15.7:0/1514291932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.319+0000 7ff1e2d9d700 1 -- 172.21.15.7:0/1514291932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e4106870 con 0x7ff1e4105d00 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.319+0000 7ff1e2d9d700 1 --2- 172.21.15.7:0/1514291932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e4106120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80098a0 tx=0x7ff1d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8151792f9292a66c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.319+0000 7ff1e1d9b700 1 -- 172.21.15.7:0/1514291932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d8004160 con 0x7ff1e4105d00 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.319+0000 7ff1e1d9b700 1 -- 172.21.15.7:0/1514291932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d8004300 con 0x7ff1e4105d00 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.319+0000 7ff1e1d9b700 1 -- 172.21.15.7:0/1514291932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d8011a50 con 0x7ff1e4105d00 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 -- 172.21.15.7:0/1514291932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 msgr2=0x7ff1e4106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/1514291932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e4106120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80098a0 tx=0x7ff1d8009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 -- 172.21.15.7:0/1514291932 shutdown_connections 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/1514291932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e4106120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 -- 172.21.15.7:0/1514291932 >> 172.21.15.7:0/1514291932 conn(0x7ff1e4101660 msgr2=0x7ff1e4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 -- 172.21.15.7:0/1514291932 shutdown_connections 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.320+0000 7ff1e91cc700 1 -- 172.21.15.7:0/1514291932 wait complete. 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e91cc700 1 Processor -- start 2024-03-19T00:28:57.768 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e91cc700 1 -- start start 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e91cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e91cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410bfb0 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e419b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47512/0 (socket says 172.21.15.7:47512) 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e2d9d700 1 -- 172.21.15.7:0/3575536020 learned_addr learned my addr 172.21.15.7:0/3575536020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.321+0000 7ff1e2d9d700 1 -- 172.21.15.7:0/3575536020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e419bf30 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.322+0000 7ff1e2d9d700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e419b9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8011a20 tx=0x7ff1d8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.322+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d8003af0 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.322+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d801a8e0 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.322+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e419c1c0 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.322+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d8024860 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.322+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e419eeb0 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.323+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7ff1d8022020 con 0x7ff1e4105d00 2024-03-19T00:28:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.323+0000 7ff1cbfff700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.323+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 --> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1e4063940 con 0x7ff1cc038fd0 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.323+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff1d804e0b0 con 0x7ff1e4105d00 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.323+0000 7ff1e259c700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 msgr2=0x7ff1cc03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.323+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.524+0000 7ff1e259c700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 msgr2=0x7ff1cc03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.524+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.925+0000 7ff1e259c700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 msgr2=0x7ff1cc03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:51.925+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:52.726+0000 7ff1e259c700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 msgr2=0x7ff1cc03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:52.726+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:54.327+0000 7ff1e259c700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 msgr2=0x7ff1cc03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/501265234 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:54.327+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T00:28:57.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:55.917+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7ff1d802dc90 con 0x7ff1e4105d00 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:55.917+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 msgr2=0x7ff1cc03b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:55.917+0000 7ff1cbfff700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.919+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7ff1d804ef60 con 0x7ff1e4105d00 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.919+0000 7ff1cbfff700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1cc03c890 0x7ff1cc03ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.920+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1d8022e50 con 0x7ff1cc03c890 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.920+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1cc03c890 0x7ff1cc03ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.921+0000 7ff1e259c700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1cc03c890 0x7ff1cc03ec80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4003a10 tx=0x7ff1d4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.922+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff1d8022e50 con 0x7ff1cc03c890 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.934+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff1e4102870 con 0x7ff1cc03c890 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.934+0000 7ff1cbfff700 1 -- 172.21.15.7:0/3575536020 <== mgr.14152 v2:172.21.15.7:6800/1790452807 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff1e4102870 con 0x7ff1cc03c890 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1cc03c890 msgr2=0x7ff1cc03ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1cc03c890 0x7ff1cc03ec80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4003a10 tx=0x7ff1d4009270 comp rx=0 tx=0).stop 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 msgr2=0x7ff1e419b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e419b9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8011a20 tx=0x7ff1d8011b00 comp rx=0 tx=0).stop 2024-03-19T00:28:57.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 shutdown_connections 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1cc03c890 0x7ff1cc03ec80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e4105d00 0x7ff1e419b9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 --2- 172.21.15.7:0/3575536020 >> [v2:172.21.15.7:6800/501265234,v1:172.21.15.7:6801/501265234] conn(0x7ff1cc038fd0 0x7ff1cc03b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 >> 172.21.15.7:0/3575536020 conn(0x7ff1e4101660 msgr2=0x7ff1e4102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 shutdown_connections 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:56.935+0000 7ff1e91cc700 1 -- 172.21.15.7:0/3575536020 wait complete. 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-03-19T00:28:57.772 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-03-19T00:28:58.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:28:58.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:58 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:57] ENGINE Bus STARTING 2024-03-19T00:28:58.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:58 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:57] ENGINE Serving on http://172.21.15.7:8765 2024-03-19T00:28:58.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:58 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:57] ENGINE Serving on https://172.21.15.7:7150 2024-03-19T00:28:58.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:58 smithi007 ceph-mon[62854]: [19/Mar/2024:00:28:57] ENGINE Bus STARTED 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.171+0000 7f68cbfda700 1 Processor -- start 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.171+0000 7f68cbfda700 1 -- start start 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.171+0000 7f68cbfda700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c410a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.171+0000 7f68cbfda700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68c407be90 con 0x7f68c41084b0 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c9d76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c410a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c9d76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c410a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42040/0 (socket says 172.21.15.7:42040) 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c9d76700 1 -- 172.21.15.7:0/4153368295 learned_addr learned my addr 172.21.15.7:0/4153368295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c9d76700 1 -- 172.21.15.7:0/4153368295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c410ade0 con 0x7f68c41084b0 2024-03-19T00:28:59.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c9d76700 1 --2- 172.21.15.7:0/4153368295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c410a8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f68c00098a0 tx=0x7f68c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac84a972c11150bf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c8d74700 1 -- 172.21.15.7:0/4153368295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68c0004160 con 0x7f68c41084b0 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c8d74700 1 -- 172.21.15.7:0/4153368295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68c0004300 con 0x7f68c41084b0 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.172+0000 7f68c8d74700 1 -- 172.21.15.7:0/4153368295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68c0011a50 con 0x7f68c41084b0 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 -- 172.21.15.7:0/4153368295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 msgr2=0x7f68c410a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 --2- 172.21.15.7:0/4153368295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c410a8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f68c00098a0 tx=0x7f68c0009bb0 comp rx=0 tx=0).stop 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 -- 172.21.15.7:0/4153368295 shutdown_connections 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 --2- 172.21.15.7:0/4153368295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c410a8a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 -- 172.21.15.7:0/4153368295 >> 172.21.15.7:0/4153368295 conn(0x7f68c4101ee0 msgr2=0x7f68c4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 -- 172.21.15.7:0/4153368295 shutdown_connections 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.173+0000 7f68cbfda700 1 -- 172.21.15.7:0/4153368295 wait complete. 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68cbfda700 1 Processor -- start 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68cbfda700 1 -- start start 2024-03-19T00:28:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68cbfda700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c4107d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68cbfda700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410bd80 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68c9d76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c4107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68c9d76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c4107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42044/0 (socket says 172.21.15.7:42044) 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.174+0000 7f68c9d76700 1 -- 172.21.15.7:0/3063282922 learned_addr learned my addr 172.21.15.7:0/3063282922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68c9d76700 1 -- 172.21.15.7:0/3063282922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c4108270 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68c9d76700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c4107d30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68c0011a00 tx=0x7f68c0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68c0003af0 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c4106460 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68c001a8e0 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68c0024840 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.175+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c41069a0 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.176+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f68c0022020 con 0x7f68c41084b0 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.176+0000 7f68b6ffd700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68b0038e50 0x7f68b003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:28:59.066 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.176+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c4063940 con 0x7f68c41084b0 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.176+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f68c004d0e0 con 0x7f68c41084b0 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.176+0000 7f68c9575700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68b0038e50 0x7f68b003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.177+0000 7f68c9575700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68b0038e50 0x7f68b003b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006fd0 tx=0x7f68b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.181+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f68c001adb0 con 0x7f68c41084b0 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.394+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f68c4107200 con 0x7f68b0038e50 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.487+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f68c001aa90 con 0x7f68c41084b0 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.491+0000 7f68b6ffd700 1 -- 172.21.15.7:0/3063282922 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f68c4107200 con 0x7f68b0038e50 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68b0038e50 msgr2=0x7f68b003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68b0038e50 0x7f68b003b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006fd0 tx=0x7f68b8006e40 comp rx=0 tx=0).stop 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 msgr2=0x7f68c4107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c4107d30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68c0011a00 tx=0x7f68c0011ae0 comp rx=0 tx=0).stop 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 shutdown_connections 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68b0038e50 0x7f68b003b310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 --2- 172.21.15.7:0/3063282922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68c41084b0 0x7f68c4107d30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:28:59.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.493+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 >> 172.21.15.7:0/3063282922 conn(0x7f68c4101ee0 msgr2=0x7f68c41029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:28:59.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.494+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 shutdown_connections 2024-03-19T00:28:59.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:58.494+0000 7f68cbfda700 1 -- 172.21.15.7:0/3063282922 wait complete. 2024-03-19T00:28:59.068 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-03-19T00:28:59.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:59 smithi007 ceph-mon[62854]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:28:59.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:59 smithi007 ceph-mon[62854]: mgrmap e12: x(active, since 2s) 2024-03-19T00:28:59.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:28:59.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:28:59.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:28:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:00.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:00 smithi007 ceph-mon[62854]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:00.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$/1elBu0Ig50D3I54OB/pwugHHo3RG4LMhQgPn0rmZ5o2VZoRfYdBS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710808139, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-19T00:29:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.504+0000 7ff28d054700 1 Processor -- start 2024-03-19T00:29:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.504+0000 7ff28d054700 1 -- start start 2024-03-19T00:29:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff28d054700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff288107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff28d054700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff288107830 con 0x7ff288106e40 2024-03-19T00:29:00.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff286d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff288107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff286d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff288107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42048/0 (socket says 172.21.15.7:42048) 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff286d9d700 1 -- 172.21.15.7:0/897745947 learned_addr learned my addr 172.21.15.7:0/897745947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff286d9d700 1 -- 172.21.15.7:0/897745947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff288107b60 con 0x7ff288106e40 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff286d9d700 1 --2- 172.21.15.7:0/897745947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff288107260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2780098a0 tx=0x7ff278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28cac0a98b6e577 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.505+0000 7ff285d9b700 1 -- 172.21.15.7:0/897745947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff278004160 con 0x7ff288106e40 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff285d9b700 1 -- 172.21.15.7:0/897745947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff278004300 con 0x7ff288106e40 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff285d9b700 1 -- 172.21.15.7:0/897745947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff278011a50 con 0x7ff288106e40 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 -- 172.21.15.7:0/897745947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 msgr2=0x7ff288107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 --2- 172.21.15.7:0/897745947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff288107260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2780098a0 tx=0x7ff278009bb0 comp rx=0 tx=0).stop 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 -- 172.21.15.7:0/897745947 shutdown_connections 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 --2- 172.21.15.7:0/897745947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff288107260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 -- 172.21.15.7:0/897745947 >> 172.21.15.7:0/897745947 conn(0x7ff288101660 msgr2=0x7ff288103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 -- 172.21.15.7:0/897745947 shutdown_connections 2024-03-19T00:29:00.608 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.506+0000 7ff28d054700 1 -- 172.21.15.7:0/897745947 wait complete. 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.507+0000 7ff28d054700 1 Processor -- start 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.507+0000 7ff28d054700 1 -- start start 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff28d054700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff28819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff28d054700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff28810af10 con 0x7ff288106e40 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff286d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff28819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff286d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff28819b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42062/0 (socket says 172.21.15.7:42062) 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff286d9d700 1 -- 172.21.15.7:0/2572986910 learned_addr learned my addr 172.21.15.7:0/2572986910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff286d9d700 1 -- 172.21.15.7:0/2572986910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28819bf10 con 0x7ff288106e40 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff286d9d700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff28819b9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff278011a00 tx=0x7ff278011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff278003af0 con 0x7ff288106e40 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff27801a8e0 con 0x7ff288106e40 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.508+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28819c1a0 con 0x7ff288106e40 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.509+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff278024840 con 0x7ff288106e40 2024-03-19T00:29:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.509+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28819ee90 con 0x7ff288106e40 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.509+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7ff278022020 con 0x7ff288106e40 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.510+0000 7ff26ffff700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff270038fd0 0x7ff27003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.510+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff27804d2a0 con 0x7ff288106e40 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.510+0000 7ff28659c700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff270038fd0 0x7ff27003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.510+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff288063940 con 0x7ff288106e40 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.510+0000 7ff28659c700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff270038fd0 0x7ff27003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff27c006fd0 tx=0x7ff27c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.515+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff27801adb0 con 0x7ff288106e40 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.750+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff28802dd10 con 0x7ff270038fd0 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.995+0000 7ff26ffff700 1 -- 172.21.15.7:0/2572986910 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff28802dd10 con 0x7ff270038fd0 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.997+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff270038fd0 msgr2=0x7ff27003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.997+0000 7ff28d054700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff270038fd0 0x7ff27003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff27c006fd0 tx=0x7ff27c006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.997+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 msgr2=0x7ff28819b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:00.610 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.997+0000 7ff28d054700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff28819b9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff278011a00 tx=0x7ff278011ae0 comp rx=0 tx=0).stop 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.998+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 shutdown_connections 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.998+0000 7ff28d054700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff270038fd0 0x7ff27003b490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.998+0000 7ff28d054700 1 --2- 172.21.15.7:0/2572986910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff288106e40 0x7ff28819b9d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.998+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 >> 172.21.15.7:0/2572986910 conn(0x7ff288101660 msgr2=0x7ff2881024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.998+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 shutdown_connections 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:28:59.998+0000 7ff28d054700 1 -- 172.21.15.7:0/2572986910 wait complete. 2024-03-19T00:29:00.611 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-03-19T00:29:01.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:01 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3842910785' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-19T00:29:01.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f34411a8700 1 Processor -- start 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f34411a8700 1 -- start start 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f34411a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c106740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f34411a8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f343c07be30 con 0x7f343c106320 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f343ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f343ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42068/0 (socket says 172.21.15.7:42068) 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f343ad9d700 1 -- 172.21.15.7:0/2916118826 learned_addr learned my addr 172.21.15.7:0/2916118826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.019+0000 7f343ad9d700 1 -- 172.21.15.7:0/2916118826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c106c80 con 0x7f343c106320 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f343ad9d700 1 --2- 172.21.15.7:0/2916118826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c106740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f342c0098a0 tx=0x7f342c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76504eeb4bd89fd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f3439d9b700 1 -- 172.21.15.7:0/2916118826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f342c004160 con 0x7f343c106320 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f3439d9b700 1 -- 172.21.15.7:0/2916118826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c004300 con 0x7f343c106320 2024-03-19T00:29:01.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f3439d9b700 1 -- 172.21.15.7:0/2916118826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f342c011a50 con 0x7f343c106320 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f34411a8700 1 -- 172.21.15.7:0/2916118826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 msgr2=0x7f343c106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f34411a8700 1 --2- 172.21.15.7:0/2916118826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c106740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f342c0098a0 tx=0x7f342c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f34411a8700 1 -- 172.21.15.7:0/2916118826 shutdown_connections 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f34411a8700 1 --2- 172.21.15.7:0/2916118826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c106740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f34411a8700 1 -- 172.21.15.7:0/2916118826 >> 172.21.15.7:0/2916118826 conn(0x7f343c101ee0 msgr2=0x7f343c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.020+0000 7f34411a8700 1 -- 172.21.15.7:0/2916118826 shutdown_connections 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.021+0000 7f34411a8700 1 -- 172.21.15.7:0/2916118826 wait complete. 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.021+0000 7f34411a8700 1 Processor -- start 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.021+0000 7f34411a8700 1 -- start start 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f34411a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f34411a8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f343c107cf0 con 0x7f343c106320 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f343ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f343ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42072/0 (socket says 172.21.15.7:42072) 2024-03-19T00:29:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f343ad9d700 1 -- 172.21.15.7:0/3842910785 learned_addr learned my addr 172.21.15.7:0/3842910785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f343ad9d700 1 -- 172.21.15.7:0/3842910785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c19dfe0 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f343ad9d700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c19daa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f342c011a20 tx=0x7f342c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f342c003b70 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c01a8e0 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f342c024990 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.022+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c19e270 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.023+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c1a0f60 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.023+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f342c022020 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.023+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c063940 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.024+0000 7f3423fff700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3424038f80 0x7f342403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.024+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f342c04d440 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.024+0000 7f343a59c700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3424038f80 0x7f342403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.024+0000 7f343a59c700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3424038f80 0x7f342403b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3430006fd0 tx=0x7f3430006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.029+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f342c01a5d0 con 0x7f343c106320 2024-03-19T00:29:01.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.255+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f343c02de00 con 0x7f343c106320 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.255+0000 7f3423fff700 1 -- 172.21.15.7:0/3842910785 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f342c02d910 con 0x7f343c106320 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3424038f80 msgr2=0x7f342403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3424038f80 0x7f342403b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3430006fd0 tx=0x7f3430006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 msgr2=0x7f343c19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c19daa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f342c011a20 tx=0x7f342c011b00 comp rx=0 tx=0).stop 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 shutdown_connections 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3424038f80 0x7f342403b440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 --2- 172.21.15.7:0/3842910785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c106320 0x7f343c19daa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 >> 172.21.15.7:0/3842910785 conn(0x7f343c101ee0 msgr2=0x7f343c194740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 shutdown_connections 2024-03-19T00:29:01.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:01.257+0000 7f34411a8700 1 -- 172.21.15.7:0/3842910785 wait complete. 2024-03-19T00:29:01.858 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:29:01.858 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-19T00:29:01.865 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:29:01.866 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-19T00:29:01.866 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-19T00:29:01.866 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-19T00:29:01.868 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-03-19T00:29:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-03-19T00:29:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-03-19T00:29:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: Password: xgueqymjcj 2024-03-19T00:29:01.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:01.869 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config directory 2024-03-19T00:29:01.869 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.298+0000 7fab13275700 1 Processor -- start 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.298+0000 7fab13275700 1 -- start start 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.298+0000 7fab13275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c107e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.298+0000 7fab13275700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c108440 con 0x7fab0c107a50 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab11011700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c107e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab11011700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c107e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42086/0 (socket says 172.21.15.7:42086) 2024-03-19T00:29:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab11011700 1 -- 172.21.15.7:0/3772920730 learned_addr learned my addr 172.21.15.7:0/3772920730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab11011700 1 -- 172.21.15.7:0/3772920730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c108c50 con 0x7fab0c107a50 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab11011700 1 --2- 172.21.15.7:0/3772920730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c107e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab080098a0 tx=0x7fab08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f515b33672fe7ec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab03fff700 1 -- 172.21.15.7:0/3772920730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab08004160 con 0x7fab0c107a50 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab03fff700 1 -- 172.21.15.7:0/3772920730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab08004300 con 0x7fab0c107a50 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.299+0000 7fab03fff700 1 -- 172.21.15.7:0/3772920730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab08011a50 con 0x7fab0c107a50 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 -- 172.21.15.7:0/3772920730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 msgr2=0x7fab0c107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 --2- 172.21.15.7:0/3772920730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c107e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab080098a0 tx=0x7fab08009bb0 comp rx=0 tx=0).stop 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 -- 172.21.15.7:0/3772920730 shutdown_connections 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 --2- 172.21.15.7:0/3772920730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c107e70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 -- 172.21.15.7:0/3772920730 >> 172.21.15.7:0/3772920730 conn(0x7fab0c103050 msgr2=0x7fab0c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 -- 172.21.15.7:0/3772920730 shutdown_connections 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.300+0000 7fab13275700 1 -- 172.21.15.7:0/3772920730 wait complete. 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab13275700 1 Processor -- start 2024-03-19T00:29:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab13275700 1 -- start start 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab13275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab13275700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10c000 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab11011700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab11011700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c19b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42098/0 (socket says 172.21.15.7:42098) 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab11011700 1 -- 172.21.15.7:0/3165154972 learned_addr learned my addr 172.21.15.7:0/3165154972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab11011700 1 -- 172.21.15.7:0/3165154972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19be90 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab11011700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c19b950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab08006b20 tx=0x7fab08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab080039e0 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0801a8e0 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19c120 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.301+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab080247e0 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.302+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c19ee10 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.302+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fab08022020 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.302+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c063940 con 0x7fab0c107a50 2024-03-19T00:29:03.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.303+0000 7fab027fc700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faaf8038f70 0x7faaf803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.303+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fab0804d530 con 0x7fab0c107a50 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.303+0000 7fab10810700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faaf8038f70 0x7faaf803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.303+0000 7fab10810700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faaf8038f70 0x7faaf803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faafc006fd0 tx=0x7faafc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.308+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fab0802d910 con 0x7fab0c107a50 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.489+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7fab0802d3f0 con 0x7fab0c107a50 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.514+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fab0c19efe0 con 0x7fab0c107a50 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.515+0000 7fab027fc700 1 -- 172.21.15.7:0/3165154972 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fab0801ae00 con 0x7fab0c107a50 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faaf8038f70 msgr2=0x7faaf803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faaf8038f70 0x7faaf803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faafc006fd0 tx=0x7faafc006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 msgr2=0x7fab0c19b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c19b950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab08006b20 tx=0x7fab08011ad0 comp rx=0 tx=0).stop 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 shutdown_connections 2024-03-19T00:29:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faaf8038f70 0x7faaf803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:03.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 --2- 172.21.15.7:0/3165154972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c107a50 0x7fab0c19b950 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:03.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 >> 172.21.15.7:0/3165154972 conn(0x7fab0c103050 msgr2=0x7fab0c103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:03.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 shutdown_connections 2024-03-19T00:29:03.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:02.517+0000 7fab13275700 1 -- 172.21.15.7:0/3165154972 wait complete. 2024-03-19T00:29:03.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:03 smithi007 ceph-mon[62854]: mgrmap e13: x(active, since 6s) 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f78541f7700 1 Processor -- start 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f78541f7700 1 -- start start 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f78541f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c106190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f78541f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f784c106760 con 0x7f784c105d70 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f7851f93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f7851f93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42104/0 (socket says 172.21.15.7:42104) 2024-03-19T00:29:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.432+0000 7f7851f93700 1 -- 172.21.15.7:0/1806445373 learned_addr learned my addr 172.21.15.7:0/1806445373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f7851f93700 1 -- 172.21.15.7:0/1806445373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c108d00 con 0x7f784c105d70 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f7851f93700 1 --2- 172.21.15.7:0/1806445373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c106190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78480098a0 tx=0x7f7848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ef9d714adc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f7850f91700 1 -- 172.21.15.7:0/1806445373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7848004160 con 0x7f784c105d70 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f7850f91700 1 -- 172.21.15.7:0/1806445373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7848004300 con 0x7f784c105d70 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f7850f91700 1 -- 172.21.15.7:0/1806445373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7848011a50 con 0x7f784c105d70 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f78541f7700 1 -- 172.21.15.7:0/1806445373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 msgr2=0x7f784c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.433+0000 7f78541f7700 1 --2- 172.21.15.7:0/1806445373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c106190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78480098a0 tx=0x7f7848009bb0 comp rx=0 tx=0).stop 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 -- 172.21.15.7:0/1806445373 shutdown_connections 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 --2- 172.21.15.7:0/1806445373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c106190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 -- 172.21.15.7:0/1806445373 >> 172.21.15.7:0/1806445373 conn(0x7f784c101680 msgr2=0x7f784c103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 -- 172.21.15.7:0/1806445373 shutdown_connections 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 -- 172.21.15.7:0/1806445373 wait complete. 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 Processor -- start 2024-03-19T00:29:04.307 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 -- start start 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f78541f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f784c10c0b0 con 0x7f784c105d70 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.434+0000 7f7851f93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7851f93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c19b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42108/0 (socket says 172.21.15.7:42108) 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7851f93700 1 -- 172.21.15.7:0/3778094066 learned_addr learned my addr 172.21.15.7:0/3778094066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7851f93700 1 -- 172.21.15.7:0/3778094066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c19bf30 con 0x7f784c105d70 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7851f93700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c19b9f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7848011a20 tx=0x7f7848011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7848003af0 con 0x7f784c105d70 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f784801a8e0 con 0x7f784c105d70 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7848024860 con 0x7f784c105d70 2024-03-19T00:29:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c19c1c0 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.435+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c19eeb0 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.436+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f7848022020 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.436+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c063940 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.436+0000 7f7842ffd700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7838039010 0x7f783803b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.436+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f784804d360 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.436+0000 7f7851792700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7838039010 0x7f783803b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.437+0000 7f7851792700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7838039010 0x7f783803b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f783c006fd0 tx=0x7f783c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.441+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f78480094f0 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.763+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f784c19f080 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.765+0000 7f7842ffd700 1 -- 172.21.15.7:0/3778094066 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f784801a5c0 con 0x7f784c105d70 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7838039010 msgr2=0x7f783803b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7838039010 0x7f783803b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f783c006fd0 tx=0x7f783c006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 msgr2=0x7f784c19b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c19b9f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7848011a20 tx=0x7f7848011b00 comp rx=0 tx=0).stop 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 shutdown_connections 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7838039010 0x7f783803b4d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 --2- 172.21.15.7:0/3778094066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f784c105d70 0x7f784c19b9f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 >> 172.21.15.7:0/3778094066 conn(0x7f784c101680 msgr2=0x7f784c102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 shutdown_connections 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-19T00:29:03.768+0000 7f78541f7700 1 -- 172.21.15.7:0/3778094066 wait complete. 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:04.311 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-03-19T00:29:04.345 INFO:tasks.cephadm:Fetching config... 2024-03-19T00:29:04.346 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:29:04.346 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-19T00:29:04.361 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-19T00:29:04.361 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:29:04.361 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-19T00:29:04.416 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-19T00:29:04.417 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:29:04.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-19T00:29:04.483 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-19T00:29:04.484 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:29:04.484 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-19T00:29:04.538 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-19T00:29:04.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T00:29:04.617 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:04.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T00:29:04.675 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:04.690 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T00:29:04.737 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaAd+rM+MslSOCa+b8w2ejLC7jX7mHPL+JGX4pakf49JsfnuwF82bDH/mDxt/SUQIlytn3bBf0rIC26UXbYshyPq9Rk2NGtCFsAWKTc9t9A6+Qct2nETbc+3+uywWlV0lcmKYjzYq/maKLQl4Ee1tlC3uvThXpMEJPysvwE+KyCBW9xr7DL7HcxmAV1xgwk3Bcb4gQrm5eAHJz2MN0klhgXOVX8IOl6nTR9tplbxF9QOUaxs++/6DI+0jp8RpyMSBZJ+ErHzmM/M7wI8TCPJ5qPeVAODA++spesFXzE8s09W0BDgA7cmnLrmx7qx+kHgAX73ecHBbhcxXuujNDx/S/uyVznTzJ8/ImaBrroesQ9NYJFe22jf8fSNM3CC72Nyy/f4rcgs/sgG1fMh/zTNaeEPON3wdb+9npeVErfAe+BxanxtERbDhKzE6/ePY+9he30Qbv95DEhJKjvRPDvPGjEtZi355vpEMHGTHEzjEKpcAeTejyE9up6seqscMwah8= ceph-60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:04.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-19T00:29:04.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:04 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3778094066' entity='client.admin' 2024-03-19T00:29:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 -- 172.21.15.7:0/2454757440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 msgr2=0x7f15d0101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 --2- 172.21.15.7:0/2454757440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d0101060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009910 tx=0x7f15cc009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 -- 172.21.15.7:0/2454757440 shutdown_connections 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 --2- 172.21.15.7:0/2454757440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d0101060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 -- 172.21.15.7:0/2454757440 >> 172.21.15.7:0/2454757440 conn(0x7f15d00fc1e0 msgr2=0x7f15d00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 -- 172.21.15.7:0/2454757440 shutdown_connections 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.070+0000 7f15d7bee700 1 -- 172.21.15.7:0/2454757440 wait complete. 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d7bee700 1 Processor -- start 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d7bee700 1 -- start start 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d7bee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d7bee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0106b80 con 0x7f15d0100c40 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d598a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d598a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d006a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45634/0 (socket says 172.21.15.7:45634) 2024-03-19T00:29:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d598a700 1 -- 172.21.15.7:0/1778595382 learned_addr learned my addr 172.21.15.7:0/1778595382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.071+0000 7f15d598a700 1 -- 172.21.15.7:0/1778595382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d019a0b0 con 0x7f15d0100c40 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.072+0000 7f15d598a700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d006a1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005800 tx=0x7f15cc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.072+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15cc027070 con 0x7f15d0100c40 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.072+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d019a280 con 0x7f15d0100c40 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.072+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15cc00be10 con 0x7f15d0100c40 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.072+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15cc00f460 con 0x7f15d0100c40 2024-03-19T00:29:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.072+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d019a6a0 con 0x7f15d0100c40 2024-03-19T00:29:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.073+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f15cc02a070 con 0x7f15d0100c40 2024-03-19T00:29:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.073+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d0107300 con 0x7f15d0100c40 2024-03-19T00:29:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.073+0000 7f15c6ffd700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15bc038fc0 0x7f15bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.073+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f15cc01e030 con 0x7f15d0100c40 2024-03-19T00:29:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.073+0000 7f15d5189700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15bc038fc0 0x7f15bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:07.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.073+0000 7f15d5189700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15bc038fc0 0x7f15bc03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:07.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.078+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15cc013070 con 0x7f15d0100c40 2024-03-19T00:29:07.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.281+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f15d002dde0 con 0x7f15d0100c40 2024-03-19T00:29:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:29:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:07.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.284+0000 7f15c6ffd700 1 -- 172.21.15.7:0/1778595382 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f15cc031070 con 0x7f15d0100c40 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15bc038fc0 msgr2=0x7f15bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15bc038fc0 0x7f15bc03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 msgr2=0x7f15d006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d006a1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005800 tx=0x7f15cc009f60 comp rx=0 tx=0).stop 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 shutdown_connections 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15bc038fc0 0x7f15bc03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 --2- 172.21.15.7:0/1778595382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0100c40 0x7f15d006a1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 >> 172.21.15.7:0/1778595382 conn(0x7f15d00fc1e0 msgr2=0x7f15d00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 shutdown_connections 2024-03-19T00:29:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:07.286+0000 7f15d7bee700 1 -- 172.21.15.7:0/1778595382 wait complete. 2024-03-19T00:29:07.864 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-19T00:29:07.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-19T00:29:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:08.513 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:08 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1778595382' entity='client.admin' 2024-03-19T00:29:08.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:08.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 -- 172.21.15.7:0/2302917614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 msgr2=0x7f1fa41010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 --2- 172.21.15.7:0/2302917614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41010a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009910 tx=0x7f1f9c009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 -- 172.21.15.7:0/2302917614 shutdown_connections 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 --2- 172.21.15.7:0/2302917614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41010a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 -- 172.21.15.7:0/2302917614 >> 172.21.15.7:0/2302917614 conn(0x7f1fa40fc1e0 msgr2=0x7f1fa40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 -- 172.21.15.7:0/2302917614 shutdown_connections 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.562+0000 7f1fa9638700 1 -- 172.21.15.7:0/2302917614 wait complete. 2024-03-19T00:29:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa9638700 1 Processor -- start 2024-03-19T00:29:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa9638700 1 -- start start 2024-03-19T00:29:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa9638700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa9638700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4106bc0 con 0x7f1fa4100c80 2024-03-19T00:29:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45644/0 (socket says 172.21.15.7:45644) 2024-03-19T00:29:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa2ffd700 1 -- 172.21.15.7:0/914505725 learned_addr learned my addr 172.21.15.7:0/914505725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.563+0000 7f1fa2ffd700 1 -- 172.21.15.7:0/914505725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa4199ae0 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.564+0000 7f1fa2ffd700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41995a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c005800 tx=0x7f1f9c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.564+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f9c027070 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.564+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f9c00be10 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.564+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa4199d70 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.564+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f9c00f460 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.564+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa419ca60 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.565+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f1f9c02a070 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.565+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa4107340 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.565+0000 7f1f9bfff700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f90038fc0 0x7f1f9003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.565+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f9c01e030 con 0x7f1fa4100c80 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.565+0000 7f1fa27fc700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f90038fc0 0x7f1f9003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:09.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.565+0000 7f1fa27fc700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f90038fc0 0x7f1f9003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c006fd0 tx=0x7f1f8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.570+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f9c013070 con 0x7f1fa4100c80 2024-03-19T00:29:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.779+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f1fa40fd910 con 0x7f1f90038fc0 2024-03-19T00:29:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.782+0000 7f1f9bfff700 1 -- 172.21.15.7:0/914505725 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1fa40fd910 con 0x7f1f90038fc0 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f90038fc0 msgr2=0x7f1f9003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f90038fc0 0x7f1f9003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c006fd0 tx=0x7f1f8c006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 msgr2=0x7f1fa41995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41995a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c005800 tx=0x7f1f9c009f60 comp rx=0 tx=0).stop 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 shutdown_connections 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f90038fc0 0x7f1f9003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 --2- 172.21.15.7:0/914505725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4100c80 0x7f1fa41995a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 >> 172.21.15.7:0/914505725 conn(0x7f1fa40fc1e0 msgr2=0x7f1fa406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 shutdown_connections 2024-03-19T00:29:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:09.784+0000 7f1fa9638700 1 -- 172.21.15.7:0/914505725 wait complete. 2024-03-19T00:29:10.514 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi044 2024-03-19T00:29:10.514 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:29:10.515 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.conf 2024-03-19T00:29:10.533 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:29:10.533 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:10.590 INFO:tasks.cephadm:Adding host smithi044 to orchestrator... 2024-03-19T00:29:10.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch host add smithi044 2024-03-19T00:29:10.833 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:11.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:11.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:11.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:11.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:11.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:11.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-19T00:29:11.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:11.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:11.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:11 smithi007 ceph-mon[62854]: Updating smithi007:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:11 smithi007 ceph-mon[62854]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:11 smithi007 ceph-mon[62854]: Updating smithi007:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:29:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.267+0000 7fa41df93700 1 -- 172.21.15.7:0/2159373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 msgr2=0x7fa418103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.267+0000 7fa41df93700 1 --2- 172.21.15.7:0/2159373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa418103290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa408009910 tx=0x7fa408009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.267+0000 7fa41df93700 1 -- 172.21.15.7:0/2159373868 shutdown_connections 2024-03-19T00:29:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.267+0000 7fa41df93700 1 --2- 172.21.15.7:0/2159373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa418103290 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.267+0000 7fa41df93700 1 -- 172.21.15.7:0/2159373868 >> 172.21.15.7:0/2159373868 conn(0x7fa4180fc700 msgr2=0x7fa4180feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.268+0000 7fa41df93700 1 -- 172.21.15.7:0/2159373868 shutdown_connections 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.268+0000 7fa41df93700 1 -- 172.21.15.7:0/2159373868 wait complete. 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.268+0000 7fa41df93700 1 Processor -- start 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.268+0000 7fa41df93700 1 -- start start 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.268+0000 7fa41df93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa41819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.268+0000 7fa41df93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa41810c250 con 0x7fa418102eb0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa4177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa41819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa4177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa41819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45658/0 (socket says 172.21.15.7:45658) 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa4177fe700 1 -- 172.21.15.7:0/4252957997 learned_addr learned my addr 172.21.15.7:0/4252957997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa4177fe700 1 -- 172.21.15.7:0/4252957997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41819d970 con 0x7fa418102eb0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa4177fe700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa41819d430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4080056d0 tx=0x7fa4080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa408026070 con 0x7fa418102eb0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa408004370 con 0x7fa418102eb0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41819dc00 con 0x7fa418102eb0 2024-03-19T00:29:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4080039b0 con 0x7fa418102eb0 2024-03-19T00:29:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.269+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181975f0 con 0x7fa418102eb0 2024-03-19T00:29:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.270+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7fa408029070 con 0x7fa418102eb0 2024-03-19T00:29:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.270+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41806aa50 con 0x7fa418102eb0 2024-03-19T00:29:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.270+0000 7fa414ff9700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa400038fc0 0x7fa40003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.270+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa40801e070 con 0x7fa418102eb0 2024-03-19T00:29:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.270+0000 7fa416ffd700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa400038fc0 0x7fa40003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.271+0000 7fa416ffd700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa400038fc0 0x7fa40003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006fd0 tx=0x7fa40c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.275+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa408013070 con 0x7fa418102eb0 2024-03-19T00:29:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:12.485+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}) -- 0x7fa418198b20 con 0x7fa400038fc0 2024-03-19T00:29:13.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:12 smithi007 ceph-mon[62854]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:14.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:13 smithi007 ceph-mon[62854]: Deploying cephadm binary to smithi044 2024-03-19T00:29:16.867 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi044' with addr '172.21.15.44' 2024-03-19T00:29:16.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.866+0000 7fa414ff9700 1 -- 172.21.15.7:0/4252957997 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa418198b20 con 0x7fa400038fc0 2024-03-19T00:29:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa400038fc0 msgr2=0x7fa40003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa400038fc0 0x7fa40003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006fd0 tx=0x7fa40c006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 msgr2=0x7fa41819d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa41819d430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4080056d0 tx=0x7fa4080057b0 comp rx=0 tx=0).stop 2024-03-19T00:29:16.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 shutdown_connections 2024-03-19T00:29:16.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa400038fc0 0x7fa40003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:16.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 --2- 172.21.15.7:0/4252957997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa418102eb0 0x7fa41819d430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:16.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 >> 172.21.15.7:0/4252957997 conn(0x7fa4180fc700 msgr2=0x7fa418103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:16.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 shutdown_connections 2024-03-19T00:29:16.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:16.868+0000 7fa41df93700 1 -- 172.21.15.7:0/4252957997 wait complete. 2024-03-19T00:29:17.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:16 smithi007 ceph-mon[62854]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:17.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:17.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:17.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-19T00:29:17.696 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:18.118 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:17 smithi007 ceph-mon[62854]: Added host smithi044 2024-03-19T00:29:18.119 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:17 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:18.119 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:17 smithi007 ceph-mon[62854]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 -- 172.21.15.7:0/1114335831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 msgr2=0x7f26e4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 --2- 172.21.15.7:0/1114335831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4102330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009910 tx=0x7f26d4009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 -- 172.21.15.7:0/1114335831 shutdown_connections 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 --2- 172.21.15.7:0/1114335831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4102330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 -- 172.21.15.7:0/1114335831 >> 172.21.15.7:0/1114335831 conn(0x7f26e40fd820 msgr2=0x7f26e40ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 -- 172.21.15.7:0/1114335831 shutdown_connections 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.153+0000 7f26e9dc5700 1 -- 172.21.15.7:0/1114335831 wait complete. 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e9dc5700 1 Processor -- start 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e9dc5700 1 -- start start 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e9dc5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e9dc5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26e410c1d0 con 0x7f26e4101f50 2024-03-19T00:29:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47634/0 (socket says 172.21.15.7:47634) 2024-03-19T00:29:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e37fe700 1 -- 172.21.15.7:0/879085844 learned_addr learned my addr 172.21.15.7:0/879085844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.154+0000 7f26e37fe700 1 -- 172.21.15.7:0/879085844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e4076500 con 0x7f26e4101f50 2024-03-19T00:29:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.155+0000 7f26e37fe700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4074610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f26d40056d0 tx=0x7f26d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.155+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4026070 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.155+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e4074bb0 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.155+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26d400bc00 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.155+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d40039b0 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.155+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e4075070 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.156+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f26d4029070 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.156+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e4103430 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.156+0000 7f26e0ff9700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26cc038fc0 0x7f26cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.156+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f26d401e070 con 0x7f26e4101f50 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.156+0000 7f26e2ffd700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26cc038fc0 0x7f26cc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.156+0000 7f26e2ffd700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26cc038fc0 0x7f26cc03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.161+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26d4013070 con 0x7f26e4101f50 2024-03-19T00:29:19.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.371+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26e4075300 con 0x7f26cc038fc0 2024-03-19T00:29:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.372+0000 7f26e0ff9700 1 -- 172.21.15.7:0/879085844 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f26e4075300 con 0x7f26cc038fc0 2024-03-19T00:29:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:19.373 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}] 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26cc038fc0 msgr2=0x7f26cc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26cc038fc0 0x7f26cc03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 msgr2=0x7f26e4074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4074610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f26d40056d0 tx=0x7f26d40057b0 comp rx=0 tx=0).stop 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 shutdown_connections 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26cc038fc0 0x7f26cc03b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 --2- 172.21.15.7:0/879085844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e4101f50 0x7f26e4074610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 >> 172.21.15.7:0/879085844 conn(0x7f26e40fd820 msgr2=0x7f26e4103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 shutdown_connections 2024-03-19T00:29:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:19.374+0000 7f26e9dc5700 1 -- 172.21.15.7:0/879085844 wait complete. 2024-03-19T00:29:19.947 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-03-19T00:29:19.948 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:29:19.948 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-03-19T00:29:19.966 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:29:19.966 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:20.024 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-03-19T00:29:20.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch host add smithi204 2024-03-19T00:29:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:20.682 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:20 smithi007 ceph-mon[62854]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:29:20.682 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:20.682 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:20 smithi007 ceph-mon[62854]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- 172.21.15.7:0/2076032188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 msgr2=0x7fef60102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 --2- 172.21.15.7:0/2076032188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef60102330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef50009910 tx=0x7fef50009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- 172.21.15.7:0/2076032188 shutdown_connections 2024-03-19T00:29:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 --2- 172.21.15.7:0/2076032188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef60102330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- 172.21.15.7:0/2076032188 >> 172.21.15.7:0/2076032188 conn(0x7fef600fd820 msgr2=0x7fef600ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- 172.21.15.7:0/2076032188 shutdown_connections 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- 172.21.15.7:0/2076032188 wait complete. 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 Processor -- start 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- start start 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef6019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.649+0000 7fef65c36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef6010c1d0 con 0x7fef60101f50 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef6019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef6019d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47638/0 (socket says 172.21.15.7:47638) 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5f7fe700 1 -- 172.21.15.7:0/1647909945 learned_addr learned my addr 172.21.15.7:0/1647909945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5f7fe700 1 -- 172.21.15.7:0/1647909945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6019d910 con 0x7fef60101f50 2024-03-19T00:29:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5f7fe700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef6019d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef500056d0 tx=0x7fef500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef50026070 con 0x7fef60101f50 2024-03-19T00:29:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef5000bba0 con 0x7fef60101f50 2024-03-19T00:29:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6019dc00 con 0x7fef60101f50 2024-03-19T00:29:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef500037f0 con 0x7fef60101f50 2024-03-19T00:29:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.650+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef60197590 con 0x7fef60101f50 2024-03-19T00:29:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.651+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7fef50029070 con 0x7fef60101f50 2024-03-19T00:29:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.651+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef60103430 con 0x7fef60101f50 2024-03-19T00:29:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.651+0000 7fef5d7fa700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fef4c038fc0 0x7fef4c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.651+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef5001e070 con 0x7fef60101f50 2024-03-19T00:29:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.651+0000 7fef57fff700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fef4c038fc0 0x7fef4c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.652+0000 7fef57fff700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fef4c038fc0 0x7fef4c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef48006fd0 tx=0x7fef48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.656+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef50013070 con 0x7fef60101f50 2024-03-19T00:29:21.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:21.866+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7fef60198ac0 con 0x7fef4c038fc0 2024-03-19T00:29:22.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:21 smithi007 ceph-mon[62854]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:21 smithi007 ceph-mon[62854]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:23.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:23 smithi007 ceph-mon[62854]: Deploying cephadm binary to smithi204 2024-03-19T00:29:24.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:24 smithi007 ceph-mon[62854]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:24.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:24 smithi007 ceph-mon[62854]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:29:24.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:24.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:24.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:24.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:24 smithi007 ceph-mon[62854]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:26.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:25 smithi007 ceph-mon[62854]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:26.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.603+0000 7fef5d7fa700 1 -- 172.21.15.7:0/1647909945 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fef60198ac0 con 0x7fef4c038fc0 2024-03-19T00:29:26.605 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fef4c038fc0 msgr2=0x7fef4c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fef4c038fc0 0x7fef4c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef48006fd0 tx=0x7fef48006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 msgr2=0x7fef6019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef6019d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef500056d0 tx=0x7fef500057b0 comp rx=0 tx=0).stop 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 shutdown_connections 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fef4c038fc0 0x7fef4c03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 --2- 172.21.15.7:0/1647909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef60101f50 0x7fef6019d3d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 >> 172.21.15.7:0/1647909945 conn(0x7fef600fd820 msgr2=0x7fef60103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 shutdown_connections 2024-03-19T00:29:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:26.606+0000 7fef65c36700 1 -- 172.21.15.7:0/1647909945 wait complete. 2024-03-19T00:29:27.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-19T00:29:27.488 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:27.887 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:27.887 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:27 smithi007 ceph-mon[62854]: Added host smithi204 2024-03-19T00:29:27.887 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:27.887 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:28.869 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:28 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:28.869 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:28 smithi007 ceph-mon[62854]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 -- 172.21.15.7:0/1311560029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 msgr2=0x7f225c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 --2- 172.21.15.7:0/1311560029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c100910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f225400b0a0 tx=0x7f225400b3b0 comp rx=0 tx=0).stop 2024-03-19T00:29:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 -- 172.21.15.7:0/1311560029 shutdown_connections 2024-03-19T00:29:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 --2- 172.21.15.7:0/1311560029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c100910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 -- 172.21.15.7:0/1311560029 >> 172.21.15.7:0/1311560029 conn(0x7f225c0772c0 msgr2=0x7f225c0776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 -- 172.21.15.7:0/1311560029 shutdown_connections 2024-03-19T00:29:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.895+0000 7f2261027700 1 -- 172.21.15.7:0/1311560029 wait complete. 2024-03-19T00:29:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f2261027700 1 Processor -- start 2024-03-19T00:29:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f2261027700 1 -- start start 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f2261027700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f2261027700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10c270 con 0x7f225c100530 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f225ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f225ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46056/0 (socket says 172.21.15.7:46056) 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f225ad9d700 1 -- 172.21.15.7:0/1529432741 learned_addr learned my addr 172.21.15.7:0/1529432741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.896+0000 7f225ad9d700 1 -- 172.21.15.7:0/1529432741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c197c90 con 0x7f225c100530 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.897+0000 7f225ad9d700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c197750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22540076a0 tx=0x7f2254003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.897+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2254017040 con 0x7f225c100530 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.897+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2254022070 con 0x7f225c100530 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.897+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c197f20 con 0x7f225c100530 2024-03-19T00:29:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.897+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f225401d5e0 con 0x7f225c100530 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.897+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c19bfd0 con 0x7f225c100530 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.898+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f225401d8a0 con 0x7f225c100530 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.898+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c1035f0 con 0x7f225c100530 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.898+0000 7f2253fff700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2248038fc0 0x7f224803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.898+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f225400f050 con 0x7f225c100530 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.898+0000 7f225a59c700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2248038fc0 0x7f224803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.898+0000 7f225a59c700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2248038fc0 0x7f224803b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2244006fd0 tx=0x7f2244006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:28.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:28.903+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f225401c3b0 con 0x7f225c100530 2024-03-19T00:29:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.113+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f225c198a60 con 0x7f2248038fc0 2024-03-19T00:29:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.114+0000 7f2253fff700 1 -- 172.21.15.7:0/1529432741 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f225c198a60 con 0x7f2248038fc0 2024-03-19T00:29:29.115 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:29:29.115 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-03-19T00:29:29.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2248038fc0 msgr2=0x7f224803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:29.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2248038fc0 0x7f224803b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2244006fd0 tx=0x7f2244006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:29.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 msgr2=0x7f225c197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:29.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c197750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22540076a0 tx=0x7f2254003ce0 comp rx=0 tx=0).stop 2024-03-19T00:29:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 shutdown_connections 2024-03-19T00:29:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2248038fc0 0x7f224803b480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 --2- 172.21.15.7:0/1529432741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f225c100530 0x7f225c197750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 >> 172.21.15.7:0/1529432741 conn(0x7f225c0772c0 msgr2=0x7f225c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 shutdown_connections 2024-03-19T00:29:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:29.116+0000 7f2261027700 1 -- 172.21.15.7:0/1529432741 wait complete. 2024-03-19T00:29:29.658 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-19T00:29:29.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-19T00:29:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:30.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:30 smithi007 ceph-mon[62854]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:29:30.301 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:30.301 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:30 smithi007 ceph-mon[62854]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 -- 172.21.15.7:0/1299489860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 msgr2=0x7f1970103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 --2- 172.21.15.7:0/1299489860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970103300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1968009910 tx=0x7f1968009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 -- 172.21.15.7:0/1299489860 shutdown_connections 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 --2- 172.21.15.7:0/1299489860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970103300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 -- 172.21.15.7:0/1299489860 >> 172.21.15.7:0/1299489860 conn(0x7f1970077850 msgr2=0x7f19700feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 -- 172.21.15.7:0/1299489860 shutdown_connections 2024-03-19T00:29:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 -- 172.21.15.7:0/1299489860 wait complete. 2024-03-19T00:29:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.229+0000 7f19752cc700 1 Processor -- start 2024-03-19T00:29:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f19752cc700 1 -- start start 2024-03-19T00:29:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f19752cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f19752cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f197010c230 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f196effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f196effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970100b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46078/0 (socket says 172.21.15.7:46078) 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f196effd700 1 -- 172.21.15.7:0/3862089139 learned_addr learned my addr 172.21.15.7:0/3862089139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f196effd700 1 -- 172.21.15.7:0/3862089139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1970102a20 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.230+0000 7f196effd700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970100b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19680056d0 tx=0x7f19680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.231+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1968026070 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.231+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1968004370 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.231+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19680039b0 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.231+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701010c0 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.231+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1970101580 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.232+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f1968029070 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.232+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1970069f60 con 0x7f1970102f20 2024-03-19T00:29:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.232+0000 7f1967fff700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f195c038fc0 0x7f195c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.232+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f196801e070 con 0x7f1970102f20 2024-03-19T00:29:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.232+0000 7f196e7fc700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f195c038fc0 0x7f195c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:31.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.232+0000 7f196e7fc700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f195c038fc0 0x7f195c03b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:31.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.237+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1968013070 con 0x7f1970102f20 2024-03-19T00:29:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.446+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1970102060 con 0x7f1970102f20 2024-03-19T00:29:31.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.497+0000 7f1967fff700 1 -- 172.21.15.7:0/3862089139 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1968030090 con 0x7f1970102f20 2024-03-19T00:29:31.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f195c038fc0 msgr2=0x7f195c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:31.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f195c038fc0 0x7f195c03b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:31.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 msgr2=0x7f1970100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:31.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970100b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19680056d0 tx=0x7f19680057b0 comp rx=0 tx=0).stop 2024-03-19T00:29:31.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 shutdown_connections 2024-03-19T00:29:31.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f195c038fc0 0x7f195c03b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:31.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 --2- 172.21.15.7:0/3862089139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1970102f20 0x7f1970100b80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:31.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 >> 172.21.15.7:0/3862089139 conn(0x7f1970077850 msgr2=0x7f1970103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:31.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.500+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 shutdown_connections 2024-03-19T00:29:31.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:31.501+0000 7f19752cc700 1 -- 172.21.15.7:0/3862089139 wait complete. 2024-03-19T00:29:31.502 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-03-19T00:29:31.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:31 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3862089139' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T00:29:32.092 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-03-19T00:29:32.092 INFO:tasks.cephadm:Adding mon.b on smithi044 2024-03-19T00:29:32.092 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-03-19T00:29:32.093 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c' 2024-03-19T00:29:32.351 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-19T00:29:32.399 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3862089139' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:32.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:29:32.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:32.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:33 smithi007 ceph-mon[62854]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-19T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:33 smithi007 ceph-mon[62854]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:33 smithi007 ceph-mon[62854]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 -- 172.21.15.204:0/2809026474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 msgr2=0x7f1a44101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 --2- 172.21.15.204:0/2809026474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a44101210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009910 tx=0x7f1a34009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 -- 172.21.15.204:0/2809026474 shutdown_connections 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 --2- 172.21.15.204:0/2809026474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a44101210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 -- 172.21.15.204:0/2809026474 >> 172.21.15.204:0/2809026474 conn(0x7f1a440fc6c0 msgr2=0x7f1a440feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 -- 172.21.15.204:0/2809026474 shutdown_connections 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.082+0000 7f1a49f10700 1 -- 172.21.15.204:0/2809026474 wait complete. 2024-03-19T00:29:34.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.083+0000 7f1a49f10700 1 Processor -- start 2024-03-19T00:29:34.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.083+0000 7f1a49f10700 1 -- start start 2024-03-19T00:29:34.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.083+0000 7f1a49f10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a441028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:34.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.083+0000 7f1a49f10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410b0b0 con 0x7f1a44100e30 2024-03-19T00:29:34.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a441028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:34.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a441028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.204:51098/0 (socket says 172.21.15.204:51098) 2024-03-19T00:29:34.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a437fe700 1 -- 172.21.15.204:0/1377086633 learned_addr learned my addr 172.21.15.204:0/1377086633 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:29:34.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a437fe700 1 -- 172.21.15.204:0/1377086633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a441047c0 con 0x7f1a44100e30 2024-03-19T00:29:34.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a437fe700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a441028d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1a340056d0 tx=0x7f1a340057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:34.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a34026070 con 0x7f1a44100e30 2024-03-19T00:29:34.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a44102e70 con 0x7f1a44100e30 2024-03-19T00:29:34.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a3400bc00 con 0x7f1a44100e30 2024-03-19T00:29:34.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a340039b0 con 0x7f1a44100e30 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.084+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a44103330 con 0x7f1a44100e30 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.085+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f1a34029070 con 0x7f1a44100e30 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.085+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44102310 con 0x7f1a44100e30 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.085+0000 7f1a417fa700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a30038fc0 0x7f1a3003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.085+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1a3401e070 con 0x7f1a44100e30 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.086+0000 7f1a3affd700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a30038fc0 0x7f1a3003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:34.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.086+0000 7f1a3affd700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a30038fc0 0x7f1a3003b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006fd0 tx=0x7f1a2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:34.092 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.091+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1a34013070 con 0x7f1a44100e30 2024-03-19T00:29:34.297 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.296+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7f1a44103e00 con 0x7f1a30038fc0 2024-03-19T00:29:34.303 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.302+0000 7f1a417fa700 1 -- 172.21.15.204:0/1377086633 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1a44103e00 con 0x7f1a30038fc0 2024-03-19T00:29:34.304 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a30038fc0 msgr2=0x7f1a3003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a30038fc0 0x7f1a3003b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006fd0 tx=0x7f1a2c006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 msgr2=0x7f1a441028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a441028d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1a340056d0 tx=0x7f1a340057b0 comp rx=0 tx=0).stop 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 shutdown_connections 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a30038fc0 0x7f1a3003b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 --2- 172.21.15.204:0/1377086633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a44100e30 0x7f1a441028d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 >> 172.21.15.204:0/1377086633 conn(0x7f1a440fc6c0 msgr2=0x7f1a44069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 shutdown_connections 2024-03-19T00:29:34.306 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:34.305+0000 7f1a49f10700 1 -- 172.21.15.204:0/1377086633 wait complete. 2024-03-19T00:29:34.897 DEBUG:teuthology.orchestra.run.smithi044:mon.b> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.b.service 2024-03-19T00:29:34.899 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.c.service 2024-03-19T00:29:34.901 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T00:29:34.902 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T00:29:34.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:34.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:35 smithi007 ceph-mon[62854]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:35 smithi007 ceph-mon[62854]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c;count:3 2024-03-19T00:29:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:35 smithi007 ceph-mon[62854]: Deploying daemon mon.c on smithi204 2024-03-19T00:29:35.983 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:36.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:36 smithi007 ceph-mon[62854]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:38.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:37 smithi007 ceph-mon[62854]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:38.769 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.767+0000 7f3528d3b700 1 -- 172.21.15.204:0/1210234434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 msgr2=0x7f3524101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.767+0000 7f3528d3b700 1 --2- 172.21.15.204:0/1210234434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f3524101220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b0a0 tx=0x7f351c00b3b0 comp rx=0 tx=0).stop 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.767+0000 7f3528d3b700 1 -- 172.21.15.204:0/1210234434 shutdown_connections 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.767+0000 7f3528d3b700 1 --2- 172.21.15.204:0/1210234434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f3524101220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.767+0000 7f3528d3b700 1 -- 172.21.15.204:0/1210234434 >> 172.21.15.204:0/1210234434 conn(0x7f3524077850 msgr2=0x7f35240feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.768+0000 7f3528d3b700 1 -- 172.21.15.204:0/1210234434 shutdown_connections 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.768+0000 7f3528d3b700 1 -- 172.21.15.204:0/1210234434 wait complete. 2024-03-19T00:29:38.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.768+0000 7f3528d3b700 1 Processor -- start 2024-03-19T00:29:38.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.768+0000 7f3528d3b700 1 -- start start 2024-03-19T00:29:38.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f3528d3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f35241976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:38.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f3528d3b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f352410b0c0 con 0x7f3524100e40 2024-03-19T00:29:38.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f352259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f35241976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:38.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f352259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f35241976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.204:51114/0 (socket says 172.21.15.204:51114) 2024-03-19T00:29:38.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f352259c700 1 -- 172.21.15.204:0/1981876232 learned_addr learned my addr 172.21.15.204:0/1981876232 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:29:38.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f352259c700 1 -- 172.21.15.204:0/1981876232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3524197c10 con 0x7f3524100e40 2024-03-19T00:29:38.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f352259c700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f35241976d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f351c0076a0 tx=0x7f351c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:38.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f351c017040 con 0x7f3524100e40 2024-03-19T00:29:38.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351c022070 con 0x7f3524100e40 2024-03-19T00:29:38.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3524197ea0 con 0x7f3524100e40 2024-03-19T00:29:38.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.769+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f351c01d5e0 con 0x7f3524100e40 2024-03-19T00:29:38.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.770+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352419bfa0 con 0x7f3524100e40 2024-03-19T00:29:38.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.770+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f351c004580 con 0x7f3524100e40 2024-03-19T00:29:38.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.770+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3524102320 con 0x7f3524100e40 2024-03-19T00:29:38.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.771+0000 7f35137fe700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f350c039000 0x7f350c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:38.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.771+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f351c00f050 con 0x7f3524100e40 2024-03-19T00:29:38.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.771+0000 7f3521d9b700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f350c039000 0x7f350c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:38.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.771+0000 7f3521d9b700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f350c039000 0x7f350c03b4c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3518006fd0 tx=0x7f3518006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:38.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:38.776+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f351c01c3b0 con 0x7f3524100e40 2024-03-19T00:29:39.070 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.069+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3524063940 con 0x7f3524100e40 2024-03-19T00:29:39.071 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.070+0000 7f35137fe700 1 -- 172.21.15.204:0/1981876232 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f351c022220 con 0x7f3524100e40 2024-03-19T00:29:39.072 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:29:39.072 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","modified":"2024-03-19T00:28:00.008746Z","created":"2024-03-19T00:28:00.008746Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-19T00:29:39.073 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f350c039000 msgr2=0x7f350c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:39.073 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f350c039000 0x7f350c03b4c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3518006fd0 tx=0x7f3518006e40 comp rx=0 tx=0).stop 2024-03-19T00:29:39.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 msgr2=0x7f35241976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:39.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f35241976d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f351c0076a0 tx=0x7f351c003ce0 comp rx=0 tx=0).stop 2024-03-19T00:29:39.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 shutdown_connections 2024-03-19T00:29:39.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f350c039000 0x7f350c03b4c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:39.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 --2- 172.21.15.204:0/1981876232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3524100e40 0x7f35241976d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:39.075 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 >> 172.21.15.204:0/1981876232 conn(0x7f3524077850 msgr2=0x7f3524069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:39.075 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 shutdown_connections 2024-03-19T00:29:39.075 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:39.072+0000 7f3528d3b700 1 -- 172.21.15.204:0/1981876232 wait complete. 2024-03-19T00:29:39.075 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-03-19T00:29:39.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:39 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/1981876232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:40.843 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T00:29:40.843 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T00:29:41.092 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:29:44.871 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:29:44.871 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: mon.a calling monitor election 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: mon.c calling monitor election 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: monmap epoch 2 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:44.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: last_changed 2024-03-19T00:29:39.448970+0000 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: min_mon_release 19 (squid) 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: election_strategy: 1 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: fsmap 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: mgrmap e13: x(active, since 48s) 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: overall HEALTH_OK 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:44.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:45.534 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/1909741780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b00035c0 msgr2=0x7fc9b0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:45.535 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/1909741780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b00035c0 0x7fc9b0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4101d10 tx=0x7fc9c000feb0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.535 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/1909741780 shutdown_connections 2024-03-19T00:29:45.535 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/1909741780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b00035c0 0x7fc9b0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.535 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/1909741780 >> 172.21.15.204:0/1909741780 conn(0x7fc9c40fd8b0 msgr2=0x7fc9c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:45.535 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/1909741780 shutdown_connections 2024-03-19T00:29:45.535 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.533+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/1909741780 wait complete. 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.534+0000 7fc9cb4f4700 1 Processor -- start 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.534+0000 7fc9cb4f4700 1 -- start start 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.534+0000 7fc9cb4f4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.534+0000 7fc9cb4f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c419daa0 0x7fc9c4197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.534+0000 7fc9cb4f4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4109d20 con 0x7fc9c419daa0 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.534+0000 7fc9cb4f4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4109ba0 con 0x7fc9b00035c0 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44654/0 (socket says 172.21.15.204:44654) 2024-03-19T00:29:45.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 -- 172.21.15.204:0/495233228 learned_addr learned my addr 172.21.15.204:0/495233228 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 -- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 msgr2=0x7fc9c419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 -- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 msgr2=0x7fc9c419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c9290700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c8a8f700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c419daa0 0x7fc9c4197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c8a8f700 1 -- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 msgr2=0x7fc9c419d560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c8a8f700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c8a8f700 1 -- 172.21.15.204:0/495233228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c419e020 con 0x7fc9c419daa0 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.535+0000 7fc9c8a8f700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c419daa0 0x7fc9c4197630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b4d0 tx=0x7fc9b400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.536+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc9b4011630 con 0x7fc9c419daa0 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.536+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9b4011c80 con 0x7fc9c419daa0 2024-03-19T00:29:45.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.536+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c4197bd0 con 0x7fc9c419daa0 2024-03-19T00:29:45.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.536+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc9b4010da0 con 0x7fc9c419daa0 2024-03-19T00:29:45.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.536+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c4198160 con 0x7fc9c419daa0 2024-03-19T00:29:45.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.537+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7fc9b4018020 con 0x7fc9c419daa0 2024-03-19T00:29:45.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.537+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c4103520 con 0x7fc9c419daa0 2024-03-19T00:29:45.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.537+0000 7fc9ba7fc700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc9a8038d60 0x7fc9a803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:45.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.537+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc9b404bc70 con 0x7fc9c419daa0 2024-03-19T00:29:45.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.537+0000 7fc9c9290700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc9a8038d60 0x7fc9a803b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:45.540 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.538+0000 7fc9c9290700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc9a8038d60 0x7fc9a803b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000fa60 tx=0x7fc9c0005cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:45.543 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.543+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc9b401f020 con 0x7fc9c419daa0 2024-03-19T00:29:45.824 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.823+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc9c41984d0 con 0x7fc9c419daa0 2024-03-19T00:29:45.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:45 smithi204 ceph-mon[66572]: Deploying daemon mon.b on smithi044 2024-03-19T00:29:45.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:45.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.824+0000 7fc9ba7fc700 1 -- 172.21.15.204:0/495233228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fc9b4018570 con 0x7fc9c419daa0 2024-03-19T00:29:45.825 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:29:45.825 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","modified":"2024-03-19T00:29:39.448970Z","created":"2024-03-19T00:28:00.008746Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc9a8038d60 msgr2=0x7fc9a803b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc9a8038d60 0x7fc9a803b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000fa60 tx=0x7fc9c0005cb0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c419daa0 msgr2=0x7fc9c4197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c419daa0 0x7fc9c4197630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b4d0 tx=0x7fc9b400b890 comp rx=0 tx=0).stop 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 shutdown_connections 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc9a8038d60 0x7fc9a803b220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b00035c0 0x7fc9c419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 --2- 172.21.15.204:0/495233228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c419daa0 0x7fc9c4197630 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:45.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 >> 172.21.15.204:0/495233228 conn(0x7fc9c40fd8b0 msgr2=0x7fc9c40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:45.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 shutdown_connections 2024-03-19T00:29:45.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:45.826+0000 7fc9cb4f4700 1 -- 172.21.15.204:0/495233228 wait complete. 2024-03-19T00:29:45.828 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-03-19T00:29:45.883 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:45 smithi007 ceph-mon[62854]: Deploying daemon mon.b on smithi044 2024-03-19T00:29:45.883 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:46.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:46 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/495233228' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:46.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:46 smithi007 ceph-mon[62854]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:46.715 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:29:46 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:29:46.448+0000 7fc8bbb48700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-19T00:29:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:46 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/495233228' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:46 smithi204 ceph-mon[66572]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:47.433 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T00:29:47.433 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T00:29:47.678 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:29:48.055 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:47 smithi044 podman[66346]: 2024-03-19 00:29:47.865107928 +0000 UTC m=+0.017868078 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:29:48.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:47 smithi204 ceph-mon[66572]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 podman[66346]: 2024-03-19 00:29:48.111236327 +0000 UTC m=+0.263996467 container init cbc1839b7815ff81225d8a184503f171452d4b35cb7d100de8f37abd5688c8b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-19T00:29:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 podman[66346]: 2024-03-19 00:29:48.114259604 +0000 UTC m=+0.267019738 container start cbc1839b7815ff81225d8a184503f171452d4b35cb7d100de8f37abd5688c8b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: set uid:gid to 167:167 (ceph:ceph) 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pidfile_write: ignore empty --pid-file 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: load: jerasure load: lrc 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: RocksDB version: 7.9.2 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Git sha 0 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: DB SUMMARY 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: DB Session ID: AZXMDL430QR8CBZVXZPK 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: CURRENT file: CURRENT 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: IDENTITY file: IDENTITY 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-19T00:29:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.error_if_exists: 0 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.create_if_missing: 0 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.paranoid_checks: 1 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.env: 0x559949bf8c60 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.fs: PosixFileSystem 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.info_log: 0x55994b411ae0 2024-03-19T00:29:48.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_file_opening_threads: 16 2024-03-19T00:29:48.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.statistics: (nil) 2024-03-19T00:29:48.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.use_fsync: 0 2024-03-19T00:29:48.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_log_file_size: 0 2024-03-19T00:29:48.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-19T00:29:48.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.keep_log_file_num: 1000 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.recycle_log_file_num: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.allow_fallocate: 1 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.allow_mmap_reads: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.allow_mmap_writes: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.use_direct_reads: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.create_missing_column_families: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.db_log_dir: 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.wal_dir: 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-19T00:29:48.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.advise_random_on_open: 1 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.db_write_buffer_size: 0 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.write_buffer_manager: 0x55994b3e41e0 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.rate_limiter: (nil) 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.wal_recovery_mode: 2 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.enable_thread_tracking: 0 2024-03-19T00:29:48.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.enable_pipelined_write: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.unordered_write: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.row_cache: None 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.wal_filter: None 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.allow_ingest_behind: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.two_write_queues: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.manual_wal_flush: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.wal_compression: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.atomic_flush: 0 2024-03-19T00:29:48.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.log_readahead_size: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.best_efforts_recovery: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.allow_data_in_errors: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.db_host_id: __hostname__ 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_background_jobs: 2 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_background_compactions: -1 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_subcompactions: 1 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-19T00:29:48.364 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_total_wal_size: 0 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_open_files: -1 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bytes_per_sync: 0 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_readahead_size: 0 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_background_flushes: -1 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Compression algorithms supported: 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-19T00:29:48.365 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kZSTD supported: 0 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kXpressCompression supported: 0 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kLZ4HCCompression supported: 1 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kZlibCompression supported: 1 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kSnappyCompression supported: 1 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kLZ4Compression supported: 1 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: kBZip2Compression supported: 0 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.merge_operator: 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_filter: None 2024-03-19T00:29:48.366 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_filter_factory: None 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.sst_partitioner_factory: None 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55994b411be0) 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: index_type: 0 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: data_block_index_type: 0 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: index_shortening: 1 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: checksum: 4 2024-03-19T00:29:48.367 INFO:journalctl@ceph.mon.b.smithi044.stdout: no_block_cache: 0 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache: 0x55994b417090 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache_options: 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: capacity : 536870912 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: num_shard_bits : 4 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: strict_capacity_limit : 0 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_cache_compressed: (nil) 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: persistent_cache: (nil) 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_size: 4096 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_size_deviation: 10 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_restart_interval: 16 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: index_block_restart_interval: 1 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: metadata_block_size: 4096 2024-03-19T00:29:48.368 INFO:journalctl@ceph.mon.b.smithi044.stdout: partition_filters: 0 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: use_delta_encoding: 1 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: filter_policy: bloomfilter 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: whole_key_filtering: 1 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: verify_compression: 0 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: format_version: 5 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: enable_index_compression: 1 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: block_align: 0 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: max_auto_readahead_size: 262144 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: prepopulate_block_cache: 0 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: initial_auto_readahead_size: 8192 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.write_buffer_size: 33554432 2024-03-19T00:29:48.369 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_write_buffer_number: 2 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression: NoCompression 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression: Disabled 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.prefix_extractor: nullptr 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.num_levels: 7 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-19T00:29:48.370 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.level: 32767 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.strategy: 0 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.enabled: false 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-19T00:29:48.371 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.target_file_size_base: 67108864 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-19T00:29:48.372 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.arena_block_size: 1048576 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.disable_auto_compactions: 0 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-19T00:29:48.373 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.table_properties_collectors: 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.inplace_update_support: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.bloom_locality: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.max_successive_merges: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.paranoid_file_checks: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.force_consistency_checks: 1 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.report_bg_io_stats: 0 2024-03-19T00:29:48.374 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.ttl: 2592000 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.enable_blob_files: false 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.min_blob_size: 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.blob_file_size: 268435456 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.blob_file_starting_level: 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-19T00:29:48.375 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f8986bff-37aa-4c82-8139-646a12dd0924 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710808188145688, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710808188146102, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710808188, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f8986bff-37aa-4c82-8139-646a12dd0924", "db_session_id": "AZXMDL430QR8CBZVXZPK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710808188146182, "job": 1, "event": "recovery_finished"} 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55994b50a000 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: DB pointer 0x55994b4f4000 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** DB Stats ** 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T00:29:48.376 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** Compaction Stats [default] ** 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** Compaction Stats [default] ** 2024-03-19T00:29:48.377 INFO:journalctl@ceph.mon.b.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T00:29:48.378 INFO:journalctl@ceph.mon.b.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: Block cache BinnedLRUCache@0x55994b417090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: using public_addr v2:172.21.15.44:0/0 -> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: starting mon.b rank -1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(???) e0 preinit fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).mds e1 new map 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: e1 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T00:29:48.379 INFO:journalctl@ceph.mon.b.smithi044.stdout: legacy client fscid: -1 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout: 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout: No filesystems configured 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:48.380 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Deploying cephadm binary to smithi204 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.381 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Added host smithi204 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3862089139' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3862089139' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T00:29:48.382 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.383 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c;count:3 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Deploying daemon mon.c on smithi204 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.384 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/1981876232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.a calling monitor election 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.c calling monitor election 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: monmap epoch 2 2024-03-19T00:29:48.385 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: last_changed 2024-03-19T00:29:39.448970+0000 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: min_mon_release 19 (squid) 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: election_strategy: 1 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: fsmap 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mgrmap e13: x(active, since 48s) 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: overall HEALTH_OK 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:48.386 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: Deploying daemon mon.b on smithi044 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/495233228' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 ceph-mon[66366]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 bash[66346]: cbc1839b7815ff81225d8a184503f171452d4b35cb7d100de8f37abd5688c8b7 2024-03-19T00:29:48.387 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:48 smithi044 systemd[1]: Started Ceph mon.b for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:29:48.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:47 smithi007 ceph-mon[62854]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:53.499 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.497+0000 7f18669a1700 1 -- 172.21.15.204:0/591956063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18480035c0 msgr2=0x7f1848005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:53.499 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.497+0000 7f18669a1700 1 --2- 172.21.15.204:0/591956063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18480035c0 0x7f1848005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f186006a5d0 tx=0x7f185000fe60 comp rx=0 tx=0).stop 2024-03-19T00:29:53.499 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.497+0000 7f18669a1700 1 -- 172.21.15.204:0/591956063 shutdown_connections 2024-03-19T00:29:53.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.497+0000 7f18669a1700 1 --2- 172.21.15.204:0/591956063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18480035c0 0x7f1848005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.497+0000 7f18669a1700 1 -- 172.21.15.204:0/591956063 >> 172.21.15.204:0/591956063 conn(0x7f18600fc850 msgr2=0x7f18600fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:53.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.498+0000 7f18669a1700 1 -- 172.21.15.204:0/591956063 shutdown_connections 2024-03-19T00:29:53.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.498+0000 7f18669a1700 1 -- 172.21.15.204:0/591956063 wait complete. 2024-03-19T00:29:53.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.498+0000 7f18669a1700 1 Processor -- start 2024-03-19T00:29:53.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.498+0000 7f18669a1700 1 -- start start 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f18669a1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f18669a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186019e5a0 0x7f18601a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f18669a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 0x7f18601a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f18669a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1860110320 con 0x7f186019e5a0 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f18669a1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18601101a0 con 0x7f18601a2e70 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f18669a1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18601104a0 con 0x7f18480035c0 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33064/0 (socket says 172.21.15.204:33064) 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186019e5a0 0x7f18601a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:53.503 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 -- 172.21.15.204:0/4069809389 learned_addr learned my addr 172.21.15.204:0/4069809389 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f1864f3e700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 0x7f18601a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 msgr2=0x7f186019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 msgr2=0x7f186019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f185ffff700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f1864f3e700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 msgr2=0x7f18601a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f1864f3e700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 msgr2=0x7f18601a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f1864f3e700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 0x7f18601a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.499+0000 7f1864f3e700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 0x7f18601a3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185f7fe700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 msgr2=0x7f186019e060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185f7fe700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185f7fe700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 msgr2=0x7f18601a3320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185f7fe700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 0x7f18601a3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185f7fe700 1 -- 172.21.15.204:0/4069809389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18601aabe0 con 0x7f186019e5a0 2024-03-19T00:29:53.504 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185f7fe700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186019e5a0 0x7f18601a2930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f185400b550 tx=0x7f185400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854004230 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18540043d0 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18601aae10 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854011b80 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.500+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18601ab2f0 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.501+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f185401a400 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.501+0000 7f185d7fa700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f184c039450 0x7f184c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.501+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f185404cc10 con 0x7f186019e5a0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.502+0000 7f185ffff700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f184c039450 0x7f184c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:53.505 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.502+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1860063940 con 0x7f186019e5a0 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: mon.c calling monitor election 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: mon.a calling monitor election 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.506 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: monmap epoch 3 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: last_changed 2024-03-19T00:29:48.168753+0000 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: min_mon_release 19 (squid) 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: election_strategy: 1 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: fsmap 2024-03-19T00:29:53.507 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: mgrmap e13: x(active, since 57s) 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: mon.b (rank 2) addr [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] is down (out of quorum) 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:53.508 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:53.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.502+0000 7f185ffff700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f184c039450 0x7f184c03b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1850005b40 tx=0x7f1850005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:53.509 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.507+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1854018750 con 0x7f186019e5a0 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: mon.c calling monitor election 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: mon.a calling monitor election 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: monmap epoch 3 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: last_changed 2024-03-19T00:29:48.168753+0000 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: min_mon_release 19 (squid) 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: election_strategy: 1 2024-03-19T00:29:53.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: fsmap 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: mgrmap e13: x(active, since 57s) 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: mon.b (rank 2) addr [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] is down (out of quorum) 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:53.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:53.795 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.793+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f186002dbb0 con 0x7f186019e5a0 2024-03-19T00:29:53.795 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.794+0000 7f185d7fa700 1 -- 172.21.15.204:0/4069809389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f185401a7e0 con 0x7f186019e5a0 2024-03-19T00:29:53.795 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-19T00:29:53.796 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","modified":"2024-03-19T00:29:48.168753Z","created":"2024-03-19T00:28:00.008746Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f184c039450 msgr2=0x7f184c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f184c039450 0x7f184c03b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1850005b40 tx=0x7f1850005ad0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186019e5a0 msgr2=0x7f18601a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186019e5a0 0x7f18601a2930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f185400b550 tx=0x7f185400b910 comp rx=0 tx=0).stop 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 shutdown_connections 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f184c039450 0x7f184c03b910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18601a2e70 0x7f18601a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.798 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18480035c0 0x7f186019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.799 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 --2- 172.21.15.204:0/4069809389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186019e5a0 0x7f18601a2930 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:53.799 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.796+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 >> 172.21.15.204:0/4069809389 conn(0x7f18600fc850 msgr2=0x7f1860104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:53.799 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.797+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 shutdown_connections 2024-03-19T00:29:53.799 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:53.797+0000 7f18669a1700 1 -- 172.21.15.204:0/4069809389 wait complete. 2024-03-19T00:29:53.800 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-03-19T00:29:54.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:54.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:54.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:54 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/4069809389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:54.367 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:54.393 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-19T00:29:54.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-19T00:29:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:29:54.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:54.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:54.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:54 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/4069809389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:54.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: mon.c calling monitor election 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: mon.a calling monitor election 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: monmap epoch 3 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: last_changed 2024-03-19T00:29:48.168753+0000 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: min_mon_release 19 (squid) 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: election_strategy: 1 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: fsmap 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: mgrmap e13: x(active, since 57s) 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: mon.b (rank 2) addr [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] is down (out of quorum) 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:55.609 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:29:55.610 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/4069809389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T00:29:55.610 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:56.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 -- 172.21.15.7:0/3197421729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048100e40 msgr2=0x7fc048101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 --2- 172.21.15.7:0/3197421729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048100e40 0x7fc048101220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc038009910 tx=0x7fc038009c20 comp rx=0 tx=0).stop 2024-03-19T00:29:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 -- 172.21.15.7:0/3197421729 shutdown_connections 2024-03-19T00:29:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 --2- 172.21.15.7:0/3197421729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048100e40 0x7fc048101220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 -- 172.21.15.7:0/3197421729 >> 172.21.15.7:0/3197421729 conn(0x7fc048077850 msgr2=0x7fc0480feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:56.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 -- 172.21.15.7:0/3197421729 shutdown_connections 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 -- 172.21.15.7:0/3197421729 wait complete. 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 Processor -- start 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.269+0000 7fc04ea29700 1 -- start start 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04ea29700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc048100e40 0x7fc04819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04ea29700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04819fd80 0x7fc048199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04ea29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 0x7fc04819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04ea29700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc04810b0c0 con 0x7fc048199f40 2024-03-19T00:29:56.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04ea29700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc04810af40 con 0x7fc048100e40 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04ea29700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc04810b240 con 0x7fc04819fd80 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 0x7fc04819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 0x7fc04819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54606/0 (socket says 172.21.15.7:54606) 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 -- 172.21.15.7:0/1004226173 learned_addr learned my addr 172.21.15.7:0/1004226173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc047fff700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc048100e40 0x7fc04819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc0477fe700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04819fd80 0x7fc048199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 -- 172.21.15.7:0/1004226173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04819fd80 msgr2=0x7fc048199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:56.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04819fd80 0x7fc048199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 -- 172.21.15.7:0/1004226173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc048100e40 msgr2=0x7fc04819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:56.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc048100e40 0x7fc04819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 -- 172.21.15.7:0/1004226173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0481a6710 con 0x7fc048199f40 2024-03-19T00:29:56.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.270+0000 7fc04cfc6700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 0x7fc04819a3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc04000ba50 tx=0x7fc04000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:56.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.271+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040019070 con 0x7fc048199f40 2024-03-19T00:29:56.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.271+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc04000cde0 con 0x7fc048199f40 2024-03-19T00:29:56.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.271+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0481a6940 con 0x7fc048199f40 2024-03-19T00:29:56.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.271+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040007350 con 0x7fc048199f40 2024-03-19T00:29:56.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.271+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0481a6e30 con 0x7fc048199f40 2024-03-19T00:29:56.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.271+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7fc04001c070 con 0x7fc048199f40 2024-03-19T00:29:56.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.272+0000 7fc0457fa700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc030039450 0x7fc03003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:56.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.272+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc04004b7b0 con 0x7fc048199f40 2024-03-19T00:29:56.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.272+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc048102320 con 0x7fc048199f40 2024-03-19T00:29:56.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.272+0000 7fc047fff700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc030039450 0x7fc03003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:56.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.272+0000 7fc047fff700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc030039450 0x7fc03003b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc038006010 tx=0x7fc03801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:56.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.277+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc040014440 con 0x7fc048199f40 2024-03-19T00:29:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: mon.b calling monitor election 2024-03-19T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-19T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-19T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-19T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: mon.b calling monitor election 2024-03-19T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: mon.c calling monitor election 2024-03-19T00:29:56.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: mon.a calling monitor election 2024-03-19T00:29:56.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T00:29:56.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: monmap epoch 3 2024-03-19T00:29:56.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:56.358 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: last_changed 2024-03-19T00:29:48.168753+0000 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: min_mon_release 19 (squid) 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: election_strategy: 1 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-03-19T00:29:56.359 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: fsmap 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: mgrmap e13: x(active, since 59s) 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Cluster is now healthy 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: overall HEALTH_OK 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Updating smithi007:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.360 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.361 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.362 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: Reconfiguring daemon mon.a on smithi007 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:56.363 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: mon.b calling monitor election 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: mon.b calling monitor election 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: mon.c calling monitor election 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: mon.a calling monitor election 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: monmap epoch 3 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:56.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: last_changed 2024-03-19T00:29:48.168753+0000 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: min_mon_release 19 (squid) 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: election_strategy: 1 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: fsmap 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: mgrmap e13: x(active, since 59s) 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Cluster is now healthy 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: overall HEALTH_OK 2024-03-19T00:29:56.424 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Updating smithi007:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.425 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T00:29:56.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:56.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:29:56.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:56.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: Reconfiguring daemon mon.a on smithi007 2024-03-19T00:29:56.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:56.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.488+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc0481a70c0 con 0x7fc048199f40 2024-03-19T00:29:56.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.488+0000 7fc0457fa700 1 -- 172.21.15.7:0/1004226173 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fc040012c60 con 0x7fc048199f40 2024-03-19T00:29:56.489 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:56.489 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-03-19T00:29:56.489 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:56.489 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc030039450 msgr2=0x7fc03003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc030039450 0x7fc03003b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc038006010 tx=0x7fc03801a040 comp rx=0 tx=0).stop 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 msgr2=0x7fc04819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 0x7fc04819a3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc04000ba50 tx=0x7fc04000bd60 comp rx=0 tx=0).stop 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 shutdown_connections 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc030039450 0x7fc03003b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc048199f40 0x7fc04819a3f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc048100e40 0x7fc04819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 --2- 172.21.15.7:0/1004226173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04819fd80 0x7fc048199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:56.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 >> 172.21.15.7:0/1004226173 conn(0x7fc048077850 msgr2=0x7fc048104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:56.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 shutdown_connections 2024-03-19T00:29:56.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:29:56.490+0000 7fc04ea29700 1 -- 172.21.15.7:0/1004226173 wait complete. 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: mon.b calling monitor election 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: mon.b calling monitor election 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: mon.c calling monitor election 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: mon.a calling monitor election 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: monmap epoch 3 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: last_changed 2024-03-19T00:29:48.168753+0000 2024-03-19T00:29:56.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: created 2024-03-19T00:28:00.008746+0000 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: min_mon_release 19 (squid) 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: election_strategy: 1 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: fsmap 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: mgrmap e13: x(active, since 59s) 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Cluster is now healthy 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: overall HEALTH_OK 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Updating smithi007:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Updating smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Updating smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.517 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:56.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: Reconfiguring daemon mon.a on smithi007 2024-03-19T00:29:56.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:56.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:57.177 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:57.178 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1004226173' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1004226173' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:29:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1004226173' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:57.626 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-19T00:29:57.626 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:29:57.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T00:29:57.656 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:29:57.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:57.724 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:29:57.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T00:29:57.759 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:29:57.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:57.831 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:29:57.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T00:29:57.865 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:29:57.865 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:29:57.936 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-03-19T00:29:57.936 INFO:tasks.cephadm:Adding mgr.y on smithi044 2024-03-19T00:29:57.936 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-03-19T00:29:57.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi007=x;smithi044=y;smithi204=z' 2024-03-19T00:29:58.213 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:29:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:58 smithi204 ceph-mon[66572]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:29:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:58 smithi044 ceph-mon[66366]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:29:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:58.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:58 smithi007 ceph-mon[62854]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:29:58.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:58.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:29:58.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:29:58.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:29:58.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:29:59.101 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:59 smithi204 ceph-mon[66572]: Reconfiguring mon.b (monmap changed)... 2024-03-19T00:29:59.101 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:29:59 smithi204 ceph-mon[66572]: Reconfiguring daemon mon.b on smithi044 2024-03-19T00:29:59.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:59 smithi044 ceph-mon[66366]: Reconfiguring mon.b (monmap changed)... 2024-03-19T00:29:59.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:29:59 smithi044 ceph-mon[66366]: Reconfiguring daemon mon.b on smithi044 2024-03-19T00:29:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:59 smithi007 ceph-mon[62854]: Reconfiguring mon.b (monmap changed)... 2024-03-19T00:29:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:29:59 smithi007 ceph-mon[62854]: Reconfiguring daemon mon.b on smithi044 2024-03-19T00:29:59.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 -- 172.21.15.204:0/3567705261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d41034c0 msgr2=0x7f05d41038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 --2- 172.21.15.204:0/3567705261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d41034c0 0x7f05d41038e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00b0a0 tx=0x7f05cc00b3b0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 -- 172.21.15.204:0/3567705261 shutdown_connections 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 --2- 172.21.15.204:0/3567705261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d41034c0 0x7f05d41038e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 -- 172.21.15.204:0/3567705261 >> 172.21.15.204:0/3567705261 conn(0x7f05d40fd400 msgr2=0x7f05d40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 -- 172.21.15.204:0/3567705261 shutdown_connections 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.597+0000 7f05d942e700 1 -- 172.21.15.204:0/3567705261 wait complete. 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 Processor -- start 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 -- start start 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d41034c0 0x7f05d40760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 0x7f05d4074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d40750c0 0x7f05d41a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:59.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05d4107de0 con 0x7f05d40750c0 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f05d4107c60 con 0x7f05d41034c0 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.598+0000 7f05d942e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05d4107f60 con 0x7f05d4074700 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 0x7f05d4074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 0x7f05d4074b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:49040/0 (socket says 172.21.15.204:49040) 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 -- 172.21.15.204:0/2025920067 learned_addr learned my addr 172.21.15.204:0/2025920067 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d37fe700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d40750c0 0x7f05d41a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d2ffd700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d41034c0 0x7f05d40760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 -- 172.21.15.204:0/2025920067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d41034c0 msgr2=0x7f05d40760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d41034c0 0x7f05d40760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 -- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d40750c0 msgr2=0x7f05d41a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d40750c0 0x7f05d41a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 -- 172.21.15.204:0/2025920067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d41a1c20 con 0x7f05d4074700 2024-03-19T00:29:59.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d37fe700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d40750c0 0x7f05d41a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:29:59.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d27fc700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 0x7f05d4074b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b780 tx=0x7f05bc00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:59.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004230 con 0x7f05d4074700 2024-03-19T00:29:59.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05bc0043d0 con 0x7f05d4074700 2024-03-19T00:29:59.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d41a1f10 con 0x7f05d4074700 2024-03-19T00:29:59.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc011b60 con 0x7f05d4074700 2024-03-19T00:29:59.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.599+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d41a2440 con 0x7f05d4074700 2024-03-19T00:29:59.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.600+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f05bc011d00 con 0x7f05d4074700 2024-03-19T00:29:59.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.600+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d4063940 con 0x7f05d4074700 2024-03-19T00:29:59.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.600+0000 7f05cbfff700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05c0039450 0x7f05c003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:29:59.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.600+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f05bc04bb30 con 0x7f05d4074700 2024-03-19T00:29:59.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.601+0000 7f05d2ffd700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05c0039450 0x7f05c003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:29:59.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.601+0000 7f05d2ffd700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05c0039450 0x7f05c003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f05cc003fd0 tx=0x7f05cc005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:29:59.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.606+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05bc020500 con 0x7f05d4074700 2024-03-19T00:29:59.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.821+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi044=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f05d40628a0 con 0x7f05c0039450 2024-03-19T00:29:59.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.827+0000 7f05cbfff700 1 -- 172.21.15.204:0/2025920067 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f05d40628a0 con 0x7f05c0039450 2024-03-19T00:29:59.828 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05c0039450 msgr2=0x7f05c003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05c0039450 0x7f05c003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f05cc003fd0 tx=0x7f05cc005040 comp rx=0 tx=0).stop 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 msgr2=0x7f05d4074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 0x7f05d4074b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b780 tx=0x7f05bc00ba90 comp rx=0 tx=0).stop 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 shutdown_connections 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05c0039450 0x7f05c003b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05d40750c0 0x7f05d41a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d41034c0 0x7f05d40760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 --2- 172.21.15.204:0/2025920067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05d4074700 0x7f05d4074b80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.829+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 >> 172.21.15.204:0/2025920067 conn(0x7f05d40fd400 msgr2=0x7f05d40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:29:59.831 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.830+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 shutdown_connections 2024-03-19T00:29:59.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:29:59.830+0000 7f05d942e700 1 -- 172.21.15.204:0/2025920067 wait complete. 2024-03-19T00:30:00.444 DEBUG:teuthology.orchestra.run.smithi044:mgr.y> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.y.service 2024-03-19T00:30:00.447 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.z.service 2024-03-19T00:30:00.449 INFO:tasks.cephadm:Deploying OSDs... 2024-03-19T00:30:00.449 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:30:00.449 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T00:30:00.468 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T00:30:00.469 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-19 00:29:05.257758646 +0000 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-19 00:00:48.618542901 +0000 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-19 00:00:48.618542901 +0000 2024-03-19T00:30:00.526 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-19 00:00:48.618542901 +0000 2024-03-19T00:30:00.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T00:30:00.598 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-19T00:30:00.599 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-19T00:30:00.599 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000122719 s, 4.2 MB/s 2024-03-19T00:30:00.600 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T00:30:00.659 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: Reconfiguring mon.c (monmap changed)... 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: Reconfiguring daemon mon.c on smithi204 2024-03-19T00:30:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi044=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: Saving service mgr spec with placement smithi007=x;smithi044=y;smithi204=z;count:3 2024-03-19T00:30:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:00 smithi204 ceph-mon[66572]: overall HEALTH_OK 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-19 00:29:05.284758260 +0000 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-19 00:00:48.962538208 +0000 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-19 00:00:48.962538208 +0000 2024-03-19T00:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-19 00:00:48.962538208 +0000 2024-03-19T00:30:00.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T00:30:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-19T00:30:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-19T00:30:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000101911 s, 5.0 MB/s 2024-03-19T00:30:00.785 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T00:30:00.841 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-03-19T00:30:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: Reconfiguring mon.c (monmap changed)... 2024-03-19T00:30:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:30:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: Reconfiguring daemon mon.c on smithi204 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi044=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: Saving service mgr spec with placement smithi007=x;smithi044=y;smithi204=z;count:3 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:00.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:00 smithi044 ceph-mon[66366]: overall HEALTH_OK 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-19 00:29:05.311757875 +0000 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-19 00:00:49.290533733 +0000 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-19 00:00:49.290533733 +0000 2024-03-19T00:30:00.898 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-19 00:00:49.290533733 +0000 2024-03-19T00:30:00.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T00:30:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: Reconfiguring mon.c (monmap changed)... 2024-03-19T00:30:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T00:30:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T00:30:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: Reconfiguring daemon mon.c on smithi204 2024-03-19T00:30:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi044=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: Saving service mgr spec with placement smithi007=x;smithi044=y;smithi204=z;count:3 2024-03-19T00:30:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:00 smithi007 ceph-mon[62854]: overall HEALTH_OK 2024-03-19T00:30:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-19T00:30:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-19T00:30:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.2886e-05 s, 5.5 MB/s 2024-03-19T00:30:00.968 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T00:30:01.024 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-03-19T00:30:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T00:30:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:01.080 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-19T00:30:01.080 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:01.080 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:01.080 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-19 00:29:05.337757503 +0000 2024-03-19T00:30:01.081 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-19 00:00:49.602529475 +0000 2024-03-19T00:30:01.081 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-19 00:00:49.602529475 +0000 2024-03-19T00:30:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-19 00:00:49.602529475 +0000 2024-03-19T00:30:01.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T00:30:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-19T00:30:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-19T00:30:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000113438 s, 4.5 MB/s 2024-03-19T00:30:01.149 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T00:30:01.206 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-19T00:30:01.206 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T00:30:01.224 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T00:30:01.225 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-19 00:29:47.675551533 +0000 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-19 00:07:19.271013533 +0000 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-19 00:07:19.271013533 +0000 2024-03-19T00:30:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-19 00:07:19.271013533 +0000 2024-03-19T00:30:01.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T00:30:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-19T00:30:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-19T00:30:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000208766 s, 2.5 MB/s 2024-03-19T00:30:01.353 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T00:30:01.411 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-03-19T00:30:01.468 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T00:30:01.468 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-19 00:29:47.676551518 +0000 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-19 00:07:19.627008274 +0000 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-19 00:07:19.627008274 +0000 2024-03-19T00:30:01.469 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-19 00:07:19.627008274 +0000 2024-03-19T00:30:01.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T00:30:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-19T00:30:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-19T00:30:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000191465 s, 2.7 MB/s 2024-03-19T00:30:01.538 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T00:30:01.594 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-03-19T00:30:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T00:30:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:01.649 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-19T00:30:01.650 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:01.650 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:01.650 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-19 00:29:47.676551518 +0000 2024-03-19T00:30:01.650 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-19 00:07:19.978003088 +0000 2024-03-19T00:30:01.650 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-19 00:07:19.978003088 +0000 2024-03-19T00:30:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-19 00:07:19.978003088 +0000 2024-03-19T00:30:01.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T00:30:01.716 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-19T00:30:01.716 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-19T00:30:01.716 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000186886 s, 2.7 MB/s 2024-03-19T00:30:01.717 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T00:30:01.774 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-19 00:29:47.677551503 +0000 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-19 00:07:20.329997888 +0000 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-19 00:07:20.329997888 +0000 2024-03-19T00:30:01.829 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-19 00:07:20.329997888 +0000 2024-03-19T00:30:01.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T00:30:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-19T00:30:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-19T00:30:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00028596 s, 1.8 MB/s 2024-03-19T00:30:01.897 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T00:30:01.953 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-19T00:30:01.953 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T00:30:01.970 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T00:30:01.970 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-03-19T00:30:02.027 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-19 00:29:38.526497522 +0000 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-19 00:07:12.377241258 +0000 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-19 00:07:12.377241258 +0000 2024-03-19T00:30:02.028 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-19 00:07:12.377241258 +0000 2024-03-19T00:30:02.028 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T00:30:02.097 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-19T00:30:02.097 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-19T00:30:02.097 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000257636 s, 2.0 MB/s 2024-03-19T00:30:02.099 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T00:30:02.155 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-19 00:29:38.526497522 +0000 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-19 00:07:12.734236072 +0000 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-19 00:07:12.734236072 +0000 2024-03-19T00:30:02.210 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-19 00:07:12.734236072 +0000 2024-03-19T00:30:02.211 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T00:30:02.276 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-19T00:30:02.277 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-19T00:30:02.277 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000212168 s, 2.4 MB/s 2024-03-19T00:30:02.278 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T00:30:02.335 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-03-19T00:30:02.390 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T00:30:02.390 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-19 00:29:38.526497522 +0000 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-19 00:07:13.059231350 +0000 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-19 00:07:13.059231350 +0000 2024-03-19T00:30:02.391 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-19 00:07:13.059231350 +0000 2024-03-19T00:30:02.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T00:30:02.456 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-19T00:30:02.456 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-19T00:30:02.456 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000192225 s, 2.7 MB/s 2024-03-19T00:30:02.457 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T00:30:02.514 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-19 00:29:38.527497508 +0000 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-19 00:07:13.378226715 +0000 2024-03-19T00:30:02.571 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-19 00:07:13.378226715 +0000 2024-03-19T00:30:02.572 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-19 00:07:13.378226715 +0000 2024-03-19T00:30:02.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T00:30:02.637 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-19T00:30:02.638 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-19T00:30:02.638 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000198202 s, 2.6 MB/s 2024-03-19T00:30:02.639 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T00:30:02.695 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-03-19T00:30:02.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:02.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T00:30:02.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:02.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:02.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:02.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T00:30:02.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:02.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:02.942 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:30:03.689 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:03 smithi204 ceph-mon[66572]: Deploying daemon mgr.z on smithi204 2024-03-19T00:30:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:03 smithi007 ceph-mon[62854]: Deploying daemon mgr.z on smithi204 2024-03-19T00:30:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:03 smithi044 ceph-mon[66366]: Deploying daemon mgr.z on smithi204 2024-03-19T00:30:04.015 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:03 smithi204 systemd[1]: Starting Ceph mgr.z for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:30:04.399 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 podman[68414]: 2024-03-19 00:30:04.028168914 +0000 UTC m=+0.018415510 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:30:04.399 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 podman[68414]: 2024-03-19 00:30:04.130244649 +0000 UTC m=+0.120491224 container create 8ac1793993d0fbf0ad0855c9a609cf9b7e32894c12e92a57599656a6177fd95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:30:04.400 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 podman[68414]: 2024-03-19 00:30:04.315845758 +0000 UTC m=+0.306092350 container init 8ac1793993d0fbf0ad0855c9a609cf9b7e32894c12e92a57599656a6177fd95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:30:04.400 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 podman[68414]: 2024-03-19 00:30:04.318477283 +0000 UTC m=+0.308723867 container start 8ac1793993d0fbf0ad0855c9a609cf9b7e32894c12e92a57599656a6177fd95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0) 2024-03-19T00:30:04.400 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 ceph-mgr[68433]: -- 172.21.15.204:0/1084422004 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e45a5e05a0 con 0x55e45a5b4c00 2024-03-19T00:30:04.400 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 bash[68414]: 8ac1793993d0fbf0ad0855c9a609cf9b7e32894c12e92a57599656a6177fd95f 2024-03-19T00:30:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:04 smithi007 ceph-mon[62854]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:04.773 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:04 smithi204 ceph-mon[66572]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:04.774 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 systemd[1]: Started Ceph mgr.z for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:30:04.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:04 smithi044 ceph-mon[66366]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T00:30:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T00:30:04.968 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-19T00:30:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-19T00:30:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0089904 s, 1.2 GB/s 2024-03-19T00:30:04.969 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 2024-03-19T00:30:04.969 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-19T00:30:05.265 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:04.868+0000 7fe7f00c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:30:05.265 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:04.960+0000 7fe7f00c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:30:05.600 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:05.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-mon[66572]: Deploying daemon mgr.y on smithi044 2024-03-19T00:30:05.630 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.630 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.630 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:05.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T00:30:05.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:05.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:05.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:05 smithi007 ceph-mon[62854]: Deploying daemon mgr.y on smithi044 2024-03-19T00:30:05.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-03-19T00:30:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:05.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:05.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T00:30:05.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:05.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:05.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:05 smithi044 ceph-mon[66366]: Deploying daemon mgr.y on smithi044 2024-03-19T00:30:05.926 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:05.599+0000 7fe7f00c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:30:05.926 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:05.691+0000 7fe7f00c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:30:05.926 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:05.831+0000 7fe7f00c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:30:05.939 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:30:06.265 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:05.925+0000 7fe7f00c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:30:06.265 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:06.007+0000 7fe7f00c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:30:06.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-mon[66572]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:06.540 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:06.268+0000 7fe7f00c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:30:06.541 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:06.383+0000 7fe7f00c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:30:06.541 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:06.472+0000 7fe7f00c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:30:06.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:06 smithi044 ceph-mon[66366]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:06.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:06 smithi007 ceph-mon[62854]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:07.015 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:06.573+0000 7fe7f00c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:30:07.015 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:06.758+0000 7fe7f00c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:30:07.048 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:06 smithi044 systemd[1]: Started Ceph mgr.y for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:30:07.606 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:07.400+0000 7f8ad3eae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:30:07.606 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:07.501+0000 7f8ad3eae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 -- 172.21.15.7:0/1642242939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c102440 msgr2=0x7f726c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 --2- 172.21.15.7:0/1642242939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c102440 0x7f726c1028a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f72600099c0 tx=0x7f7260009cd0 comp rx=0 tx=0).stop 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 -- 172.21.15.7:0/1642242939 shutdown_connections 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 --2- 172.21.15.7:0/1642242939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f726c102de0 0x7f726c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 --2- 172.21.15.7:0/1642242939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726c101240 0x7f726c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 --2- 172.21.15.7:0/1642242939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c102440 0x7f726c1028a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 -- 172.21.15.7:0/1642242939 >> 172.21.15.7:0/1642242939 conn(0x7f726c0fc7e0 msgr2=0x7f726c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 -- 172.21.15.7:0/1642242939 shutdown_connections 2024-03-19T00:30:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 -- 172.21.15.7:0/1642242939 wait complete. 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.612+0000 7f7270877700 1 Processor -- start 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 -- start start 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 0x7f726c1051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726c102440 0x7f726c103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f726c102de0 0x7f726c103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f726c10ccd0 con 0x7f726c101240 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f726c10cb50 con 0x7f726c102440 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f726c10ce50 con 0x7f726c102de0 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 0x7f726c1051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f726c102de0 0x7f726c103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 0x7f726c1051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44602/0 (socket says 172.21.15.7:44602) 2024-03-19T00:30:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 -- 172.21.15.7:0/1673239751 learned_addr learned my addr 172.21.15.7:0/1673239751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7269d9b700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726c102440 0x7f726c103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 -- 172.21.15.7:0/1673239751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f726c102de0 msgr2=0x7f726c103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f726c102de0 0x7f726c103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 -- 172.21.15.7:0/1673239751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726c102440 msgr2=0x7f726c103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726c102440 0x7f726c103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 -- 172.21.15.7:0/1673239751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c104610 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f726a59c700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 0x7f726c1051d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e970 tx=0x7f725c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c004220 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c1a2640 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.613+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f725c0043c0 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.614+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c003d50 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.614+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c1a2b80 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.614+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46161+0+0 (secure 0 0 0) 0x7f725c018430 con 0x7f726c101240 2024-03-19T00:30:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.614+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726c068fd0 con 0x7f726c101240 2024-03-19T00:30:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.615+0000 7f725b7fe700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7254039200 0x7f725403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.615+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f725c04c4f0 con 0x7f726c101240 2024-03-19T00:30:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.615+0000 7f7269d9b700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7254039200 0x7f725403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.615+0000 7f7269d9b700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7254039200 0x7f725403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7260000c00 tx=0x7f7260005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:30:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.620+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f725c01fbb0 con 0x7f726c101240 2024-03-19T00:30:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:07.835+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f726c0fea00 con 0x7f7254039200 2024-03-19T00:30:08.076 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:07.798+0000 7fe7f00c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:07.930+0000 7fe7f00c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:08.077 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:08.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:08.219 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:08.220 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:08.379 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:08.274+0000 7fe7f00c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:30:08.379 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:08.378+0000 7fe7f00c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:30:08.568 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:08.217+0000 7f8ad3eae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:30:08.568 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:08.315+0000 7f8ad3eae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:30:08.568 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:08.460+0000 7f8ad3eae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:30:08.568 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:08.566+0000 7f8ad3eae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:30:08.765 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:08.574+0000 7fe7f00c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:30:08.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:08.653+0000 7f8ad3eae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:08.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:09.053 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:08.766+0000 7fe7f00c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:30:09.053 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:09.054 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:09.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T00:30:09.223 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T00:30:09.223 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:09.223 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:08.921+0000 7f8ad3eae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:30:09.223 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:09.034+0000 7f8ad3eae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:30:09.223 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:09.123+0000 7f8ad3eae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:30:09.223 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:09.220+0000 7f8ad3eae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:30:09.315 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:09.052+0000 7fe7f00c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:30:09.315 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:09.314+0000 7fe7f00c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:30:09.606 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:09.398+0000 7f8ad3eae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:30:09.765 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:09.548+0000 7fe7f00c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:30:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:09 smithi007 ceph-mon[62854]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:09 smithi007 ceph-mon[62854]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T00:30:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:09 smithi007 ceph-mon[62854]: Reconfiguring daemon mgr.x on smithi007 2024-03-19T00:30:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:09 smithi007 ceph-mon[62854]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-mon[66572]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-mon[66572]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T00:30:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-mon[66572]: Reconfiguring daemon mgr.x on smithi007 2024-03-19T00:30:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:09 smithi204 ceph-mon[66572]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-mon[66366]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-mon[66366]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T00:30:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-mon[66366]: Reconfiguring daemon mgr.x on smithi007 2024-03-19T00:30:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:09 smithi044 ceph-mon[66366]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:10.765 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:10.351+0000 7fe7f00c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:30:10.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:10.395+0000 7f8ad3eae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:30:10.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:10.529+0000 7f8ad3eae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:30:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2123473878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea"}]: dispatch 2024-03-19T00:30:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2123473878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea"}]': finished 2024-03-19T00:30:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:10 smithi007 ceph-mon[62854]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T00:30:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:11.180 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2123473878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea"}]: dispatch 2024-03-19T00:30:11.180 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2123473878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea"}]': finished 2024-03-19T00:30:11.180 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-mon[66366]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T00:30:11.180 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:11.181 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:10.887+0000 7f8ad3eae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:30:11.181 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:10.978+0000 7f8ad3eae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:30:11.181 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:11.179+0000 7f8ad3eae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:10 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2123473878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea"}]: dispatch 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:10 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2123473878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea"}]': finished 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:10 smithi204 ceph-mon[66572]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:10 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:10.923+0000 7fe7f00c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:11.023+0000 7fe7f00c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:30:11.245 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z[68429]: 2024-03-19T00:30:11.244+0000 7fe7f00c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:30:11.606 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:11.368+0000 7f8ad3eae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:30:11.920 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:11.658+0000 7f8ad3eae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:30:11.920 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:11.920+0000 7f8ad3eae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:30:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:11.986+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 14) ==== 91697+0+0 (secure 0 0 0) 0x7f725c017cd0 con 0x7f726c101240 2024-03-19T00:30:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1409533192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: Standby manager daemon z started 2024-03-19T00:30:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T00:30:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T00:30:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T00:30:12.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:11 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:12.163+0000 7f8ad3eae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1409533192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: Standby manager daemon z started 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T00:30:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:11 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1409533192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: Standby manager daemon z started 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T00:30:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:11 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.204:0/2564464164' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T00:30:13.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:12 smithi204 ceph-mon[66572]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:13.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:12 smithi204 ceph-mon[66572]: mgrmap e14: x(active, since 76s), standbys: z 2024-03-19T00:30:13.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:12 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T00:30:13.356 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:12.990+0000 7f8ad3eae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:30:13.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:12 smithi044 ceph-mon[66366]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:13.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:12 smithi044 ceph-mon[66366]: mgrmap e14: x(active, since 76s), standbys: z 2024-03-19T00:30:13.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:12 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T00:30:13.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:12 smithi007 ceph-mon[62854]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:13.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:12 smithi007 ceph-mon[62854]: mgrmap e14: x(active, since 76s), standbys: z 2024-03-19T00:30:13.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:12 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T00:30:13.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:13.576+0000 7f8ad3eae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:30:13.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:13.676+0000 7f8ad3eae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:30:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:13.995+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 15) ==== 137233+0+0 (secure 0 0 0) 0x7f725c00f3c0 con 0x7f726c101240 2024-03-19T00:30:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:13 smithi204 ceph-mon[66572]: Standby manager daemon y started 2024-03-19T00:30:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:13 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T00:30:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:13 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T00:30:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:13 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T00:30:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:13 smithi204 ceph-mon[66572]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T00:30:14.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:13 smithi007 ceph-mon[62854]: Standby manager daemon y started 2024-03-19T00:30:14.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:13 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T00:30:14.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:13 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T00:30:14.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:13 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T00:30:14.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:13 smithi007 ceph-mon[62854]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T00:30:14.356 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y[67741]: 2024-03-19T00:30:13.896+0000 7f8ad3eae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:30:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-mon[66366]: Standby manager daemon y started 2024-03-19T00:30:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T00:30:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T00:30:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T00:30:14.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:13 smithi044 ceph-mon[66366]: from='mgr.? 172.21.15.44:0/3902629149' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T00:30:15.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:14 smithi007 ceph-mon[62854]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:15.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:14 smithi007 ceph-mon[62854]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-03-19T00:30:15.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:14 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T00:30:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:14 smithi204 ceph-mon[66572]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:14 smithi204 ceph-mon[66572]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-03-19T00:30:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:14 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T00:30:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:14 smithi044 ceph-mon[66366]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:14 smithi044 ceph-mon[66366]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-03-19T00:30:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:14 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T00:30:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:16.988 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:16.988 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:16.988 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:16.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:16.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:16.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:17.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:17 smithi007 ceph-mon[62854]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:18.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:17 smithi204 ceph-mon[66572]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:17 smithi044 ceph-mon[66366]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T00:30:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:18 smithi007 ceph-mon[62854]: Deploying daemon osd.0 on smithi007 2024-03-19T00:30:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T00:30:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:18 smithi204 ceph-mon[66572]: Deploying daemon osd.0 on smithi007 2024-03-19T00:30:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T00:30:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:18 smithi044 ceph-mon[66366]: Deploying daemon osd.0 on smithi007 2024-03-19T00:30:19.847 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:19 smithi007 ceph-mon[62854]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:20.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:19 smithi204 ceph-mon[66572]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:19 smithi044 ceph-mon[66366]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:21 smithi007 ceph-mon[62854]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:21 smithi204 ceph-mon[66572]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:21 smithi044 ceph-mon[66366]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:22.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:22 smithi007 ceph-mon[62854]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:23.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:22 smithi204 ceph-mon[66572]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:23.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:22 smithi044 ceph-mon[66366]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:24.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:24.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:24.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:24.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:24.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:24.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:25.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:25 smithi044 ceph-mon[66366]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:25.423 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:25 smithi007 ceph-mon[62854]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:25.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:25 smithi204 ceph-mon[66572]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:26.304 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:26 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.305 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:26 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.305 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:26 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:26.305 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:26 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:26.305 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:26 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:26 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:26 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:26 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:26 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:26 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.602+0000 7f725b7fe700 1 -- 172.21.15.7:0/1673239751 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f726c0fea00 con 0x7f7254039200 2024-03-19T00:30:26.603 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-03-19T00:30:26.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7254039200 msgr2=0x7f725403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:26.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7254039200 0x7f725403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7260000c00 tx=0x7f7260005fb0 comp rx=0 tx=0).stop 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 msgr2=0x7f726c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 0x7f726c1051d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e970 tx=0x7f725c00ec80 comp rx=0 tx=0).stop 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 shutdown_connections 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7254039200 0x7f725403b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f726c102de0 0x7f726c103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c101240 0x7f726c1051d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 --2- 172.21.15.7:0/1673239751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726c102440 0x7f726c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.604+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 >> 172.21.15.7:0/1673239751 conn(0x7f726c0fc7e0 msgr2=0x7f726c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.605+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 shutdown_connections 2024-03-19T00:30:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:30:26.605+0000 7f7270877700 1 -- 172.21.15.7:0/1673239751 wait complete. 2024-03-19T00:30:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:26 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:26 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:26 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:26 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:26 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:27.218 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.0.service 2024-03-19T00:30:27.220 INFO:tasks.cephadm:Deploying osd.1 on smithi044 with /dev/vg_nvme/lv_4... 2024-03-19T00:30:27.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T00:30:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:30:27.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:27 smithi044 ceph-mon[66366]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:27.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:27.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:27.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:27.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:27 smithi044 ceph-mon[66366]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T00:30:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:27 smithi007 ceph-mon[62854]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:27 smithi007 ceph-mon[62854]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T00:30:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:27 smithi204 ceph-mon[66572]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:27 smithi204 ceph-mon[66572]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T00:30:28.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:28 smithi007 ceph-mon[62854]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T00:30:28.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:28 smithi007 ceph-mon[62854]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T00:30:28.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:28 smithi007 ceph-mon[62854]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:30:28.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:28 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:28.718 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:30:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0[74813]: 2024-03-19T00:30:28.599+0000 7f77218ca700 -1 osd.0 0 waiting for initial osdmap 2024-03-19T00:30:28.719 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:30:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0[74813]: 2024-03-19T00:30:28.602+0000 7f771d6bd700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:30:28.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:28 smithi044 ceph-mon[66366]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T00:30:28.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:28 smithi044 ceph-mon[66366]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T00:30:28.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:28 smithi044 ceph-mon[66366]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:30:28.816 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:28 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:29.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:28 smithi204 ceph-mon[66572]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T00:30:29.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:28 smithi204 ceph-mon[66572]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T00:30:29.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:28 smithi204 ceph-mon[66572]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:30:29.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:28 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T00:30:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T00:30:29.465 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-19T00:30:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-19T00:30:29.465 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109208 s, 960 MB/s 2024-03-19T00:30:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-19T00:30:29.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:29 smithi044 ceph-mon[66366]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:29.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:29 smithi044 ceph-mon[66366]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:30:29.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:29 smithi044 ceph-mon[66366]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T00:30:29.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:29.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:29.890 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:29 smithi007 ceph-mon[62854]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:29.890 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:29 smithi007 ceph-mon[62854]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:30:29.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:29 smithi007 ceph-mon[62854]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T00:30:29.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:29.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:29 smithi204 ceph-mon[66572]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:30:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:29 smithi204 ceph-mon[66572]: from='osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:30:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:29 smithi204 ceph-mon[66572]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T00:30:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:30.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-03-19T00:30:30.382 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] boot 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:30.694 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] boot 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:30:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:30.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:30.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:30.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: osd.0 [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] boot 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:31.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:31.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:30:31.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:31.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:31.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:31.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:31.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:31 smithi044 ceph-mon[66366]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:31.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:31 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi007 2024-03-19T00:30:31.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:31 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-19T00:30:31.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:31 smithi044 ceph-mon[66366]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 -- 172.21.15.44:0/2738390233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101240 msgr2=0x7fe67c101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 --2- 172.21.15.44:0/2738390233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101240 0x7fe67c101660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe6700099c0 tx=0x7fe670009cd0 comp rx=0 tx=0).stop 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 -- 172.21.15.44:0/2738390233 shutdown_connections 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 --2- 172.21.15.44:0/2738390233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 0x7fe67c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 --2- 172.21.15.44:0/2738390233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101240 0x7fe67c101660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 --2- 172.21.15.44:0/2738390233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c102440 0x7fe67c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 -- 172.21.15.44:0/2738390233 >> 172.21.15.44:0/2738390233 conn(0x7fe67c0fc800 msgr2=0x7fe67c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 -- 172.21.15.44:0/2738390233 shutdown_connections 2024-03-19T00:30:31.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.863+0000 7fe682cbe700 1 -- 172.21.15.44:0/2738390233 wait complete. 2024-03-19T00:30:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 Processor -- start 2024-03-19T00:30:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 -- start start 2024-03-19T00:30:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c101240 0x7fe67c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 0x7fe67c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 0x7fe67c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c10ccb0 con 0x7fe67c101240 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c10cb30 con 0x7fe67c102440 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe682cbe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c10ce30 con 0x7fe67c102de0 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe67bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 0x7fe67c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe67bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 0x7fe67c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49756/0 (socket says 172.21.15.44:49756) 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe67bfff700 1 -- 172.21.15.44:0/1697556766 learned_addr learned my addr 172.21.15.44:0/1697556766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.864+0000 7fe680a5a700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c101240 0x7fe67c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe68125b700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 0x7fe67c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe67bfff700 1 -- 172.21.15.44:0/1697556766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 msgr2=0x7fe67c19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe67bfff700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 0x7fe67c19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:31.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe67bfff700 1 -- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c101240 msgr2=0x7fe67c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe67bfff700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c101240 0x7fe67c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe67bfff700 1 -- 172.21.15.44:0/1697556766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c1a0070 con 0x7fe67c102440 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe68125b700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 0x7fe67c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe67bfff700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 0x7fe67c198950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe66800d950 tx=0x7fe66800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668016070 con 0x7fe67c102440 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe668009df0 con 0x7fe67c102440 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c1a0360 con 0x7fe67c102440 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66800f620 con 0x7fe67c102440 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.865+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c1a08a0 con 0x7fe67c102440 2024-03-19T00:30:31.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.866+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67c068fd0 con 0x7fe67c102440 2024-03-19T00:30:31.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.867+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) ==== 137233+0+0 (secure 0 0 0) 0x7fe668019070 con 0x7fe67c102440 2024-03-19T00:30:31.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.868+0000 7fe679ffb700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe66c0a2880 0x7fe66c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:31.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.868+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe6680ccbd0 con 0x7fe67c102440 2024-03-19T00:30:31.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.868+0000 7fe680a5a700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe66c0a2880 0x7fe66c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:31.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.868+0000 7fe680a5a700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe66c0a2880 0x7fe66c0a4d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe670000c00 tx=0x7fe670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:30:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:31.873+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe66809b640 con 0x7fe67c102440 2024-03-19T00:30:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:31 smithi007 ceph-mon[62854]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:31 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi007 2024-03-19T00:30:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:31 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-19T00:30:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:31 smithi007 ceph-mon[62854]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T00:30:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:31 smithi204 ceph-mon[66572]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:31 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi007 2024-03-19T00:30:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:31 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-19T00:30:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:31 smithi204 ceph-mon[66572]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T00:30:32.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:32.086+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe67c0fe9d0 con 0x7fe66c0a2880 2024-03-19T00:30:32.919 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:32 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:32.920 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:32 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:32.920 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:32 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:32.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:32.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:32.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:33.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:32 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:33.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:32 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:33.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:32 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:33.869 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:33 smithi044 ceph-mon[66366]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:33.869 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:33 smithi044 ceph-mon[66366]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:33 smithi007 ceph-mon[62854]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:33 smithi007 ceph-mon[62854]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:33 smithi204 ceph-mon[66572]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:33 smithi204 ceph-mon[66572]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/2739660319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]: dispatch 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]: dispatch 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]': finished 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:35 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/3043422950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/2739660319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]: dispatch 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]: dispatch 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]': finished 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:35 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/3043422950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/2739660319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]: dispatch 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]: dispatch 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baa358f6-e033-488f-9078-2a0c3b3c38f1"}]': finished 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:35 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/3043422950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:37.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:37 smithi007 ceph-mon[62854]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:38.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:37 smithi204 ceph-mon[66572]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:37 smithi044 ceph-mon[66366]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:39.935 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:39 smithi044 ceph-mon[66366]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:39.946 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:39 smithi007 ceph-mon[62854]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:39 smithi204 ceph-mon[66572]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:41 smithi007 ceph-mon[62854]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:41 smithi204 ceph-mon[66572]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:42.061 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:41 smithi044 ceph-mon[66366]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:42.931 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T00:30:42.931 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T00:30:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T00:30:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:43.811 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:43 smithi044 ceph-mon[66366]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:43.811 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:43 smithi044 ceph-mon[66366]: Deploying daemon osd.1 on smithi044 2024-03-19T00:30:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:43 smithi007 ceph-mon[62854]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:43 smithi007 ceph-mon[62854]: Deploying daemon osd.1 on smithi044 2024-03-19T00:30:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:43 smithi204 ceph-mon[66572]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:43 smithi204 ceph-mon[66572]: Deploying daemon osd.1 on smithi044 2024-03-19T00:30:45.814 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:45 smithi044 ceph-mon[66366]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:45 smithi007 ceph-mon[62854]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:45 smithi204 ceph-mon[66572]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:47.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:47 smithi007 ceph-mon[62854]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:47.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:47.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:47.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:47.974 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:47 smithi044 ceph-mon[66366]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:47.974 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:47.974 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:47.974 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:47 smithi204 ceph-mon[66572]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:49.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:49 smithi044 ceph-mon[66366]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:49 smithi007 ceph-mon[62854]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:49 smithi204 ceph-mon[66572]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:50.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.499+0000 7fe679ffb700 1 -- 172.21.15.44:0/1697556766 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe67c0fe9d0 con 0x7fe66c0a2880 2024-03-19T00:30:50.501 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 1 on host 'smithi044' 2024-03-19T00:30:50.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe66c0a2880 msgr2=0x7fe66c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:50.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe66c0a2880 0x7fe66c0a4d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe670000c00 tx=0x7fe670005fb0 comp rx=0 tx=0).stop 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 msgr2=0x7fe67c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 0x7fe67c198950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe66800d950 tx=0x7fe66800dc60 comp rx=0 tx=0).stop 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 shutdown_connections 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe66c0a2880 0x7fe66c0a4d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe67c102de0 0x7fe67c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c101240 0x7fe67c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 --2- 172.21.15.44:0/1697556766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c102440 0x7fe67c198950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 >> 172.21.15.44:0/1697556766 conn(0x7fe67c0fc800 msgr2=0x7fe67c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 shutdown_connections 2024-03-19T00:30:50.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:30:50.502+0000 7fe682cbe700 1 -- 172.21.15.44:0/1697556766 wait complete. 2024-03-19T00:30:51.138 DEBUG:teuthology.orchestra.run.smithi044:osd.1> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.1.service 2024-03-19T00:30:51.140 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-03-19T00:30:51.141 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T00:30:51.385 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:30:51.485 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1[70945]: 2024-03-19T00:30:51.294+0000 7fca3df067c0 -1 osd.1 0 log_to_monitors true 2024-03-19T00:30:51.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:51.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: from='osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T00:30:51.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:51 smithi204 ceph-mon[66572]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: from='osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T00:30:51.813 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:51 smithi044 ceph-mon[66366]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: from='osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T00:30:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:51 smithi007 ceph-mon[62854]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T00:30:53.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:53 smithi204 ceph-mon[66572]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:53.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:53 smithi204 ceph-mon[66572]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T00:30:53.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:53 smithi204 ceph-mon[66572]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T00:30:53.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:53 smithi204 ceph-mon[66572]: from='osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:30:53.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:53.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:53 smithi204 ceph-mon[66572]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:30:53.307 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T00:30:53.307 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T00:30:53.307 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-19T00:30:53.308 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-19T00:30:53.308 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113789 s, 922 MB/s 2024-03-19T00:30:53.308 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-19T00:30:53.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:53 smithi007 ceph-mon[62854]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:53.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:53 smithi007 ceph-mon[62854]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T00:30:53.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:53 smithi007 ceph-mon[62854]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T00:30:53.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:53 smithi007 ceph-mon[62854]: from='osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:30:53.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:53.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:53 smithi007 ceph-mon[62854]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:30:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-mon[66366]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-mon[66366]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T00:30:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-mon[66366]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T00:30:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-mon[66366]: from='osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:30:53.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:53.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-mon[66366]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:30:53.881 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-03-19T00:30:54.122 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:30:54.196 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1[70945]: 2024-03-19T00:30:53.876+0000 7fca3356b700 -1 osd.1 0 waiting for initial osdmap 2024-03-19T00:30:54.196 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:30:53 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1[70945]: 2024-03-19T00:30:53.881+0000 7fca2b356700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:30:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:54 smithi007 ceph-mon[62854]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:30:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:54 smithi007 ceph-mon[62854]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T00:30:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:54 smithi007 ceph-mon[62854]: from='osd.1 ' entity='osd.1' 2024-03-19T00:30:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:54 smithi204 ceph-mon[66572]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:30:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:54 smithi204 ceph-mon[66572]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T00:30:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:54.474 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:54.475 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:54 smithi204 ceph-mon[66572]: from='osd.1 ' entity='osd.1' 2024-03-19T00:30:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:54 smithi044 ceph-mon[66366]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:30:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:54 smithi044 ceph-mon[66366]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T00:30:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:54 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:54 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:54 smithi044 ceph-mon[66366]: from='osd.1 ' entity='osd.1' 2024-03-19T00:30:55.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:30:55.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:30:55.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:55.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi044 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi044 to 17093M 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] boot 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:55.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:55.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:55 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:30:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:30:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi044 2024-03-19T00:30:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi044 to 17093M 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] boot 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:55.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:55.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:55 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 -- 172.21.15.204:0/2301629521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4103ed0 msgr2=0x7fccb410a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:55.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 --2- 172.21.15.204:0/2301629521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4103ed0 0x7fccb410a780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fccac00b0a0 tx=0x7fccac00b3b0 comp rx=0 tx=0).stop 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 -- 172.21.15.204:0/2301629521 shutdown_connections 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 --2- 172.21.15.204:0/2301629521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4103ed0 0x7fccb410a780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 --2- 172.21.15.204:0/2301629521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4102330 0x7fccb4102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 --2- 172.21.15.204:0/2301629521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccb4103530 0x7fccb4103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 -- 172.21.15.204:0/2301629521 >> 172.21.15.204:0/2301629521 conn(0x7fccb40fd8f0 msgr2=0x7fccb40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 -- 172.21.15.204:0/2301629521 shutdown_connections 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.558+0000 7fccbbc05700 1 -- 172.21.15.204:0/2301629521 wait complete. 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 Processor -- start 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 -- start start 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 0x7fccb41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccb4103530 0x7fccb4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:55.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4103ed0 0x7fccb419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fccb410dec0 con 0x7fccb4103ed0 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fccb410dd40 con 0x7fccb4103530 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccbbc05700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fccb410e040 con 0x7fccb4102330 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccb99a1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 0x7fccb41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccb99a1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 0x7fccb41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:53502/0 (socket says 172.21.15.204:53502) 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccb99a1700 1 -- 172.21.15.204:0/3172147855 learned_addr learned my addr 172.21.15.204:0/3172147855 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:30:55.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.559+0000 7fccb91a0700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccb4103530 0x7fccb4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccba1a2700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4103ed0 0x7fccb419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccb99a1700 1 -- 172.21.15.204:0/3172147855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccb4103530 msgr2=0x7fccb4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccb99a1700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccb4103530 0x7fccb4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccb99a1700 1 -- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4103ed0 msgr2=0x7fccb419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccb99a1700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4103ed0 0x7fccb419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccb99a1700 1 -- 172.21.15.204:0/3172147855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb41a0220 con 0x7fccb4102330 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccba1a2700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4103ed0 0x7fccb419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccb99a1700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 0x7fccb41985c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fccb000b530 tx=0x7fccb000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0015020 con 0x7fccb4102330 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccb0011ce0 con 0x7fccb4102330 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb41a0510 con 0x7fccb4102330 2024-03-19T00:30:55.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0010d60 con 0x7fccb4102330 2024-03-19T00:30:55.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.560+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb41a0b20 con 0x7fccb4102330 2024-03-19T00:30:55.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.561+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb4068fd0 con 0x7fccb4102330 2024-03-19T00:30:55.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.562+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 137233+0+0 (secure 0 0 0) 0x7fccb0018020 con 0x7fccb4102330 2024-03-19T00:30:55.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.563+0000 7fccaaffd700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcca00a2880 0x7fcca00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:30:55.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.563+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fccb00cce90 con 0x7fccb4102330 2024-03-19T00:30:55.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.563+0000 7fccb91a0700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcca00a2880 0x7fcca00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:30:55.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.564+0000 7fccb91a0700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcca00a2880 0x7fcca00a4d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009a10 tx=0x7fcca4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:30:55.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.567+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fccb001f020 con 0x7fccb4102330 2024-03-19T00:30:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:30:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:30:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T00:30:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi044 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi044 to 17093M 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: osd.1 [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] boot 2024-03-19T00:30:55.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T00:30:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T00:30:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:30:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:30:55.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:55 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:30:55.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:30:55.768+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fccb4062750 con 0x7fcca00a2880 2024-03-19T00:30:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:56 smithi007 ceph-mon[62854]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:56.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:56 smithi204 ceph-mon[66572]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:56.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:56.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:56.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:56.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:56 smithi044 ceph-mon[66366]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:30:56.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:30:56.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:30:56.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:30:57.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:57 smithi007 ceph-mon[62854]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:30:57.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:57 smithi007 ceph-mon[62854]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T00:30:57.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:57 smithi204 ceph-mon[66572]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:30:57.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:57 smithi204 ceph-mon[66572]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T00:30:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:57 smithi044 ceph-mon[66366]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:30:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:57 smithi044 ceph-mon[66366]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/1804735516' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]: dispatch 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]: dispatch 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]': finished 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:30:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:30:59 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/2595100714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/1804735516' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]: dispatch 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]: dispatch 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]': finished 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:30:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:30:59 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/2595100714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:30:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:30:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/1804735516' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]: dispatch 2024-03-19T00:30:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]: dispatch 2024-03-19T00:30:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4bd1927-870b-4e32-ad51-aff87143cb46"}]': finished 2024-03-19T00:30:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T00:30:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:30:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:30:59 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/2595100714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:01 smithi007 ceph-mon[62854]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:01 smithi204 ceph-mon[66572]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:01 smithi044 ceph-mon[66366]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:03.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:03 smithi204 ceph-mon[66572]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:03.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:03 smithi007 ceph-mon[62854]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:03 smithi044 ceph-mon[66366]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:05.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:05 smithi007 ceph-mon[62854]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:05.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T00:31:05.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:05 smithi204 ceph-mon[66572]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T00:31:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:05 smithi044 ceph-mon[66366]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T00:31:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:06.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:06 smithi204 ceph-mon[66572]: Deploying daemon osd.2 on smithi204 2024-03-19T00:31:06.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:06 smithi044 ceph-mon[66366]: Deploying daemon osd.2 on smithi204 2024-03-19T00:31:06.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:06 smithi007 ceph-mon[62854]: Deploying daemon osd.2 on smithi204 2024-03-19T00:31:07.436 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:07 smithi204 ceph-mon[66572]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:07 smithi044 ceph-mon[66366]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:07.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:07 smithi007 ceph-mon[62854]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:09.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:09 smithi204 ceph-mon[66572]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:09.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:09 smithi044 ceph-mon[66366]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:09.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:09 smithi007 ceph-mon[62854]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:11.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:11 smithi007 ceph-mon[62854]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:11.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:11.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:11.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:11.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:11 smithi204 ceph-mon[66572]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:11.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:11.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:11.738 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:11.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:11 smithi044 ceph-mon[66366]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:11.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:11.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:11.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:13 smithi204 ceph-mon[66572]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:13 smithi044 ceph-mon[66366]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:13 smithi007 ceph-mon[62854]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:13.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:13.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:13.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:13.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:14.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.085+0000 7fccaaffd700 1 -- 172.21.15.204:0/3172147855 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccb4062750 con 0x7fcca00a2880 2024-03-19T00:31:14.086 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.087+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcca00a2880 msgr2=0x7fcca00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcca00a2880 0x7fcca00a4d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009a10 tx=0x7fcca4009190 comp rx=0 tx=0).stop 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 msgr2=0x7fccb41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 0x7fccb41985c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fccb000b530 tx=0x7fccb000b840 comp rx=0 tx=0).stop 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 shutdown_connections 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcca00a2880 0x7fcca00a4d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb4103ed0 0x7fccb419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:14.089 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb4102330 0x7fccb41985c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:14.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 --2- 172.21.15.204:0/3172147855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccb4103530 0x7fccb4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:14.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 >> 172.21.15.204:0/3172147855 conn(0x7fccb40fd8f0 msgr2=0x7fccb41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:31:14.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 shutdown_connections 2024-03-19T00:31:14.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:31:14.088+0000 7fccbbc05700 1 -- 172.21.15.204:0/3172147855 wait complete. 2024-03-19T00:31:14.843 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.2.service 2024-03-19T00:31:14.846 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-03-19T00:31:14.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T00:31:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:31:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:15 smithi007 ceph-mon[62854]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:15 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:15 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:15 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:15 smithi007 ceph-mon[62854]: from='osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T00:31:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:15 smithi007 ceph-mon[62854]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T00:31:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:15 smithi044 ceph-mon[66366]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:15 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:15 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:15 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:15 smithi044 ceph-mon[66366]: from='osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T00:31:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:15 smithi044 ceph-mon[66366]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T00:31:15.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:15 smithi204 ceph-mon[66572]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:15.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:15 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:15.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:15 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:15.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:15 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:15.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:15 smithi204 ceph-mon[66572]: from='osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T00:31:15.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:15 smithi204 ceph-mon[66572]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T00:31:16.538 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-mon[66572]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T00:31:16.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-mon[66572]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T00:31:16.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:16.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-mon[66572]: from='osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:31:16.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-mon[66572]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:31:16.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:16 smithi007 ceph-mon[62854]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T00:31:16.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:16 smithi007 ceph-mon[62854]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T00:31:16.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:16.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:16 smithi007 ceph-mon[62854]: from='osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:31:16.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:16 smithi007 ceph-mon[62854]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:31:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:16 smithi044 ceph-mon[66366]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T00:31:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:16 smithi044 ceph-mon[66366]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T00:31:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:16 smithi044 ceph-mon[66366]: from='osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:31:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:16 smithi044 ceph-mon[66366]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:31:17.240 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2[71738]: 2024-03-19T00:31:16.987+0000 7f3d76956700 -1 osd.2 0 waiting for initial osdmap 2024-03-19T00:31:17.240 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2[71738]: 2024-03-19T00:31:16.993+0000 7f3d6e741700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:31:17.288 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T00:31:17.288 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T00:31:17.288 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-19T00:31:17.288 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-19T00:31:17.288 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00867308 s, 1.2 GB/s 2024-03-19T00:31:17.288 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-19T00:31:17.492 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:17 smithi204 ceph-mon[66572]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:17.492 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:17 smithi204 ceph-mon[66572]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:31:17.492 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:17 smithi204 ceph-mon[66572]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T00:31:17.492 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:17 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:17.492 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:17 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:17.492 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:17 smithi204 ceph-mon[66572]: from='osd.2 ' entity='osd.2' 2024-03-19T00:31:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:17 smithi044 ceph-mon[66366]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:17 smithi044 ceph-mon[66366]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:31:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:17 smithi044 ceph-mon[66366]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T00:31:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:17 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:17 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:17 smithi044 ceph-mon[66366]: from='osd.2 ' entity='osd.2' 2024-03-19T00:31:17.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:17 smithi007 ceph-mon[62854]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:17.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:17 smithi007 ceph-mon[62854]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:31:17.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:17 smithi007 ceph-mon[62854]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T00:31:17.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:17 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:17.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:17 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:17.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:17 smithi007 ceph-mon[62854]: from='osd.2 ' entity='osd.2' 2024-03-19T00:31:18.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-03-19T00:31:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:31:18.421 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:31:18.421 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:31:18.421 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:18.421 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi204 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi204 to 17088M 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] boot 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T00:31:18.422 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi204 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi204 to 17088M 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] boot 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T00:31:18.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:18.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:31:18.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:31:18.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:18.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi204 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi204 to 17088M 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: osd.2 [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] boot 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T00:31:18.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T00:31:19.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:19 smithi007 ceph-mon[62854]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:19.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:19 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:31:19.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:19 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T00:31:19.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:19 smithi007 ceph-mon[62854]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T00:31:19.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:19 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:31:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:19 smithi044 ceph-mon[66366]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:19 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:31:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:19 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T00:31:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:19 smithi044 ceph-mon[66366]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T00:31:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:19 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:31:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:19 smithi204 ceph-mon[66572]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T00:31:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:19 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:31:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:19 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T00:31:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:19 smithi204 ceph-mon[66572]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T00:31:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:19 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 -- 172.21.15.7:0/2276337627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 msgr2=0x7f82a4100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 --2- 172.21.15.7:0/2276337627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4100e00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82940099c0 tx=0x7f8294009cd0 comp rx=0 tx=0).stop 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 -- 172.21.15.7:0/2276337627 shutdown_connections 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 --2- 172.21.15.7:0/2276337627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82a4102580 0x7f82a4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 --2- 172.21.15.7:0/2276337627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4100e00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 --2- 172.21.15.7:0/2276337627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4101be0 0x7f82a4102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 -- 172.21.15.7:0/2276337627 >> 172.21.15.7:0/2276337627 conn(0x7f82a40fbf60 msgr2=0x7f82a40fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 -- 172.21.15.7:0/2276337627 shutdown_connections 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 -- 172.21.15.7:0/2276337627 wait complete. 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 Processor -- start 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.880+0000 7f82a9f78700 1 -- start start 2024-03-19T00:31:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4101be0 0x7f82a4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82a4102580 0x7f82a419fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82a410bbc0 con 0x7f82a41009e0 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82a410ba40 con 0x7f82a4102580 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82a410bd40 con 0x7f82a4101be0 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40174/0 (socket says 172.21.15.7:40174) 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 -- 172.21.15.7:0/1685291730 learned_addr learned my addr 172.21.15.7:0/1685291730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:31:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4101be0 0x7f82a4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a3fff700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82a4102580 0x7f82a419fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 -- 172.21.15.7:0/1685291730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4101be0 msgr2=0x7f82a4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4101be0 0x7f82a4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 -- 172.21.15.7:0/1685291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82a4102580 msgr2=0x7f82a419fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82a4102580 0x7f82a419fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 -- 172.21.15.7:0/1685291730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a41a00a0 con 0x7f82a41009e0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a37fe700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4198440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f82940057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294027070 con 0x7f82a41009e0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8294004050 con 0x7f82a41009e0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a41a0330 con 0x7f82a41009e0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294022dd0 con 0x7f82a41009e0 2024-03-19T00:31:19.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.881+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a0780 con 0x7f82a41009e0 2024-03-19T00:31:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.882+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a4068fd0 con 0x7f82a41009e0 2024-03-19T00:31:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.884+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 137233+0+0 (secure 0 0 0) 0x7f829402a070 con 0x7f82a41009e0 2024-03-19T00:31:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.884+0000 7f82a0ff9700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f828c0a2800 0x7f828c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.884+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f828c0a2800 0x7f828c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.884+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f82940ce410 con 0x7f82a41009e0 2024-03-19T00:31:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.885+0000 7f82a2ffd700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f828c0a2800 0x7f828c0a4cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f829800ac50 tx=0x7f829800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:31:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:19.889+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f829405e740 con 0x7f82a41009e0 2024-03-19T00:31:20.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:20.116+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f82a4108810 con 0x7f828c0a2800 2024-03-19T00:31:20.465 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T00:31:20.465 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:20.465 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79401]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:20.465 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79401]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:20.926 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79445]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T00:31:20.926 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:20.927 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79445]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:20.927 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:31:20 smithi007 sudo[79445]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:21.168 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:20 smithi044 sudo[75080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T00:31:21.168 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:20 smithi044 sudo[75080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:21.168 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:20 smithi044 sudo[75080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:21.168 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:20 smithi044 sudo[75080]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:21.168 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:21.168 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T00:31:21.168 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T00:31:21.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:31:21.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:31:21.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:31:21.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:21.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:31:21.169 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:31:21.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:31:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:21.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:31:21.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:21.606 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:21 smithi044 sudo[75083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T00:31:21.606 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:21 smithi044 sudo[75083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:21.606 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:21 smithi044 sudo[75083]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:21.606 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:31:21 smithi044 sudo[75083]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:21.652+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f8294098a00 con 0x7f82a41009e0 2024-03-19T00:31:21.692 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75798]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T00:31:21.692 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75798]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:21.692 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75798]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:21.693 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75798]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:22.015 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75801]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T00:31:22.015 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75801]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:22.015 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75801]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:22.015 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:31:21 smithi204 sudo[75801]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:21 smithi007 sudo[79509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T00:31:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:21 smithi007 sudo[79509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:21 smithi007 sudo[79509]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:21 smithi007 sudo[79509]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:22.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 ceph-mon[66572]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 ceph-mon[66572]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 ceph-mon[66572]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75804]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:22.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75804]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:22 smithi044 ceph-mon[66366]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T00:31:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:22 smithi044 ceph-mon[66366]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T00:31:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:22 smithi044 ceph-mon[66366]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 sudo[79517]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 sudo[79517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 sudo[79517]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 sudo[79517]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 ceph-mon[62854]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 ceph-mon[62854]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:22 smithi007 ceph-mon[62854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T00:31:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75807]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:22 smithi204 sudo[75807]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:23.270 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75086]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T00:31:23.271 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:23.271 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75086]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:23.271 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75086]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75089]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75089]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 sudo[75089]: pam_unix(sudo:session): session closed for user root 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: Cluster is now healthy 2024-03-19T00:31:23.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:23.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3297226181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e3811a3-7436-489a-ac0a-857023269d16"}]: dispatch 2024-03-19T00:31:23.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T00:31:23.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: Cluster is now healthy 2024-03-19T00:31:23.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:23.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3297226181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e3811a3-7436-489a-ac0a-857023269d16"}]: dispatch 2024-03-19T00:31:23.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: Cluster is now healthy 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3297226181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e3811a3-7436-489a-ac0a-857023269d16"}]: dispatch 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T00:31:23.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T00:31:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:24 smithi044 ceph-mon[66366]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:24 smithi044 ceph-mon[66366]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3297226181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e3811a3-7436-489a-ac0a-857023269d16"}]': finished 2024-03-19T00:31:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:24 smithi044 ceph-mon[66366]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T00:31:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1136698240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:24 smithi007 ceph-mon[62854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:24 smithi007 ceph-mon[62854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3297226181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e3811a3-7436-489a-ac0a-857023269d16"}]': finished 2024-03-19T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:24 smithi007 ceph-mon[62854]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1136698240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:24 smithi204 ceph-mon[66572]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T00:31:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:24 smithi204 ceph-mon[66572]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T00:31:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3297226181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e3811a3-7436-489a-ac0a-857023269d16"}]': finished 2024-03-19T00:31:24.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:24 smithi204 ceph-mon[66572]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T00:31:24.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:24.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1136698240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:25 smithi044 ceph-mon[66366]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:25 smithi007 ceph-mon[62854]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:25 smithi204 ceph-mon[66572]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:27 smithi044 ceph-mon[66366]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 7.3 KiB/s rd, 370 KiB/s wr, 24 op/s 2024-03-19T00:31:27.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:27 smithi007 ceph-mon[62854]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 7.3 KiB/s rd, 370 KiB/s wr, 24 op/s 2024-03-19T00:31:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:27 smithi204 ceph-mon[66572]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 7.3 KiB/s rd, 370 KiB/s wr, 24 op/s 2024-03-19T00:31:29.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:29 smithi007 ceph-mon[62854]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 263 KiB/s wr, 17 op/s 2024-03-19T00:31:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:29 smithi044 ceph-mon[66366]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 263 KiB/s wr, 17 op/s 2024-03-19T00:31:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:29 smithi204 ceph-mon[66572]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 263 KiB/s wr, 17 op/s 2024-03-19T00:31:31.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:31 smithi007 ceph-mon[62854]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 204 KiB/s wr, 13 op/s 2024-03-19T00:31:31.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T00:31:31.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:31.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:31 smithi007 ceph-mon[62854]: Deploying daemon osd.3 on smithi007 2024-03-19T00:31:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:31 smithi044 ceph-mon[66366]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 204 KiB/s wr, 13 op/s 2024-03-19T00:31:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T00:31:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:31 smithi044 ceph-mon[66366]: Deploying daemon osd.3 on smithi007 2024-03-19T00:31:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:31 smithi204 ceph-mon[66572]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 204 KiB/s wr, 13 op/s 2024-03-19T00:31:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T00:31:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:31 smithi204 ceph-mon[66572]: Deploying daemon osd.3 on smithi007 2024-03-19T00:31:33.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:33 smithi007 ceph-mon[62854]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-19T00:31:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:33 smithi044 ceph-mon[66366]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-19T00:31:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:33 smithi204 ceph-mon[66572]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-19T00:31:35.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:35 smithi204 ceph-mon[66572]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-03-19T00:31:35.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:35 smithi007 ceph-mon[62854]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-03-19T00:31:35.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:35 smithi044 ceph-mon[66366]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-03-19T00:31:37.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:37 smithi007 ceph-mon[62854]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-19T00:31:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:37.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:37 smithi204 ceph-mon[66572]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-19T00:31:37.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:37.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:37.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:37.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:37 smithi044 ceph-mon[66366]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-19T00:31:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:39.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:39 smithi007 ceph-mon[62854]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:39.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:39 smithi204 ceph-mon[66572]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:39.526 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-03-19T00:31:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.525+0000 7f82a0ff9700 1 -- 172.21.15.7:0/1685291730 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82a4108810 con 0x7f828c0a2800 2024-03-19T00:31:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f828c0a2800 msgr2=0x7f828c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f828c0a2800 0x7f828c0a4cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f829800ac50 tx=0x7f829800a380 comp rx=0 tx=0).stop 2024-03-19T00:31:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 msgr2=0x7f82a4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4198440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f82940057c0 comp rx=0 tx=0).stop 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 shutdown_connections 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f828c0a2800 0x7f828c0a4cc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82a4102580 0x7f82a419fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82a41009e0 0x7f82a4198440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 --2- 172.21.15.7:0/1685291730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82a4101be0 0x7f82a4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.527+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 >> 172.21.15.7:0/1685291730 conn(0x7f82a40fbf60 msgr2=0x7f82a41070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.528+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 shutdown_connections 2024-03-19T00:31:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:31:39.528+0000 7f82a9f78700 1 -- 172.21.15.7:0/1685291730 wait complete. 2024-03-19T00:31:39.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:39 smithi044 ceph-mon[66366]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:40.199 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.3.service 2024-03-19T00:31:40.201 INFO:tasks.cephadm:Deploying osd.4 on smithi044 with /dev/vg_nvme/lv_3... 2024-03-19T00:31:40.202 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T00:31:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:31:40.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:40 smithi204 ceph-mon[66572]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T00:31:40.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:40 smithi044 ceph-mon[66366]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T00:31:40.824 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.825 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:40 smithi007 ceph-mon[62854]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T00:31:40.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:40.902 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:41 smithi044 ceph-mon[66366]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:41 smithi044 ceph-mon[66366]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T00:31:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:41 smithi044 ceph-mon[66366]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T00:31:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:41 smithi044 ceph-mon[66366]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:31:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:41.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:41 smithi007 ceph-mon[62854]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:41.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:41 smithi007 ceph-mon[62854]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T00:31:41.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:41 smithi007 ceph-mon[62854]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T00:31:41.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:41 smithi007 ceph-mon[62854]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:31:41.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:41 smithi204 ceph-mon[66572]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:41 smithi204 ceph-mon[66572]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T00:31:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:41 smithi204 ceph-mon[66572]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T00:31:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:41 smithi204 ceph-mon[66572]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:31:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.215 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3[81603]: 2024-03-19T00:31:42.067+0000 7fbf145c9700 -1 osd.3 0 waiting for initial osdmap 2024-03-19T00:31:42.215 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3[81603]: 2024-03-19T00:31:42.073+0000 7fbf0bbb3700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:31:42.509 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T00:31:42.509 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T00:31:42.509 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-19T00:31:42.509 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-19T00:31:42.510 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128916 s, 813 MB/s 2024-03-19T00:31:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-19T00:31:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:42 smithi044 ceph-mon[66366]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:31:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:42 smithi044 ceph-mon[66366]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T00:31:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:42 smithi044 ceph-mon[66366]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' 2024-03-19T00:31:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-mon[62854]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:31:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-mon[62854]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T00:31:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-mon[62854]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' 2024-03-19T00:31:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:42 smithi204 ceph-mon[66572]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:31:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:42 smithi204 ceph-mon[66572]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T00:31:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:42 smithi204 ceph-mon[66572]: from='osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923]' entity='osd.3' 2024-03-19T00:31:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:43.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-03-19T00:31:43.459 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:31:43.682 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:31:43.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:31:43.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:43.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] boot 2024-03-19T00:31:43.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T00:31:43.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:43.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:43 smithi007 ceph-mon[62854]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T00:31:43.780 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:31:43.780 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:31:43.781 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:43.781 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] boot 2024-03-19T00:31:43.781 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T00:31:43.781 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:43.781 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:43 smithi044 ceph-mon[66366]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: osd.3 [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] boot 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T00:31:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:43 smithi204 ceph-mon[66572]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T00:31:44.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi007 2024-03-19T00:31:44.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:44.833 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:44.834 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:44 smithi044 ceph-mon[66366]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi007 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:45.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:45.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:45.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:44 smithi204 ceph-mon[66572]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T00:31:45.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 -- 172.21.15.44:0/1019672223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34102360 msgr2=0x7f2b34102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:45.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/1019672223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34102360 0x7f2b34102780 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0099c0 tx=0x7f2b2c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:31:45.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 -- 172.21.15.44:0/1019672223 shutdown_connections 2024-03-19T00:31:45.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/1019672223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34103f00 0x7f2b3410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:45.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/1019672223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34102360 0x7f2b34102780 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:45.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/1019672223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34103560 0x7f2b341039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:45.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 -- 172.21.15.44:0/1019672223 >> 172.21.15.44:0/1019672223 conn(0x7f2b340fd8c0 msgr2=0x7f2b340ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:31:45.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 -- 172.21.15.44:0/1019672223 shutdown_connections 2024-03-19T00:31:45.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.028+0000 7f2b3bca8700 1 -- 172.21.15.44:0/1019672223 wait complete. 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 Processor -- start 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 -- start start 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34102360 0x7f2b34198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34103560 0x7f2b34198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 0x7f2b3419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3410de50 con 0x7f2b34102360 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3410dcd0 con 0x7f2b34103f00 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3bca8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3410dfd0 con 0x7f2b34103560 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 0x7f2b3419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 0x7f2b3419fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48468/0 (socket says 172.21.15.44:48468) 2024-03-19T00:31:45.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 -- 172.21.15.44:0/176158364 learned_addr learned my addr 172.21.15.44:0/176158364 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b39a44700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34102360 0x7f2b34198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b39243700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34103560 0x7f2b34198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 -- 172.21.15.44:0/176158364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34103560 msgr2=0x7f2b34198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34103560 0x7f2b34198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 -- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34102360 msgr2=0x7f2b34198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34102360 0x7f2b34198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b3a245700 1 -- 172.21.15.44:0/176158364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b341a0180 con 0x7f2b34103f00 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.029+0000 7f2b39a44700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34102360 0x7f2b34198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.030+0000 7f2b3a245700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 0x7f2b3419fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b24007ee0 tx=0x7f2b2400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.030+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400dbd0 con 0x7f2b34103f00 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.030+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b2400dd70 con 0x7f2b34103f00 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.030+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341a0470 con 0x7f2b34103f00 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.030+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24005a00 con 0x7f2b34103f00 2024-03-19T00:31:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.030+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341a0a80 con 0x7f2b34103f00 2024-03-19T00:31:45.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.031+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b34068fd0 con 0x7f2b34103f00 2024-03-19T00:31:45.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.032+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f2b2401a020 con 0x7f2b34103f00 2024-03-19T00:31:45.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.033+0000 7f2b32ffd700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b1c0a2990 0x7f2b1c0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:31:45.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.033+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f2b240cd6c0 con 0x7f2b34103f00 2024-03-19T00:31:45.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.033+0000 7f2b39a44700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b1c0a2990 0x7f2b1c0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:31:45.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.033+0000 7f2b39a44700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b1c0a2990 0x7f2b1c0a4e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:31:45.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.036+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b24010070 con 0x7f2b34103f00 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi007 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-19T00:31:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:31:45.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:44 smithi007 ceph-mon[62854]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T00:31:45.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:31:45.252+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2b34062810 con 0x7f2b1c0a2990 2024-03-19T00:31:46.015 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:45 smithi044 ceph-mon[66366]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:46.015 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:31:46.015 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:31:46.015 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:45 smithi204 ceph-mon[66572]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:46.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:31:46.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:31:46.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:46.116 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:45 smithi007 ceph-mon[62854]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:46.116 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:31:46.116 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:31:46.116 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:47.038 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:46 smithi044 ceph-mon[66366]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:31:47.038 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:46 smithi044 ceph-mon[66366]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T00:31:47.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:46 smithi007 ceph-mon[62854]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:31:47.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:46 smithi007 ceph-mon[62854]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T00:31:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:46 smithi204 ceph-mon[66572]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:31:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:46 smithi204 ceph-mon[66572]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T00:31:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:47 smithi044 ceph-mon[66366]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-03-19T00:31:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:47 smithi044 ceph-mon[66366]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T00:31:48.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:47 smithi007 ceph-mon[62854]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-03-19T00:31:48.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:47 smithi007 ceph-mon[62854]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T00:31:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:47 smithi204 ceph-mon[66572]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-03-19T00:31:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:47 smithi204 ceph-mon[66572]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T00:31:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/21817351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]: dispatch 2024-03-19T00:31:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:48 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]: dispatch 2024-03-19T00:31:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:48 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]': finished 2024-03-19T00:31:49.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:48 smithi044 ceph-mon[66366]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T00:31:49.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:31:49.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/3866796073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:48 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/21817351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]: dispatch 2024-03-19T00:31:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:48 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]: dispatch 2024-03-19T00:31:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:48 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]': finished 2024-03-19T00:31:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:48 smithi007 ceph-mon[62854]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T00:31:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:31:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:48 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/3866796073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:48 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/21817351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]: dispatch 2024-03-19T00:31:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:48 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]: dispatch 2024-03-19T00:31:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:48 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "993175bf-d87f-4369-9ded-8b968d4b9160"}]': finished 2024-03-19T00:31:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:48 smithi204 ceph-mon[66572]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T00:31:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:31:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:48 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/3866796073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:31:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:49 smithi044 ceph-mon[66366]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-19T00:31:50.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:49 smithi007 ceph-mon[62854]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-19T00:31:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:49 smithi204 ceph-mon[66572]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-19T00:31:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:51 smithi044 ceph-mon[66366]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-19T00:31:52.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:51 smithi007 ceph-mon[62854]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-19T00:31:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:51 smithi204 ceph-mon[66572]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-19T00:31:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:53 smithi044 ceph-mon[66366]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:53 smithi007 ceph-mon[62854]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:54.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:53 smithi204 ceph-mon[66572]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:55 smithi044 ceph-mon[66366]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:56.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:55 smithi007 ceph-mon[62854]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:55 smithi204 ceph-mon[66572]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T00:31:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:56 smithi044 ceph-mon[66366]: Deploying daemon osd.4 on smithi044 2024-03-19T00:31:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T00:31:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:56 smithi007 ceph-mon[62854]: Deploying daemon osd.4 on smithi044 2024-03-19T00:31:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T00:31:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:31:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:56 smithi204 ceph-mon[66572]: Deploying daemon osd.4 on smithi044 2024-03-19T00:31:57.850 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:57 smithi044 ceph-mon[66366]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:58.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:57 smithi007 ceph-mon[62854]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:31:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:57 smithi204 ceph-mon[66572]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:00.098 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:31:59 smithi044 ceph-mon[66366]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:31:59 smithi007 ceph-mon[62854]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:00.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:31:59 smithi204 ceph-mon[66572]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:02.062 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:01 smithi044 ceph-mon[66366]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:02.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:02.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:02.063 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:01 smithi007 ceph-mon[62854]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:01 smithi204 ceph-mon[66572]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:03 smithi044 ceph-mon[66366]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:04.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:03 smithi007 ceph-mon[62854]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:04.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:03 smithi204 ceph-mon[66572]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.620+0000 7f2b32ffd700 1 -- 172.21.15.44:0/176158364 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2b34062810 con 0x7f2b1c0a2990 2024-03-19T00:32:05.621 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 4 on host 'smithi044' 2024-03-19T00:32:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b1c0a2990 msgr2=0x7f2b1c0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b1c0a2990 0x7f2b1c0a4e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 msgr2=0x7f2b3419fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 0x7f2b3419fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b24007ee0 tx=0x7f2b2400d3b0 comp rx=0 tx=0).stop 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 shutdown_connections 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b1c0a2990 0x7f2b1c0a4e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b34103f00 0x7f2b3419fae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b34102360 0x7f2b34198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 --2- 172.21.15.44:0/176158364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34103560 0x7f2b34198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 >> 172.21.15.44:0/176158364 conn(0x7f2b340fd8c0 msgr2=0x7f2b341068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 shutdown_connections 2024-03-19T00:32:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:32:05.622+0000 7f2b3bca8700 1 -- 172.21.15.44:0/176158364 wait complete. 2024-03-19T00:32:05.935 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:05.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T00:32:06.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T00:32:06.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:06.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:06.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:06.245 DEBUG:teuthology.orchestra.run.smithi044:osd.4> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.4.service 2024-03-19T00:32:06.247 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-03-19T00:32:06.247 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T00:32:06.499 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:32:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:06 smithi044 ceph-mon[66366]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T00:32:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:06 smithi044 ceph-mon[66366]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T00:32:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:06 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:06 smithi044 ceph-mon[66366]: from='osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:32:07.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:06 smithi044 ceph-mon[66366]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:32:07.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:06 smithi007 ceph-mon[62854]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T00:32:07.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:06 smithi007 ceph-mon[62854]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T00:32:07.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:06 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:07.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:06 smithi007 ceph-mon[62854]: from='osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:32:07.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:06 smithi007 ceph-mon[62854]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:32:07.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:06 smithi204 ceph-mon[66572]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T00:32:07.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:06 smithi204 ceph-mon[66572]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T00:32:07.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:06 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:07.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:06 smithi204 ceph-mon[66572]: from='osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:32:07.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:06 smithi204 ceph-mon[66572]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:32:07.807 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:32:07.485+0000 7f126df82700 -1 osd.4 0 waiting for initial osdmap 2024-03-19T00:32:07.807 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:32:07.492+0000 7f126556c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:32:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:07 smithi204 ceph-mon[66572]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:07 smithi204 ceph-mon[66572]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:32:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:07 smithi204 ceph-mon[66572]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T00:32:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:07 smithi204 ceph-mon[66572]: from='osd.4 ' entity='osd.4' 2024-03-19T00:32:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-mon[66366]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-mon[66366]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:32:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-mon[66366]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T00:32:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:07 smithi044 ceph-mon[66366]: from='osd.4 ' entity='osd.4' 2024-03-19T00:32:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:07 smithi007 ceph-mon[62854]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:07 smithi007 ceph-mon[62854]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:32:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:07 smithi007 ceph-mon[62854]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T00:32:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:07 smithi007 ceph-mon[62854]: from='osd.4 ' entity='osd.4' 2024-03-19T00:32:08.509 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T00:32:08.509 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T00:32:08.509 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-19T00:32:08.509 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-19T00:32:08.509 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149839 s, 700 MB/s 2024-03-19T00:32:08.509 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-19T00:32:09.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:08 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:32:09.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:08 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:32:09.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:09.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:08 smithi044 ceph-mon[66366]: osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] boot 2024-03-19T00:32:09.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:08 smithi044 ceph-mon[66366]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T00:32:09.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:09.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:08 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:32:09.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:08 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:32:09.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:09.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:08 smithi204 ceph-mon[66572]: osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] boot 2024-03-19T00:32:09.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:08 smithi204 ceph-mon[66572]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T00:32:09.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:09.120 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-03-19T00:32:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:08 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:32:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:08 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:32:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:08 smithi007 ceph-mon[62854]: osd.4 [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] boot 2024-03-19T00:32:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:08 smithi007 ceph-mon[62854]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T00:32:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T00:32:09.355 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:32:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:10.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:10.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:10.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:10.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:09 smithi044 ceph-mon[66366]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T00:32:10.184 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:10.184 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.184 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.184 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:10.184 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:10.185 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.185 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:10.185 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:10.185 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.185 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:09 smithi204 ceph-mon[66572]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T00:32:10.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:09 smithi007 ceph-mon[62854]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T00:32:10.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 -- 172.21.15.204:0/738580663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c102360 msgr2=0x7fdc1c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:10.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/738580663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c102360 0x7fdc1c102780 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b0a0 tx=0x7fdc1400b3b0 comp rx=0 tx=0).stop 2024-03-19T00:32:10.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 -- 172.21.15.204:0/738580663 shutdown_connections 2024-03-19T00:32:10.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/738580663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc1c103f00 0x7fdc1c10a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/738580663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c102360 0x7fdc1c102780 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/738580663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c103560 0x7fdc1c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 -- 172.21.15.204:0/738580663 >> 172.21.15.204:0/738580663 conn(0x7fdc1c0fd920 msgr2=0x7fdc1c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 -- 172.21.15.204:0/738580663 shutdown_connections 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.985+0000 7fdc22e7a700 1 -- 172.21.15.204:0/738580663 wait complete. 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 Processor -- start 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 -- start start 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc1c102360 0x7fdc1c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 0x7fdc1c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c103f00 0x7fdc1c19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10de00 con 0x7fdc1c103f00 2024-03-19T00:32:10.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10dc80 con 0x7fdc1c102360 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc22e7a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10df80 con 0x7fdc1c103560 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 0x7fdc1c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 0x7fdc1c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44284/0 (socket says 172.21.15.204:44284) 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 -- 172.21.15.204:0/70197721 learned_addr learned my addr 172.21.15.204:0/70197721 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc20c16700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc1c102360 0x7fdc1c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc21417700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c103f00 0x7fdc1c19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 -- 172.21.15.204:0/70197721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc1c102360 msgr2=0x7fdc1c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc1c102360 0x7fdc1c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 -- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c103f00 msgr2=0x7fdc1c19fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c103f00 0x7fdc1c19fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 -- 172.21.15.204:0/70197721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1a00e0 con 0x7fdc1c103560 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.986+0000 7fdc1bfff700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 0x7fdc1c1989c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e9d0 tx=0x7fdc0c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:32:10.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.987+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00cbd0 con 0x7fdc1c103560 2024-03-19T00:32:10.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.987+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1a03d0 con 0x7fdc1c103560 2024-03-19T00:32:10.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.987+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc0c00cd70 con 0x7fdc1c103560 2024-03-19T00:32:10.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.987+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c018680 con 0x7fdc1c103560 2024-03-19T00:32:10.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.987+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1a0960 con 0x7fdc1c103560 2024-03-19T00:32:10.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.988+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c068fd0 con 0x7fdc1c103560 2024-03-19T00:32:10.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.989+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fdc0c010430 con 0x7fdc1c103560 2024-03-19T00:32:10.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.990+0000 7fdc19ffb700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdc080a2890 0x7fdc080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:10.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.990+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fdc0c0cdcf0 con 0x7fdc1c103560 2024-03-19T00:32:10.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.990+0000 7fdc20c16700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdc080a2890 0x7fdc080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:10.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.990+0000 7fdc20c16700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdc080a2890 0x7fdc080a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009be0 tx=0x7fdc14006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:32:10.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:10.994+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc0c0d2070 con 0x7fdc1c103560 2024-03-19T00:32:11.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:10 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi044 2024-03-19T00:32:11.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:10 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi044 to 8546M 2024-03-19T00:32:11.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:10 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi044 2024-03-19T00:32:11.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:10 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi044 to 8546M 2024-03-19T00:32:11.199 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:11.198+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc1c062750 con 0x7fdc080a2890 2024-03-19T00:32:11.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:10 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi044 2024-03-19T00:32:11.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:10 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi044 to 8546M 2024-03-19T00:32:12.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:11 smithi204 ceph-mon[66572]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:12.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:32:12.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:32:12.066 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:11 smithi044 ceph-mon[66366]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:32:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:32:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:12.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:11 smithi007 ceph-mon[62854]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:12.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:32:12.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:32:12.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:12 smithi204 ceph-mon[66572]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:32:13.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:12 smithi044 ceph-mon[66366]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:32:13.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:12 smithi007 ceph-mon[62854]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:32:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:13 smithi044 ceph-mon[66366]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:14.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:13 smithi007 ceph-mon[62854]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:13 smithi204 ceph-mon[66572]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:15.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/2506945563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]: dispatch 2024-03-19T00:32:15.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:14 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]: dispatch 2024-03-19T00:32:15.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:14 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]': finished 2024-03-19T00:32:15.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:14 smithi044 ceph-mon[66366]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T00:32:15.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:14 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:15.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/2207405698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:32:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/2506945563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]: dispatch 2024-03-19T00:32:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:14 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]: dispatch 2024-03-19T00:32:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:14 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]': finished 2024-03-19T00:32:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:14 smithi007 ceph-mon[62854]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T00:32:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:14 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:15.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/2207405698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:32:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/2506945563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]: dispatch 2024-03-19T00:32:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:14 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]: dispatch 2024-03-19T00:32:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:14 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a893438-b29c-4c46-bbe8-ba8ac823e8dc"}]': finished 2024-03-19T00:32:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:14 smithi204 ceph-mon[66572]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T00:32:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:14 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/2207405698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:32:16.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:15 smithi007 ceph-mon[62854]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:15 smithi204 ceph-mon[66572]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:15 smithi044 ceph-mon[66366]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:17 smithi044 ceph-mon[66366]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:18.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:17 smithi007 ceph-mon[62854]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:18.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:17 smithi204 ceph-mon[66572]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:19.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:19 smithi204 ceph-mon[66572]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:20.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:19 smithi007 ceph-mon[62854]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:20.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:19 smithi044 ceph-mon[66366]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:21 smithi007 ceph-mon[62854]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T00:32:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:22.257 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:21 smithi204 ceph-mon[66572]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:22.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T00:32:22.258 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:22.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:21 smithi044 ceph-mon[66366]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:22.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T00:32:22.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:23.204 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:22 smithi204 ceph-mon[66572]: Deploying daemon osd.5 on smithi204 2024-03-19T00:32:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:22 smithi007 ceph-mon[62854]: Deploying daemon osd.5 on smithi204 2024-03-19T00:32:23.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:22 smithi044 ceph-mon[66366]: Deploying daemon osd.5 on smithi204 2024-03-19T00:32:23.999 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:23 smithi204 ceph-mon[66572]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:23 smithi007 ceph-mon[62854]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:23 smithi044 ceph-mon[66366]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:26.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:25 smithi204 ceph-mon[66572]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:26.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:25 smithi007 ceph-mon[62854]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:26.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:25 smithi044 ceph-mon[66366]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:27.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:27 smithi007 ceph-mon[62854]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:27 smithi204 ceph-mon[66572]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:27 smithi044 ceph-mon[66366]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:29 smithi007 ceph-mon[62854]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:29 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:29 smithi204 ceph-mon[66572]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:29 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:29 smithi044 ceph-mon[66366]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:29 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:30.415 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.414+0000 7fdc19ffb700 1 -- 172.21.15.204:0/70197721 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc1c062750 con 0x7fdc080a2890 2024-03-19T00:32:30.415 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-03-19T00:32:30.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdc080a2890 msgr2=0x7fdc080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:30.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdc080a2890 0x7fdc080a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009be0 tx=0x7fdc14006000 comp rx=0 tx=0).stop 2024-03-19T00:32:30.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 msgr2=0x7fdc1c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:30.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 0x7fdc1c1989c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e9d0 tx=0x7fdc0c00ece0 comp rx=0 tx=0).stop 2024-03-19T00:32:30.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 shutdown_connections 2024-03-19T00:32:30.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdc080a2890 0x7fdc080a4d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:30.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c103f00 0x7fdc1c19fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:30.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc1c102360 0x7fdc1c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:30.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 --2- 172.21.15.204:0/70197721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc1c103560 0x7fdc1c1989c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:30.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 >> 172.21.15.204:0/70197721 conn(0x7fdc1c0fd920 msgr2=0x7fdc1c1068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:32:30.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 shutdown_connections 2024-03-19T00:32:30.418 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:32:30.416+0000 7fdc22e7a700 1 -- 172.21.15.204:0/70197721 wait complete. 2024-03-19T00:32:31.043 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.5.service 2024-03-19T00:32:31.045 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-03-19T00:32:31.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T00:32:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:32:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:31 smithi007 ceph-mon[62854]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:31 smithi007 ceph-mon[62854]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T00:32:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:31 smithi204 ceph-mon[66572]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:31 smithi204 ceph-mon[66572]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T00:32:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:31 smithi044 ceph-mon[66366]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:31 smithi044 ceph-mon[66366]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T00:32:32.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:32 smithi007 ceph-mon[62854]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T00:32:32.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:32 smithi007 ceph-mon[62854]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T00:32:32.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:32 smithi007 ceph-mon[62854]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:32:32.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:32 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:32 smithi204 ceph-mon[66572]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T00:32:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:32 smithi204 ceph-mon[66572]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T00:32:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:32 smithi204 ceph-mon[66572]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:32:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:32 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:32.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:32 smithi044 ceph-mon[66366]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T00:32:32.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:32 smithi044 ceph-mon[66366]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T00:32:32.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:32 smithi044 ceph-mon[66366]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:32:32.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:32 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.401 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:32:33.086+0000 7ff13dfe3700 -1 osd.5 0 waiting for initial osdmap 2024-03-19T00:32:33.401 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:32:33.094+0000 7ff135dce700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T00:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T00:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-19T00:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-19T00:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00853746 s, 1.2 GB/s 2024-03-19T00:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-19T00:32:33.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] boot 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T00:32:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' 2024-03-19T00:32:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] boot 2024-03-19T00:32:33.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T00:32:33.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-19T00:32:33.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: from='osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537]' entity='osd.5' 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: osd.5 [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] boot 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T00:32:33.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T00:32:34.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-03-19T00:32:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:32:34.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:32:34.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:32:34.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:34.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:34.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:34.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:34.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.712 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:34 smithi007 ceph-mon[62854]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:34.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:34.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:34 smithi204 ceph-mon[66572]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:34.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:32:34.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:34.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:34 smithi044 ceph-mon[66366]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T00:32:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:35 smithi007 ceph-mon[62854]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:35 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi204 2024-03-19T00:32:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:35 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi204 to 8544M 2024-03-19T00:32:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:35 smithi007 ceph-mon[62854]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T00:32:35.757 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:35 smithi044 ceph-mon[66366]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:35.757 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:35 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi204 2024-03-19T00:32:35.757 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:35 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi204 to 8544M 2024-03-19T00:32:35.757 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:35 smithi044 ceph-mon[66366]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T00:32:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:35 smithi204 ceph-mon[66572]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:35 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi204 2024-03-19T00:32:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:35 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi204 to 8544M 2024-03-19T00:32:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:35 smithi204 ceph-mon[66572]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T00:32:36.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 -- 172.21.15.7:0/1955000000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 msgr2=0x7f487c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1955000000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c107b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f486c009990 tx=0x7f486c009ca0 comp rx=0 tx=0).stop 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 -- 172.21.15.7:0/1955000000 shutdown_connections 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1955000000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c108060 0x7f487c10a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1955000000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f487c102e00 0x7f487c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1955000000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c107b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 -- 172.21.15.7:0/1955000000 >> 172.21.15.7:0/1955000000 conn(0x7f487c0fc830 msgr2=0x7f487c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.150+0000 7f4883ea2700 1 -- 172.21.15.7:0/1955000000 shutdown_connections 2024-03-19T00:32:36.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 -- 172.21.15.7:0/1955000000 wait complete. 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 Processor -- start 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 -- start start 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c102e00 0x7f487c102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f487c108060 0x7f487c1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10def0 con 0x7f487c105730 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10dd70 con 0x7f487c108060 2024-03-19T00:32:36.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f4883ea2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10e070 con 0x7f487c102e00 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f488143d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f488143d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c100db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46186/0 (socket says 172.21.15.7:46186) 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f488143d700 1 -- 172.21.15.7:0/1292898342 learned_addr learned my addr 172.21.15.7:0/1292898342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.151+0000 7f488243f700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f487c108060 0x7f487c1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4881c3e700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c102e00 0x7f487c102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f488143d700 1 -- 172.21.15.7:0/1292898342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c102e00 msgr2=0x7f487c102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f488143d700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c102e00 0x7f487c102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f488143d700 1 -- 172.21.15.7:0/1292898342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f487c108060 msgr2=0x7f487c1012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f488143d700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f487c108060 0x7f487c1012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f488143d700 1 -- 172.21.15.7:0/1292898342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c101830 con 0x7f487c105730 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4881c3e700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c102e00 0x7f487c102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f488143d700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c100db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f486c000c00 tx=0x7f486c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c027070 con 0x7f487c105730 2024-03-19T00:32:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c101ac0 con 0x7f487c105730 2024-03-19T00:32:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486c022470 con 0x7f487c105730 2024-03-19T00:32:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c00f6c0 con 0x7f487c105730 2024-03-19T00:32:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.152+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1a6f90 con 0x7f487c105730 2024-03-19T00:32:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.153+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c068fd0 con 0x7f487c105730 2024-03-19T00:32:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.154+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f486c02a070 con 0x7f487c105730 2024-03-19T00:32:36.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.155+0000 7f4872ffd700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f48680a2890 0x7f48680a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:32:36.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.155+0000 7f4881c3e700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f48680a2890 0x7f48680a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:32:36.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.155+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f486c0cec10 con 0x7f487c105730 2024-03-19T00:32:36.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.155+0000 7f4881c3e700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f48680a2890 0x7f48680a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4878007900 tx=0x7f4878008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:32:36.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.159+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f486c05e740 con 0x7f487c105730 2024-03-19T00:32:36.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:36.376+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f487c062750 con 0x7f48680a2890 2024-03-19T00:32:36.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:36 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:32:36.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:36 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:32:36.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:36 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:36.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:36 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:32:36.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:36 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:32:36.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:36 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:36.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:36 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:32:36.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:36 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:32:36.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:36 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:37 smithi007 ceph-mon[62854]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:37 smithi007 ceph-mon[62854]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:32:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:37 smithi007 ceph-mon[62854]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T00:32:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:37 smithi204 ceph-mon[66572]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:37 smithi204 ceph-mon[66572]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:32:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:37 smithi204 ceph-mon[66572]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T00:32:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:37 smithi044 ceph-mon[66366]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:37 smithi044 ceph-mon[66366]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:32:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:37 smithi044 ceph-mon[66366]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T00:32:38.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:38 smithi007 ceph-mon[62854]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T00:32:38.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:38 smithi204 ceph-mon[66572]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T00:32:38.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:38 smithi044 ceph-mon[66366]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T00:32:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:39 smithi007 ceph-mon[62854]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:39 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1385874715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dfd817f-972c-4c18-8d94-76aebea473aa"}]: dispatch 2024-03-19T00:32:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:39 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1385874715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dfd817f-972c-4c18-8d94-76aebea473aa"}]': finished 2024-03-19T00:32:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:39 smithi007 ceph-mon[62854]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T00:32:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:39 smithi204 ceph-mon[66572]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:39 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1385874715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dfd817f-972c-4c18-8d94-76aebea473aa"}]: dispatch 2024-03-19T00:32:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:39 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1385874715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dfd817f-972c-4c18-8d94-76aebea473aa"}]': finished 2024-03-19T00:32:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:39 smithi204 ceph-mon[66572]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T00:32:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:39 smithi044 ceph-mon[66366]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 514 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:39 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1385874715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dfd817f-972c-4c18-8d94-76aebea473aa"}]: dispatch 2024-03-19T00:32:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:39 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1385874715' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dfd817f-972c-4c18-8d94-76aebea473aa"}]': finished 2024-03-19T00:32:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:39 smithi044 ceph-mon[66366]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T00:32:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:40.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:40 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/365049440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:32:40.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:40 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/365049440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:32:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:40 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/365049440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:32:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:41 smithi007 ceph-mon[62854]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:41.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:41 smithi204 ceph-mon[66572]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:41.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:41 smithi044 ceph-mon[66366]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:43.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:43 smithi007 ceph-mon[62854]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T00:32:43.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:43 smithi204 ceph-mon[66572]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T00:32:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:43 smithi044 ceph-mon[66366]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T00:32:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:45 smithi007 ceph-mon[62854]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-19T00:32:45.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:45 smithi204 ceph-mon[66572]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-19T00:32:45.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:45 smithi044 ceph-mon[66366]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-19T00:32:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:47 smithi007 ceph-mon[62854]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-19T00:32:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T00:32:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:47 smithi007 ceph-mon[62854]: Deploying daemon osd.6 on smithi007 2024-03-19T00:32:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:47 smithi204 ceph-mon[66572]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-19T00:32:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T00:32:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:47 smithi204 ceph-mon[66572]: Deploying daemon osd.6 on smithi007 2024-03-19T00:32:47.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:47 smithi044 ceph-mon[66366]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-19T00:32:47.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T00:32:47.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:32:47.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:47 smithi044 ceph-mon[66366]: Deploying daemon osd.6 on smithi007 2024-03-19T00:32:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:49 smithi007 ceph-mon[62854]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T00:32:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:49 smithi204 ceph-mon[66572]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T00:32:49.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:49 smithi044 ceph-mon[66366]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T00:32:51.679 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:51 smithi007 ceph-mon[62854]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-03-19T00:32:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:51 smithi204 ceph-mon[66572]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-03-19T00:32:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:51 smithi044 ceph-mon[66366]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-03-19T00:32:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:53 smithi044 ceph-mon[66366]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T00:32:53.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:53.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:53.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:53 smithi007 ceph-mon[62854]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T00:32:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:53 smithi204 ceph-mon[66572]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T00:32:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:32:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:55 smithi007 ceph-mon[62854]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:55 smithi044 ceph-mon[66366]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:55 smithi204 ceph-mon[66572]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:55.934 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-03-19T00:32:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.933+0000 7f4872ffd700 1 -- 172.21.15.7:0/1292898342 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f487c062750 con 0x7f48680a2890 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f48680a2890 msgr2=0x7f48680a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f48680a2890 0x7f48680a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4878007900 tx=0x7f4878008040 comp rx=0 tx=0).stop 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 msgr2=0x7f487c100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c100db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f486c000c00 tx=0x7f486c0047b0 comp rx=0 tx=0).stop 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 shutdown_connections 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f48680a2890 0x7f48680a4d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f487c108060 0x7f487c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f487c102e00 0x7f487c102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 --2- 172.21.15.7:0/1292898342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487c105730 0x7f487c100db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 >> 172.21.15.7:0/1292898342 conn(0x7f487c0fc830 msgr2=0x7f487c103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:32:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 shutdown_connections 2024-03-19T00:32:55.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:32:55.936+0000 7f4883ea2700 1 -- 172.21.15.7:0/1292898342 wait complete. 2024-03-19T00:32:56.696 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.6.service 2024-03-19T00:32:56.699 INFO:tasks.cephadm:Deploying osd.7 on smithi044 with /dev/vg_nvme/lv_2... 2024-03-19T00:32:56.699 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T00:32:56.953 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:32:57.075 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.075 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.075 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:56 smithi044 ceph-mon[66366]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T00:32:57.075 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.075 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:56 smithi007 ceph-mon[62854]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T00:32:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:56 smithi204 ceph-mon[66572]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T00:32:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:32:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:57 smithi044 ceph-mon[66366]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:57 smithi044 ceph-mon[66366]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T00:32:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:57 smithi044 ceph-mon[66366]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T00:32:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:57 smithi044 ceph-mon[66366]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:32:58.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:58.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:57 smithi007 ceph-mon[62854]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:58.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:57 smithi007 ceph-mon[62854]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T00:32:58.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:57 smithi007 ceph-mon[62854]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T00:32:58.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:57 smithi007 ceph-mon[62854]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:32:58.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:57 smithi204 ceph-mon[66572]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:32:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:57 smithi204 ceph-mon[66572]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T00:32:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:57 smithi204 ceph-mon[66572]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T00:32:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:57 smithi204 ceph-mon[66572]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:32:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:58.715 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:32:58.497+0000 7fb584d8d700 -1 osd.6 0 waiting for initial osdmap 2024-03-19T00:32:58.715 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:32:58.507+0000 7fb57cb78700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:32:59.009 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T00:32:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T00:32:59.010 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-19T00:32:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-19T00:32:59.010 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121348 s, 864 MB/s 2024-03-19T00:32:59.010 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-19T00:32:59.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-mon[62854]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:32:59.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-mon[62854]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T00:32:59.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-mon[62854]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' 2024-03-19T00:32:59.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:58 smithi204 ceph-mon[66572]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:32:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:58 smithi204 ceph-mon[66572]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T00:32:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:58 smithi204 ceph-mon[66572]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' 2024-03-19T00:32:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.312 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:58 smithi044 ceph-mon[66366]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:32:59.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:58 smithi044 ceph-mon[66366]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T00:32:59.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:58 smithi044 ceph-mon[66366]: from='osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259]' entity='osd.6' 2024-03-19T00:32:59.313 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:32:59.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-03-19T00:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:33:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:59 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:33:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:59 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:33:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:59 smithi044 ceph-mon[66366]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:33:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:59 smithi044 ceph-mon[66366]: osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] boot 2024-03-19T00:33:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:59 smithi044 ceph-mon[66366]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T00:33:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:32:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:33:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:59 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:33:00.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:59 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:33:00.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:59 smithi007 ceph-mon[62854]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:33:00.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:59 smithi007 ceph-mon[62854]: osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] boot 2024-03-19T00:33:00.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:59 smithi007 ceph-mon[62854]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T00:33:00.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:32:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:33:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:59 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:33:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:59 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:33:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:59 smithi204 ceph-mon[66572]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-19T00:33:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:59 smithi204 ceph-mon[66572]: osd.6 [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] boot 2024-03-19T00:33:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:59 smithi204 ceph-mon[66572]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T00:33:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:32:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T00:33:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi007 2024-03-19T00:33:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:01 smithi204 ceph-mon[66572]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T00:33:01.793 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi007 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.794 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:01.797 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:01.797 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.798 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:01.798 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:01.798 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:01.798 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.798 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:01 smithi044 ceph-mon[66366]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 -- 172.21.15.44:0/2396334133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102de0 msgr2=0x7f49a4109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 --2- 172.21.15.44:0/2396334133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102de0 0x7f49a4109690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f499c0099c0 tx=0x7f499c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 -- 172.21.15.44:0/2396334133 shutdown_connections 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 --2- 172.21.15.44:0/2396334133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4102de0 0x7f49a4109690 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 --2- 172.21.15.44:0/2396334133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4101240 0x7f49a4101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 --2- 172.21.15.44:0/2396334133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49a4102440 0x7f49a41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 -- 172.21.15.44:0/2396334133 >> 172.21.15.44:0/2396334133 conn(0x7f49a40fc800 msgr2=0x7f49a40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:33:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 -- 172.21.15.44:0/2396334133 shutdown_connections 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.900+0000 7f49a883d700 1 -- 172.21.15.44:0/2396334133 wait complete. 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 Processor -- start 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 -- start start 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 0x7f49a41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49a4102440 0x7f49a41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4102de0 0x7f49a419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410cd70 con 0x7f49a4102de0 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410cbf0 con 0x7f49a4101240 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a883d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49a410cef0 con 0x7f49a4102440 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.901+0000 7f49a259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 0x7f49a41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 0x7f49a41984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32974/0 (socket says 172.21.15.44:32974) 2024-03-19T00:33:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 -- 172.21.15.44:0/779930307 learned_addr learned my addr 172.21.15.44:0/779930307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a1d9b700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49a4102440 0x7f49a41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a2d9d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4102de0 0x7f49a419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 -- 172.21.15.44:0/779930307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49a4102440 msgr2=0x7f49a41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49a4102440 0x7f49a41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 -- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4102de0 msgr2=0x7f49a419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4102de0 0x7f49a419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 -- 172.21.15.44:0/779930307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a41a0100 con 0x7f49a4101240 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a2d9d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4102de0 0x7f49a419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a259c700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 0x7f49a41984a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f499400b9c0 tx=0x7f499400bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994019070 con 0x7f49a4101240 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f499400ce00 con 0x7f49a4101240 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a41a03f0 con 0x7f49a4101240 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994012340 con 0x7f49a4101240 2024-03-19T00:33:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.902+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41a0a00 con 0x7f49a4101240 2024-03-19T00:33:01.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.903+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a4068fd0 con 0x7f49a4101240 2024-03-19T00:33:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.904+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f499401c070 con 0x7f49a4101240 2024-03-19T00:33:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.905+0000 7f499b7fe700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f49900a2910 0x7f49900a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.905+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f49940cd000 con 0x7f49a4101240 2024-03-19T00:33:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.905+0000 7f49a1d9b700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f49900a2910 0x7f49900a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.906+0000 7f49a1d9b700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f49900a2910 0x7f49900a4dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f498c004630 tx=0x7f498c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:33:01.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:01.910+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f499409adb0 con 0x7f49a4101240 2024-03-19T00:33:01.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:01.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi007 2024-03-19T00:33:01.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:01.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:01.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:01.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:01 smithi007 ceph-mon[62854]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T00:33:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:02.129+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f49a406b140 con 0x7f49900a2910 2024-03-19T00:33:02.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:02.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:02.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:02 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:02.769 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:02.769 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:02.769 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:02 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:02.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:02.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:02.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:02 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:03 smithi204 ceph-mon[66572]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:03 smithi204 ceph-mon[66572]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:03.767 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:03 smithi044 ceph-mon[66366]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:03.767 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:03 smithi044 ceph-mon[66366]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:03.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:03 smithi007 ceph-mon[62854]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:03.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:03 smithi007 ceph-mon[62854]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:05 smithi044 ceph-mon[66366]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:05 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/2017301506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]: dispatch 2024-03-19T00:33:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:05 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]: dispatch 2024-03-19T00:33:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:05 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]': finished 2024-03-19T00:33:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:05 smithi044 ceph-mon[66366]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T00:33:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:05 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:05 smithi204 ceph-mon[66572]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:05 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/2017301506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]: dispatch 2024-03-19T00:33:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:05 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]: dispatch 2024-03-19T00:33:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:05 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]': finished 2024-03-19T00:33:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:05 smithi204 ceph-mon[66572]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T00:33:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:05 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:05 smithi007 ceph-mon[62854]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:05 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/2017301506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]: dispatch 2024-03-19T00:33:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:05 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]: dispatch 2024-03-19T00:33:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:05 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f17cf883-f837-421c-8306-e1091b9b5f73"}]': finished 2024-03-19T00:33:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:05 smithi007 ceph-mon[62854]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T00:33:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:05 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:06.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/981175019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:06.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/981175019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:07.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/981175019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:07 smithi044 ceph-mon[66366]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:07.942 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:07 smithi204 ceph-mon[66572]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:07.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:07 smithi007 ceph-mon[62854]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:09.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:09 smithi044 ceph-mon[66366]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:09 smithi007 ceph-mon[62854]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:09 smithi204 ceph-mon[66572]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:11.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:11 smithi044 ceph-mon[66366]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:11 smithi007 ceph-mon[62854]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:11 smithi204 ceph-mon[66572]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:12.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:12 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T00:33:12.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:12 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:12.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:12 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T00:33:12.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:12 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:12 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T00:33:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:12 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:13 smithi044 ceph-mon[66366]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:13 smithi044 ceph-mon[66366]: Deploying daemon osd.7 on smithi044 2024-03-19T00:33:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:13 smithi007 ceph-mon[62854]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:13 smithi007 ceph-mon[62854]: Deploying daemon osd.7 on smithi044 2024-03-19T00:33:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:13 smithi204 ceph-mon[66572]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:13 smithi204 ceph-mon[66572]: Deploying daemon osd.7 on smithi044 2024-03-19T00:33:15.805 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:15 smithi044 ceph-mon[66366]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:15.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:15 smithi007 ceph-mon[62854]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:16.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:15 smithi204 ceph-mon[66572]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:17.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:17 smithi044 ceph-mon[66366]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:17 smithi007 ceph-mon[62854]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:18.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:17 smithi204 ceph-mon[66572]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:19.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:19 smithi007 ceph-mon[62854]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:19.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:19 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:19.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:19 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:19.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:19 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:19 smithi044 ceph-mon[66366]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:19 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:19 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:19 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:19 smithi204 ceph-mon[66572]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:19 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:19 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:19 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:21 smithi044 ceph-mon[66366]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:21 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:21 smithi007 ceph-mon[62854]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:21 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:21 smithi204 ceph-mon[66572]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:21 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:22.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.349+0000 7f499b7fe700 1 -- 172.21.15.44:0/779930307 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49a406b140 con 0x7f49900a2910 2024-03-19T00:33:22.351 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 7 on host 'smithi044' 2024-03-19T00:33:22.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f49900a2910 msgr2=0x7f49900a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:22.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f49900a2910 0x7f49900a4dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f498c004630 tx=0x7f498c00b3c0 comp rx=0 tx=0).stop 2024-03-19T00:33:22.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 msgr2=0x7f49a41984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 0x7f49a41984a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f499400b9c0 tx=0x7f499400bcd0 comp rx=0 tx=0).stop 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 shutdown_connections 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f49900a2910 0x7f49900a4dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49a4102de0 0x7f49a419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49a4101240 0x7f49a41984a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 --2- 172.21.15.44:0/779930307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49a4102440 0x7f49a41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 >> 172.21.15.44:0/779930307 conn(0x7f49a40fc800 msgr2=0x7f49a406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 shutdown_connections 2024-03-19T00:33:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:33:22.351+0000 7f49a883d700 1 -- 172.21.15.44:0/779930307 wait complete. 2024-03-19T00:33:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:22 smithi044 ceph-mon[66366]: from='osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T00:33:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:22 smithi044 ceph-mon[66366]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T00:33:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:22.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:22 smithi007 ceph-mon[62854]: from='osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T00:33:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:22 smithi007 ceph-mon[62854]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T00:33:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:22 smithi204 ceph-mon[66572]: from='osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T00:33:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:22 smithi204 ceph-mon[66572]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T00:33:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:22.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:23.035 DEBUG:teuthology.orchestra.run.smithi044:osd.7> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.7.service 2024-03-19T00:33:23.038 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-03-19T00:33:23.038 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T00:33:23.285 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:33:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-mon[66366]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-mon[66366]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T00:33:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-mon[66366]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T00:33:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-mon[66366]: from='osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:33:23.562 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-mon[66366]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:33:23.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:23 smithi204 ceph-mon[66572]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:23.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:23 smithi204 ceph-mon[66572]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T00:33:23.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:23 smithi204 ceph-mon[66572]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T00:33:23.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:23.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:23 smithi204 ceph-mon[66572]: from='osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:33:23.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:23 smithi204 ceph-mon[66572]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:33:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:23 smithi007 ceph-mon[62854]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:23 smithi007 ceph-mon[62854]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T00:33:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:23 smithi007 ceph-mon[62854]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T00:33:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:23 smithi007 ceph-mon[62854]: from='osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:33:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:23 smithi007 ceph-mon[62854]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:33:24.321 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:33:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:33:23.991+0000 7f6caa22e700 -1 osd.7 0 waiting for initial osdmap 2024-03-19T00:33:24.322 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:33:23.999+0000 7f6ca2019700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:33:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: from='osd.7 ' entity='osd.7' 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] boot 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T00:33:24.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:24 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:33:24.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T00:33:24.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: from='osd.7 ' entity='osd.7' 2024-03-19T00:33:24.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] boot 2024-03-19T00:33:24.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T00:33:24.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:24 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: from='osd.7 ' entity='osd.7' 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: osd.7 [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] boot 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T00:33:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:24 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T00:33:25.203 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T00:33:25.203 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T00:33:25.203 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-19T00:33:25.203 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-19T00:33:25.204 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111021 s, 944 MB/s 2024-03-19T00:33:25.204 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-19T00:33:25.588 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:25 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:33:25.588 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:25 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:33:25.588 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:25 smithi044 ceph-mon[66366]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:25.588 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:25 smithi044 ceph-mon[66366]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T00:33:25.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:25 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:33:25.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:25 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:33:25.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:25 smithi204 ceph-mon[66572]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:25.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:25 smithi204 ceph-mon[66572]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T00:33:25.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:25 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:33:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:25 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:33:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:25 smithi007 ceph-mon[62854]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-19T00:33:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:25 smithi007 ceph-mon[62854]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T00:33:25.855 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-03-19T00:33:26.103 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi044 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi044 to 5697M 2024-03-19T00:33:27.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:27.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:27.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T00:33:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi044 2024-03-19T00:33:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi044 to 5697M 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi044 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi044 to 5697M 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:27.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:27.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.754+0000 7f969d904700 1 -- 172.21.15.204:0/3298413676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698105700 msgr2=0x7f9698107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.754+0000 7f969d904700 1 --2- 172.21.15.204:0/3298413676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698105700 0x7f9698107af0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f968c009990 tx=0x7f968c009ca0 comp rx=0 tx=0).stop 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 -- 172.21.15.204:0/3298413676 shutdown_connections 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 --2- 172.21.15.204:0/3298413676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9698108030 0x7f969810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 --2- 172.21.15.204:0/3298413676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9698102dd0 0x7f96981051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 --2- 172.21.15.204:0/3298413676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698105700 0x7f9698107af0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 -- 172.21.15.204:0/3298413676 >> 172.21.15.204:0/3298413676 conn(0x7f96980fc7e0 msgr2=0x7f96980fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 -- 172.21.15.204:0/3298413676 shutdown_connections 2024-03-19T00:33:27.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 -- 172.21.15.204:0/3298413676 wait complete. 2024-03-19T00:33:27.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 Processor -- start 2024-03-19T00:33:27.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.755+0000 7f969d904700 1 -- start start 2024-03-19T00:33:27.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 0x7f9698198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9698105700 0x7f9698198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9698108030 0x7f969819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f969810de00 con 0x7f9698105700 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f969810dc80 con 0x7f9698108030 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f969810df80 con 0x7f9698102dd0 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 0x7f9698198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 0x7f9698198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:60722/0 (socket says 172.21.15.204:60722) 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 -- 172.21.15.204:0/1736099625 learned_addr learned my addr 172.21.15.204:0/1736099625 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f96977fe700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9698108030 0x7f969819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f96967fc700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9698105700 0x7f9698198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 -- 172.21.15.204:0/1736099625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9698108030 msgr2=0x7f969819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9698108030 0x7f969819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:27.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 -- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9698105700 msgr2=0x7f9698198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9698105700 0x7f9698198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 -- 172.21.15.204:0/1736099625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96981a00b0 con 0x7f9698102dd0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f9696ffd700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 0x7f9698198450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f968800d980 tx=0x7f968800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688016070 con 0x7f9698102dd0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9688009d80 con 0x7f9698102dd0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981a03a0 con 0x7f9698102dd0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.756+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968800f3c0 con 0x7f9698102dd0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.757+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981a0930 con 0x7f9698102dd0 2024-03-19T00:33:27.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.758+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9698068fd0 con 0x7f9698102dd0 2024-03-19T00:33:27.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.759+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f9688019070 con 0x7f9698102dd0 2024-03-19T00:33:27.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.760+0000 7f969c902700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96800a2910 0x7f96800a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:27.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.760+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f96880cd920 con 0x7f9698102dd0 2024-03-19T00:33:27.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.760+0000 7f96967fc700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96800a2910 0x7f96800a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:27.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.761+0000 7f96967fc700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96800a2910 0x7f96800a4dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b5c0 tx=0x7f968c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:33:27.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.764+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f968809b520 con 0x7f9698102dd0 2024-03-19T00:33:27.970 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:27.969+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9698062750 con 0x7f96800a2910 2024-03-19T00:33:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:28 smithi044 ceph-mon[66366]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T00:33:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:28 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:28 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:28 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:28.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:28 smithi204 ceph-mon[66572]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T00:33:28.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:28 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:28.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:28 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:28.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:28 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:28.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:28 smithi007 ceph-mon[62854]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T00:33:28.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:28 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:28.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:28 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:28.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:28 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:29 smithi044 ceph-mon[66366]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:29 smithi044 ceph-mon[66366]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:29 smithi044 ceph-mon[66366]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T00:33:29.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:29 smithi204 ceph-mon[66572]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:29.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:29 smithi204 ceph-mon[66572]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:29.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:29 smithi204 ceph-mon[66572]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T00:33:29.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:29 smithi007 ceph-mon[62854]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:29.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:29 smithi007 ceph-mon[62854]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:29.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:29 smithi007 ceph-mon[62854]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T00:33:31.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:31 smithi204 ceph-mon[66572]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:31.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:31 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/3119470952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]: dispatch 2024-03-19T00:33:31.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:31 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]: dispatch 2024-03-19T00:33:31.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:31 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]': finished 2024-03-19T00:33:31.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:31 smithi204 ceph-mon[66572]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T00:33:31.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:31 smithi044 ceph-mon[66366]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:31 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/3119470952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]: dispatch 2024-03-19T00:33:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:31 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]: dispatch 2024-03-19T00:33:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:31 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]': finished 2024-03-19T00:33:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:31 smithi044 ceph-mon[66366]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T00:33:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:31 smithi007 ceph-mon[62854]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:31 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/3119470952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]: dispatch 2024-03-19T00:33:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:31 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]: dispatch 2024-03-19T00:33:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:31 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "295a9edc-0de0-4302-8887-8717d45ba545"}]': finished 2024-03-19T00:33:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:31 smithi007 ceph-mon[62854]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T00:33:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:32 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/3714904671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:32.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:32 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/3714904671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:32 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/3714904671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:33 smithi044 ceph-mon[66366]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:33.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:33 smithi007 ceph-mon[62854]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:33 smithi204 ceph-mon[66572]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:35.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:35 smithi044 ceph-mon[66366]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:35.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:35 smithi007 ceph-mon[62854]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:35.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:35 smithi204 ceph-mon[66572]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:37.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:37 smithi204 ceph-mon[66572]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:37.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:37 smithi044 ceph-mon[66366]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:37 smithi007 ceph-mon[62854]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:38.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T00:33:38.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:38.746 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T00:33:38.746 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:38.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T00:33:38.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:39 smithi007 ceph-mon[62854]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:39 smithi007 ceph-mon[62854]: Deploying daemon osd.8 on smithi204 2024-03-19T00:33:39.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:39 smithi204 ceph-mon[66572]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:39.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:39 smithi204 ceph-mon[66572]: Deploying daemon osd.8 on smithi204 2024-03-19T00:33:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:39 smithi044 ceph-mon[66366]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:39 smithi044 ceph-mon[66366]: Deploying daemon osd.8 on smithi204 2024-03-19T00:33:41.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:41 smithi204 ceph-mon[66572]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:41 smithi007 ceph-mon[62854]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:41 smithi044 ceph-mon[66366]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:43.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:43 smithi204 ceph-mon[66572]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:43.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:43 smithi007 ceph-mon[62854]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:43 smithi044 ceph-mon[66366]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:44.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:44.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:44.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:44.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:44.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:44.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:44.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:44.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:44.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:46.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:45 smithi204 ceph-mon[66572]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:45 smithi044 ceph-mon[66366]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:45 smithi007 ceph-mon[62854]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:47.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.587+0000 7f969c902700 1 -- 172.21.15.204:0/1736099625 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9698062750 con 0x7f96800a2910 2024-03-19T00:33:47.589 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-03-19T00:33:47.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96800a2910 msgr2=0x7f96800a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:47.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96800a2910 0x7f96800a4dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b5c0 tx=0x7f968c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:33:47.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 msgr2=0x7f9698198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:47.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 0x7f9698198450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f968800d980 tx=0x7f968800dc90 comp rx=0 tx=0).stop 2024-03-19T00:33:47.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 shutdown_connections 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96800a2910 0x7f96800a4dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9698108030 0x7f969819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9698102dd0 0x7f9698198450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 --2- 172.21.15.204:0/1736099625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9698105700 0x7f9698198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 >> 172.21.15.204:0/1736099625 conn(0x7f96980fc7e0 msgr2=0x7f96981037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 shutdown_connections 2024-03-19T00:33:47.592 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:33:47.590+0000 7f969d904700 1 -- 172.21.15.204:0/1736099625 wait complete. 2024-03-19T00:33:47.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:47.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:47.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:47.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:47.912 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:47.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:47.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:47.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:47.913 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:48.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:33:48.232 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.8.service 2024-03-19T00:33:48.234 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-03-19T00:33:48.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T00:33:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:33:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:48 smithi204 ceph-mon[66572]: from='osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T00:33:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:48 smithi204 ceph-mon[66572]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T00:33:48.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:48 smithi007 ceph-mon[62854]: from='osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T00:33:48.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:48 smithi007 ceph-mon[62854]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T00:33:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:48 smithi044 ceph-mon[66366]: from='osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T00:33:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:48 smithi044 ceph-mon[66366]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T00:33:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:49 smithi007 ceph-mon[62854]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:49 smithi007 ceph-mon[62854]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T00:33:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:49 smithi007 ceph-mon[62854]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T00:33:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:49 smithi007 ceph-mon[62854]: from='osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:33:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:49 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:49 smithi007 ceph-mon[62854]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:33:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:49 smithi204 ceph-mon[66572]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:49 smithi204 ceph-mon[66572]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T00:33:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:49 smithi204 ceph-mon[66572]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T00:33:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:49 smithi204 ceph-mon[66572]: from='osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:33:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:49 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:49 smithi204 ceph-mon[66572]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:33:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:49 smithi044 ceph-mon[66366]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:49 smithi044 ceph-mon[66366]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T00:33:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:49 smithi044 ceph-mon[66366]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T00:33:50.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:49 smithi044 ceph-mon[66366]: from='osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:33:50.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:49 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:50.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:49 smithi044 ceph-mon[66366]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:33:50.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:33:50.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:33:50.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:33:50.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T00:33:50.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:50.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:50.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-mon[66572]: from='osd.8 ' entity='osd.8' 2024-03-19T00:33:50.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:33:50.491+0000 7f559da29700 -1 osd.8 0 waiting for initial osdmap 2024-03-19T00:33:50.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:33:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:33:50.504+0000 7f5595814700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:33:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T00:33:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T00:33:50.771 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-19T00:33:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-19T00:33:50.771 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0084775 s, 1.2 GB/s 2024-03-19T00:33:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-19T00:33:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:33:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:33:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:33:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T00:33:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:51.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:50 smithi044 ceph-mon[66366]: from='osd.8 ' entity='osd.8' 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:50 smithi007 ceph-mon[62854]: from='osd.8 ' entity='osd.8' 2024-03-19T00:33:51.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-03-19T00:33:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] boot 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:51.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] boot 2024-03-19T00:33:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T00:33:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:52.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-19T00:33:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: osd.8 [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] boot 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:33:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:33:53.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:52 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi204 2024-03-19T00:33:53.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:52 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi204 to 5696M 2024-03-19T00:33:53.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:52 smithi204 ceph-mon[66572]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T00:33:53.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:52 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi204 2024-03-19T00:33:53.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:52 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi204 to 5696M 2024-03-19T00:33:53.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:52 smithi007 ceph-mon[62854]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T00:33:53.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:52 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi204 2024-03-19T00:33:53.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:52 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi204 to 5696M 2024-03-19T00:33:53.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:52 smithi044 ceph-mon[66366]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T00:33:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 -- 172.21.15.7:0/2933735182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590100600 msgr2=0x7fa590100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 --2- 172.21.15.7:0/2933735182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590100600 0x7fa590100a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009990 tx=0x7fa58c009ca0 comp rx=0 tx=0).stop 2024-03-19T00:33:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 -- 172.21.15.7:0/2933735182 shutdown_connections 2024-03-19T00:33:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 --2- 172.21.15.7:0/2933735182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590103f80 0x7fa59010a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 --2- 172.21.15.7:0/2933735182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590100600 0x7fa590100a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 --2- 172.21.15.7:0/2933735182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5901035e0 0x7fa590103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 -- 172.21.15.7:0/2933735182 >> 172.21.15.7:0/2933735182 conn(0x7fa5900fbf20 msgr2=0x7fa5900fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 -- 172.21.15.7:0/2933735182 shutdown_connections 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 -- 172.21.15.7:0/2933735182 wait complete. 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 Processor -- start 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.636+0000 7fa59762c700 1 -- start start 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590100600 0x7fa5901984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 0x7fa590198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa590103f80 0x7fa59019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa59010de20 con 0x7fa5901035e0 2024-03-19T00:33:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa59010dca0 con 0x7fa590100600 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa59010dfa0 con 0x7fa590103f80 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 0x7fa590198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 0x7fa590198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60236/0 (socket says 172.21.15.7:60236) 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 -- 172.21.15.7:0/176004603 learned_addr learned my addr 172.21.15.7:0/176004603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa595bc9700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa590103f80 0x7fa59019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa5953c8700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590100600 0x7fa5901984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 -- 172.21.15.7:0/176004603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa590103f80 msgr2=0x7fa59019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa590103f80 0x7fa59019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 -- 172.21.15.7:0/176004603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590100600 msgr2=0x7fa5901984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590100600 0x7fa5901984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 -- 172.21.15.7:0/176004603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5901a0150 con 0x7fa5901035e0 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa5953c8700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590100600 0x7fa5901984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa594bc7700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 0x7fa590198a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa58000b560 tx=0x7fa58000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:33:53.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580015020 con 0x7fa5901035e0 2024-03-19T00:33:53.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5901a0440 con 0x7fa5901035e0 2024-03-19T00:33:53.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa580011ce0 con 0x7fa5901035e0 2024-03-19T00:33:53.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580010d60 con 0x7fa5901035e0 2024-03-19T00:33:53.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.637+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5901a09d0 con 0x7fa5901035e0 2024-03-19T00:33:53.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.638+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa590068fd0 con 0x7fa5901035e0 2024-03-19T00:33:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.639+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fa580018020 con 0x7fa5901035e0 2024-03-19T00:33:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.640+0000 7fa5867fc700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa57c0a2890 0x7fa57c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:33:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.640+0000 7fa5953c8700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa57c0a2890 0x7fa57c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:33:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.640+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7fa5800cd890 con 0x7fa5901035e0 2024-03-19T00:33:53.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.641+0000 7fa5953c8700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa57c0a2890 0x7fa57c0a4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0036e0 tx=0x7fa58c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:33:53.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.645+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa58001f020 con 0x7fa5901035e0 2024-03-19T00:33:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:33:53.859+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa590062750 con 0x7fa57c0a2890 2024-03-19T00:33:54.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:53 smithi204 ceph-mon[66572]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:33:54.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:53 smithi204 ceph-mon[66572]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T00:33:54.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:53 smithi204 ceph-mon[66572]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T00:33:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:53 smithi044 ceph-mon[66366]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:33:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:53 smithi044 ceph-mon[66366]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T00:33:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:53 smithi044 ceph-mon[66366]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T00:33:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:53 smithi007 ceph-mon[62854]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:33:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:53 smithi007 ceph-mon[62854]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T00:33:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:53 smithi007 ceph-mon[62854]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T00:33:55.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:54 smithi204 ceph-mon[66572]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:55.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:55.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:55.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:55.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:54 smithi204 ceph-mon[66572]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T00:33:55.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:54 smithi044 ceph-mon[66366]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:55.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:54 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:55.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:54 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:55.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:54 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:55.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:54 smithi044 ceph-mon[66366]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T00:33:55.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:54 smithi007 ceph-mon[62854]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:33:55.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:33:55.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:33:55.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:33:55.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:54 smithi007 ceph-mon[62854]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T00:33:56.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:55 smithi204 ceph-mon[66572]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:33:56.023 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:55 smithi007 ceph-mon[62854]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:33:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:55 smithi044 ceph-mon[66366]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:33:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:56 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2730965644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3844148f-1d07-4f72-a3c0-e249888e8416"}]: dispatch 2024-03-19T00:33:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:56 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2730965644' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3844148f-1d07-4f72-a3c0-e249888e8416"}]': finished 2024-03-19T00:33:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:56 smithi044 ceph-mon[66366]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T00:33:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:33:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:56 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2730965644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3844148f-1d07-4f72-a3c0-e249888e8416"}]: dispatch 2024-03-19T00:33:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:56 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2730965644' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3844148f-1d07-4f72-a3c0-e249888e8416"}]': finished 2024-03-19T00:33:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:56 smithi007 ceph-mon[62854]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T00:33:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:33:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:56 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2730965644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3844148f-1d07-4f72-a3c0-e249888e8416"}]: dispatch 2024-03-19T00:33:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:56 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2730965644' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3844148f-1d07-4f72-a3c0-e249888e8416"}]': finished 2024-03-19T00:33:57.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:56 smithi204 ceph-mon[66572]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T00:33:57.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:33:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:57 smithi044 ceph-mon[66366]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-19T00:33:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/224675638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:58.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:57 smithi007 ceph-mon[62854]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-19T00:33:58.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/224675638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:33:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:57 smithi204 ceph-mon[66572]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-19T00:33:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/224675638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:33:59 smithi044 ceph-mon[66366]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-19T00:34:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:33:59 smithi007 ceph-mon[62854]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-19T00:34:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:33:59 smithi204 ceph-mon[66572]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-19T00:34:02.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:01 smithi007 ceph-mon[62854]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-19T00:34:02.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:01 smithi044 ceph-mon[66366]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-19T00:34:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:01 smithi204 ceph-mon[66572]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-19T00:34:03.879 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:03 smithi007 ceph-mon[62854]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T00:34:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:03 smithi044 ceph-mon[66366]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T00:34:04.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:03 smithi204 ceph-mon[66572]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T00:34:05.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:04 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T00:34:05.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:04 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:04 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T00:34:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:04 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:05.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:04 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T00:34:05.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:04 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:06.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:05 smithi007 ceph-mon[62854]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-19T00:34:06.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:05 smithi007 ceph-mon[62854]: Deploying daemon osd.9 on smithi007 2024-03-19T00:34:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:05 smithi044 ceph-mon[66366]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-19T00:34:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:05 smithi044 ceph-mon[66366]: Deploying daemon osd.9 on smithi007 2024-03-19T00:34:06.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:05 smithi204 ceph-mon[66572]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-19T00:34:06.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:05 smithi204 ceph-mon[66572]: Deploying daemon osd.9 on smithi007 2024-03-19T00:34:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:07 smithi044 ceph-mon[66366]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:07 smithi007 ceph-mon[62854]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:07 smithi204 ceph-mon[66572]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:09.952 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:09 smithi007 ceph-mon[62854]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:09 smithi044 ceph-mon[66366]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:09 smithi204 ceph-mon[66572]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:10.861 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:10.861 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:10.861 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:10 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:11.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:10 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:11.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:10 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:11.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:10 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:11.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:10 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:11.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:10 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:11.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:10 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:11 smithi044 ceph-mon[66366]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:12.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:11 smithi007 ceph-mon[62854]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:11 smithi204 ceph-mon[66572]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:14.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:13 smithi007 ceph-mon[62854]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:14.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:14.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:14.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:13 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:13 smithi044 ceph-mon[66366]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:13 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.131 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-03-19T00:34:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.130+0000 7fa5867fc700 1 -- 172.21.15.7:0/176004603 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa590062750 con 0x7fa57c0a2890 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa57c0a2890 msgr2=0x7fa57c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa57c0a2890 0x7fa57c0a4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0036e0 tx=0x7fa58c005fb0 comp rx=0 tx=0).stop 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 msgr2=0x7fa590198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 0x7fa590198a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa58000b560 tx=0x7fa58000b870 comp rx=0 tx=0).stop 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 shutdown_connections 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa57c0a2890 0x7fa57c0a4d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa590103f80 0x7fa59019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa590100600 0x7fa5901984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 --2- 172.21.15.7:0/176004603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901035e0 0x7fa590198a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 >> 172.21.15.7:0/176004603 conn(0x7fa5900fbf20 msgr2=0x7fa590106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:34:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 shutdown_connections 2024-03-19T00:34:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:34:14.132+0000 7fa59762c700 1 -- 172.21.15.7:0/176004603 wait complete. 2024-03-19T00:34:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:13 smithi204 ceph-mon[66572]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:14.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:13 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:14.837 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.9.service 2024-03-19T00:34:14.839 INFO:tasks.cephadm:Deploying osd.10 on smithi044 with /dev/vg_nvme/lv_1... 2024-03-19T00:34:14.839 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T00:34:15.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:14 smithi007 ceph-mon[62854]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T00:34:15.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:14 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:15.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:14 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:15.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:14 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:15.093 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:34:15.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:14 smithi044 ceph-mon[66366]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T00:34:15.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:14 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:15.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:14 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:15.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:14 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:14 smithi204 ceph-mon[66572]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T00:34:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:14 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:14 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:14 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:16.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-mon[62854]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:16.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-mon[62854]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T00:34:16.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-mon[62854]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T00:34:16.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-mon[62854]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:34:16.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:16.104 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:34:15.814+0000 7facd3560700 -1 osd.9 0 waiting for initial osdmap 2024-03-19T00:34:16.104 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:34:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:34:15.826+0000 7faccf353700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:34:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:15 smithi044 ceph-mon[66366]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:15 smithi044 ceph-mon[66366]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T00:34:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:15 smithi044 ceph-mon[66366]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T00:34:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:15 smithi044 ceph-mon[66366]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:34:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:15 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:15 smithi204 ceph-mon[66572]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:15 smithi204 ceph-mon[66572]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T00:34:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:15 smithi204 ceph-mon[66572]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T00:34:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:15 smithi204 ceph-mon[66572]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-19T00:34:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:15 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:16 smithi044 ceph-mon[66366]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:34:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:16 smithi044 ceph-mon[66366]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T00:34:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:16 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T00:34:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T00:34:17.125 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-19T00:34:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-19T00:34:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111724 s, 939 MB/s 2024-03-19T00:34:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-19T00:34:17.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:16 smithi007 ceph-mon[62854]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:34:17.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:16 smithi007 ceph-mon[62854]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T00:34:17.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:16 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:16 smithi204 ceph-mon[66572]: from='osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-19T00:34:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:16 smithi204 ceph-mon[66572]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T00:34:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:16 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:17.910 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-03-19T00:34:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:17 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:34:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:17 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:34:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:17 smithi007 ceph-mon[62854]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:17 smithi007 ceph-mon[62854]: osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] boot 2024-03-19T00:34:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:17 smithi007 ceph-mon[62854]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T00:34:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:17 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:17 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:34:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:17 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:34:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:17 smithi044 ceph-mon[66366]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:17 smithi044 ceph-mon[66366]: osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] boot 2024-03-19T00:34:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:17 smithi044 ceph-mon[66366]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T00:34:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:17 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:34:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:17 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:34:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:17 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:34:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:17 smithi204 ceph-mon[66572]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-03-19T00:34:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:17 smithi204 ceph-mon[66572]: osd.9 [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] boot 2024-03-19T00:34:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:17 smithi204 ceph-mon[66572]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T00:34:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:17 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T00:34:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T00:34:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:19.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:18 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:19.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:18 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:19.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:19.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:19.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:18 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:19.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 -- 172.21.15.44:0/4027969399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c103570 msgr2=0x7f324c1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:19.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 --2- 172.21.15.44:0/4027969399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c103570 0x7f324c1039d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f323c0099c0 tx=0x7f323c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:34:19.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 -- 172.21.15.44:0/4027969399 shutdown_connections 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 --2- 172.21.15.44:0/4027969399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324c103f10 0x7f324c10a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 --2- 172.21.15.44:0/4027969399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f324c102370 0x7f324c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 --2- 172.21.15.44:0/4027969399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c103570 0x7f324c1039d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 -- 172.21.15.44:0/4027969399 >> 172.21.15.44:0/4027969399 conn(0x7f324c0fd910 msgr2=0x7f324c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 -- 172.21.15.44:0/4027969399 shutdown_connections 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.917+0000 7f3253a4e700 1 -- 172.21.15.44:0/4027969399 wait complete. 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 Processor -- start 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 -- start start 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 0x7f324c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324c103570 0x7f324c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f324c103f10 0x7f324c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:19.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f324c10de10 con 0x7f324c103f10 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f324c10dc90 con 0x7f324c102370 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3253a4e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f324c10df90 con 0x7f324c103570 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f32517ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 0x7f324c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f32517ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 0x7f324c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54210/0 (socket says 172.21.15.44:54210) 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f32517ea700 1 -- 172.21.15.44:0/3194776765 learned_addr learned my addr 172.21.15.44:0/3194776765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3250fe9700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324c103570 0x7f324c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.918+0000 7f3251feb700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f324c103f10 0x7f324c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32517ea700 1 -- 172.21.15.44:0/3194776765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324c103570 msgr2=0x7f324c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32517ea700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324c103570 0x7f324c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32517ea700 1 -- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f324c103f10 msgr2=0x7f324c19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32517ea700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f324c103f10 0x7f324c19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32517ea700 1 -- 172.21.15.44:0/3194776765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0745c0 con 0x7f324c102370 2024-03-19T00:34:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32517ea700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 0x7f324c198390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f324800c990 tx=0x7f324800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:34:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248007940 con 0x7f324c102370 2024-03-19T00:34:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f324800f450 con 0x7f324c102370 2024-03-19T00:34:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c0748b0 con 0x7f324c102370 2024-03-19T00:34:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248018660 con 0x7f324c102370 2024-03-19T00:34:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.919+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c074e40 con 0x7f324c102370 2024-03-19T00:34:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.920+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c068fd0 con 0x7f324c102370 2024-03-19T00:34:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.921+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f324800f600 con 0x7f324c102370 2024-03-19T00:34:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.922+0000 7f32427fc700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32380a2910 0x7f32380a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.922+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f32480ce0a0 con 0x7f324c102370 2024-03-19T00:34:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.922+0000 7f3250fe9700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32380a2910 0x7f32380a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.922+0000 7f3250fe9700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32380a2910 0x7f32380a4dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f323c009fd0 tx=0x7f323c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:34:19.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:19.926+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3248014030 con 0x7f324c102370 2024-03-19T00:34:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:19 smithi044 ceph-mon[66366]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:19 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi007 2024-03-19T00:34:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:19 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-19T00:34:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:20.142+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f324c109460 con 0x7f32380a2910 2024-03-19T00:34:20.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:19 smithi007 ceph-mon[62854]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:20.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:19 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi007 2024-03-19T00:34:20.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:19 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-19T00:34:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:19 smithi204 ceph-mon[66572]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:19 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi007 2024-03-19T00:34:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:19 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-19T00:34:21.048 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:34:21.049 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:34:21.049 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:20 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:21.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:34:21.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:34:21.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:20 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:34:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:34:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:20 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:22.099 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:21 smithi044 ceph-mon[66366]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:22.099 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:21 smithi044 ceph-mon[66366]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:34:22.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:21 smithi007 ceph-mon[62854]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:22.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:21 smithi007 ceph-mon[62854]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:34:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:21 smithi204 ceph-mon[66572]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:21 smithi204 ceph-mon[66572]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:34:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/556370128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]: dispatch 2024-03-19T00:34:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]: dispatch 2024-03-19T00:34:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]': finished 2024-03-19T00:34:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T00:34:24.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:24.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.44:0/3656448464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/556370128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]: dispatch 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]: dispatch 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]': finished 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.44:0/3656448464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/556370128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]: dispatch 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]: dispatch 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88ad1139-05b0-45d2-938e-bae6421c1a4c"}]': finished 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.44:0/3656448464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:25 smithi044 ceph-mon[66366]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:26.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:25 smithi007 ceph-mon[62854]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:25 smithi204 ceph-mon[66572]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:28.098 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:27 smithi044 ceph-mon[66366]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:28.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:27 smithi007 ceph-mon[62854]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:27 smithi204 ceph-mon[66572]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:29 smithi044 ceph-mon[66366]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:30.206 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:29 smithi007 ceph-mon[62854]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:29 smithi204 ceph-mon[66572]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:31.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T00:34:31.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:30 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:31.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T00:34:31.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:30 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T00:34:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:30 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:31 smithi044 ceph-mon[66366]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:31 smithi044 ceph-mon[66366]: Deploying daemon osd.10 on smithi044 2024-03-19T00:34:32.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:31 smithi007 ceph-mon[62854]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:32.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:31 smithi007 ceph-mon[62854]: Deploying daemon osd.10 on smithi044 2024-03-19T00:34:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:31 smithi204 ceph-mon[66572]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:31 smithi204 ceph-mon[66572]: Deploying daemon osd.10 on smithi044 2024-03-19T00:34:34.000 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:33 smithi044 ceph-mon[66366]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:34.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:33 smithi007 ceph-mon[62854]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:33 smithi204 ceph-mon[66572]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:35 smithi044 ceph-mon[66366]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:36.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:35 smithi007 ceph-mon[62854]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:35 smithi204 ceph-mon[66572]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:37.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:37 smithi204 ceph-mon[66572]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:37.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:37.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:37.675 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:37 smithi007 ceph-mon[62854]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:37 smithi044 ceph-mon[66366]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:39.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:39 smithi007 ceph-mon[62854]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:39 smithi204 ceph-mon[66572]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:39.799 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:39 smithi044 ceph-mon[66366]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:40.561 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 10 on host 'smithi044' 2024-03-19T00:34:40.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.559+0000 7f32427fc700 1 -- 172.21.15.44:0/3194776765 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f324c109460 con 0x7f32380a2910 2024-03-19T00:34:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32380a2910 msgr2=0x7f32380a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32380a2910 0x7f32380a4dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f323c009fd0 tx=0x7f323c005fb0 comp rx=0 tx=0).stop 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 msgr2=0x7f324c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 0x7f324c198390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f324800c990 tx=0x7f324800cca0 comp rx=0 tx=0).stop 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 shutdown_connections 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32380a2910 0x7f32380a4dd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f324c103f10 0x7f324c19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c102370 0x7f324c198390 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 --2- 172.21.15.44:0/3194776765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324c103570 0x7f324c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 >> 172.21.15.44:0/3194776765 conn(0x7f324c0fd910 msgr2=0x7f324c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 shutdown_connections 2024-03-19T00:34:40.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-19T00:34:40.562+0000 7f3253a4e700 1 -- 172.21.15.44:0/3194776765 wait complete. 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:40 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:40.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:40 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:41.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:41.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:40 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:34:41.247 DEBUG:teuthology.orchestra.run.smithi044:osd.10> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.10.service 2024-03-19T00:34:41.249 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-03-19T00:34:41.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T00:34:41.498 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:34:41.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-mon[66366]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:41.849 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-mon[66366]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T00:34:41.850 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-mon[66366]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T00:34:41.850 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:41.850 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-mon[66366]: from='osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:34:41.850 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-mon[66366]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:34:41.850 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:34:41.688+0000 7fc49941d700 -1 osd.10 0 waiting for initial osdmap 2024-03-19T00:34:41.850 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:34:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:34:41.704+0000 7fc495210700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:34:41.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:41 smithi204 ceph-mon[66572]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:41.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:41 smithi204 ceph-mon[66572]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T00:34:41.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:41 smithi204 ceph-mon[66572]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T00:34:41.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:41.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:41 smithi204 ceph-mon[66572]: from='osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:34:41.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:41 smithi204 ceph-mon[66572]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:34:41.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:41 smithi007 ceph-mon[62854]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:41 smithi007 ceph-mon[62854]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T00:34:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:41 smithi007 ceph-mon[62854]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T00:34:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:41 smithi007 ceph-mon[62854]: from='osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:34:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:41 smithi007 ceph-mon[62854]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-19T00:34:42.805 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:42 smithi044 ceph-mon[66366]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:34:42.805 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:42 smithi044 ceph-mon[66366]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T00:34:42.805 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:42.805 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:42 smithi007 ceph-mon[62854]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:34:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:42 smithi007 ceph-mon[62854]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T00:34:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:42 smithi204 ceph-mon[66572]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-19T00:34:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:42 smithi204 ceph-mon[66572]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T00:34:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.577 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T00:34:43.577 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T00:34:43.577 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-19T00:34:43.577 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-19T00:34:43.577 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112423 s, 933 MB/s 2024-03-19T00:34:43.577 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-19T00:34:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:34:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:34:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] boot 2024-03-19T00:34:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T00:34:43.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.966 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:43 smithi007 ceph-mon[62854]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T00:34:43.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:34:43.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:34:43.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:43.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.968 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] boot 2024-03-19T00:34:43.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T00:34:43.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:43 smithi204 ceph-mon[66572]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T00:34:43.995 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:34:43.995 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:34:43.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-19T00:34:43.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: osd.10 [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] boot 2024-03-19T00:34:43.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T00:34:43.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T00:34:43.996 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:43 smithi044 ceph-mon[66366]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T00:34:44.298 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-03-19T00:34:44.546 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:34:45.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:45.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T00:34:45.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi044 2024-03-19T00:34:45.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi044 to 4273M 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:45.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi044 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi044 to 4273M 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:45.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi044 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi044 to 4273M 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:34:45.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:34:46.214 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 -- 172.21.15.204:0/1333477320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454103540 msgr2=0x7f64541039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 --2- 172.21.15.204:0/1333477320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454103540 0x7f64541039a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6448009990 tx=0x7f6448009ca0 comp rx=0 tx=0).stop 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 -- 172.21.15.204:0/1333477320 shutdown_connections 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 --2- 172.21.15.204:0/1333477320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 0x7f645410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 --2- 172.21.15.204:0/1333477320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6454102340 0x7f6454102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 --2- 172.21.15.204:0/1333477320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454103540 0x7f64541039a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.211+0000 7f6459dab700 1 -- 172.21.15.204:0/1333477320 >> 172.21.15.204:0/1333477320 conn(0x7f64540fd920 msgr2=0x7f64540ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 -- 172.21.15.204:0/1333477320 shutdown_connections 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 -- 172.21.15.204:0/1333477320 wait complete. 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 Processor -- start 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 -- start start 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 0x7f6454198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6454103540 0x7f6454198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:46.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 0x7f645419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f645410ddc0 con 0x7f6454103ee0 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f645410dc40 con 0x7f6454103540 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.212+0000 7f6459dab700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f645410df40 con 0x7f6454102340 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 0x7f6454198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 0x7f6454198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38052/0 (socket says 172.21.15.204:38052) 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 -- 172.21.15.204:0/1089115189 learned_addr learned my addr 172.21.15.204:0/1089115189 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6452ffd700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6454103540 0x7f6454198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6453fff700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 0x7f645419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 -- 172.21.15.204:0/1089115189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6454103540 msgr2=0x7f6454198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6454103540 0x7f6454198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 -- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 msgr2=0x7f645419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 0x7f645419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 -- 172.21.15.204:0/1089115189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64541a0060 con 0x7f6454102340 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6453fff700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 0x7f645419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:34:46.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f64537fe700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 0x7f6454198400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f644400c9a0 tx=0x7f644400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:34:46.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444004220 con 0x7f6454102340 2024-03-19T00:34:46.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64541a0350 con 0x7f6454102340 2024-03-19T00:34:46.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6444007d10 con 0x7f6454102340 2024-03-19T00:34:46.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444003d00 con 0x7f6454102340 2024-03-19T00:34:46.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.213+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64541a08e0 con 0x7f6454102340 2024-03-19T00:34:46.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.214+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6454068fd0 con 0x7f6454102340 2024-03-19T00:34:46.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.216+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f6444007860 con 0x7f6454102340 2024-03-19T00:34:46.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.216+0000 7f6450ff9700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f643c0a2890 0x7f643c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:34:46.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.217+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f64440ce3d0 con 0x7f6454102340 2024-03-19T00:34:46.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.217+0000 7f6452ffd700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f643c0a2890 0x7f643c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:34:46.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.217+0000 7f6452ffd700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f643c0a2890 0x7f643c0a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f644800b5c0 tx=0x7f6448005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:34:46.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.220+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6444014030 con 0x7f6454102340 2024-03-19T00:34:46.431 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:34:46.430+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6454062750 con 0x7f643c0a2890 2024-03-19T00:34:46.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:46 smithi044 ceph-mon[66366]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T00:34:46.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:46 smithi007 ceph-mon[62854]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T00:34:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:46 smithi204 ceph-mon[66572]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T00:34:47.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:47 smithi044 ceph-mon[66366]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-19T00:34:47.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:47 smithi044 ceph-mon[66366]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T00:34:47.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:47 smithi044 ceph-mon[66366]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:34:47.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:34:47.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:34:47.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:47.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:47 smithi007 ceph-mon[62854]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-19T00:34:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:47 smithi007 ceph-mon[62854]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T00:34:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:47 smithi007 ceph-mon[62854]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:34:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:34:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:34:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:47.736 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:47 smithi204 ceph-mon[66572]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-19T00:34:47.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:47 smithi204 ceph-mon[66572]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T00:34:47.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:47 smithi204 ceph-mon[66572]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:34:47.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T00:34:47.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T00:34:47.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:49.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:49 smithi044 ceph-mon[66366]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-19T00:34:49.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:49 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/3814338830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]: dispatch 2024-03-19T00:34:49.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:49 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]: dispatch 2024-03-19T00:34:49.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:49 smithi044 ceph-mon[66366]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]': finished 2024-03-19T00:34:49.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:49 smithi044 ceph-mon[66366]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T00:34:49.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:49 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:34:49.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:49 smithi007 ceph-mon[62854]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-19T00:34:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:49 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/3814338830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]: dispatch 2024-03-19T00:34:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:49 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]: dispatch 2024-03-19T00:34:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:49 smithi007 ceph-mon[62854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]': finished 2024-03-19T00:34:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:49 smithi007 ceph-mon[62854]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T00:34:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:49 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:34:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:49 smithi204 ceph-mon[66572]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-03-19T00:34:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:49 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/3814338830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]: dispatch 2024-03-19T00:34:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:49 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]: dispatch 2024-03-19T00:34:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:49 smithi204 ceph-mon[66572]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b004ee9-0ccb-4258-bc42-4b24baf7931f"}]': finished 2024-03-19T00:34:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:49 smithi204 ceph-mon[66572]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T00:34:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:49 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:34:50.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:50 smithi044 ceph-mon[66366]: from='client.? 172.21.15.204:0/2660951977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:50.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:50 smithi007 ceph-mon[62854]: from='client.? 172.21.15.204:0/2660951977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:50.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:50 smithi204 ceph-mon[66572]: from='client.? 172.21.15.204:0/2660951977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T00:34:51.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:51 smithi044 ceph-mon[66366]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-19T00:34:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:51 smithi007 ceph-mon[62854]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-19T00:34:51.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:51 smithi204 ceph-mon[66572]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-19T00:34:53.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:53 smithi204 ceph-mon[66572]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:53 smithi044 ceph-mon[66366]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:53.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:53 smithi007 ceph-mon[62854]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:55.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:55 smithi204 ceph-mon[66572]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:55 smithi044 ceph-mon[66366]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:55.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:55 smithi007 ceph-mon[62854]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:57 smithi204 ceph-mon[66572]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T00:34:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:57 smithi204 ceph-mon[66572]: Deploying daemon osd.11 on smithi204 2024-03-19T00:34:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:57 smithi044 ceph-mon[66366]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T00:34:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:57 smithi044 ceph-mon[66366]: Deploying daemon osd.11 on smithi204 2024-03-19T00:34:57.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:57 smithi007 ceph-mon[62854]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:57.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T00:34:57.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:34:57.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:57 smithi007 ceph-mon[62854]: Deploying daemon osd.11 on smithi204 2024-03-19T00:34:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:34:59 smithi204 ceph-mon[66572]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:34:59 smithi044 ceph-mon[66366]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:34:59.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:34:59 smithi007 ceph-mon[62854]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:01 smithi204 ceph-mon[66572]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:01.605 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:01 smithi044 ceph-mon[66366]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:01.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:01 smithi007 ceph-mon[62854]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:03 smithi204 ceph-mon[66572]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:03 smithi044 ceph-mon[66366]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:03.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:03 smithi007 ceph-mon[62854]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:05 smithi204 ceph-mon[66572]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:05 smithi044 ceph-mon[66366]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:05.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:05 smithi007 ceph-mon[62854]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:05.850 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.848+0000 7f6450ff9700 1 -- 172.21.15.204:0/1089115189 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6454062750 con 0x7f643c0a2890 2024-03-19T00:35:05.850 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-03-19T00:35:05.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f643c0a2890 msgr2=0x7f643c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:05.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f643c0a2890 0x7f643c0a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f644800b5c0 tx=0x7f6448005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:05.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 msgr2=0x7f6454198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:05.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 0x7f6454198400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f644400c9a0 tx=0x7f644400ccb0 comp rx=0 tx=0).stop 2024-03-19T00:35:05.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 shutdown_connections 2024-03-19T00:35:05.852 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f643c0a2890 0x7f643c0a4d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:05.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6454103ee0 0x7f645419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:05.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6454102340 0x7f6454198400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:05.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 --2- 172.21.15.204:0/1089115189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6454103540 0x7f6454198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:05.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 >> 172.21.15.204:0/1089115189 conn(0x7f64540fd920 msgr2=0x7f64541068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:05.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 shutdown_connections 2024-03-19T00:35:05.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:05.850+0000 7f6459dab700 1 -- 172.21.15.204:0/1089115189 wait complete. 2024-03-19T00:35:06.509 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.11.service 2024-03-19T00:35:06.512 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-19T00:35:06.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-19T00:35:06.783 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:35:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:06 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:06 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:06 smithi044 ceph-mon[66366]: from='osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T00:35:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:06 smithi044 ceph-mon[66366]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T00:35:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:06 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:06 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:06 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:06 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:06 smithi007 ceph-mon[62854]: from='osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T00:35:07.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:06 smithi007 ceph-mon[62854]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T00:35:07.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:06 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.125 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:06 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:06 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:06 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:06 smithi204 ceph-mon[66572]: from='osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T00:35:07.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:06 smithi204 ceph-mon[66572]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T00:35:07.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:06 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:07.252 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:06 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:07 smithi044 ceph-mon[66366]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:07 smithi044 ceph-mon[66366]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T00:35:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:07 smithi044 ceph-mon[66366]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T00:35:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:07 smithi044 ceph-mon[66366]: from='osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:35:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:07 smithi044 ceph-mon[66366]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:35:08.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:07 smithi007 ceph-mon[62854]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:08.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:07 smithi007 ceph-mon[62854]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T00:35:08.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:07 smithi007 ceph-mon[62854]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T00:35:08.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:08.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:07 smithi007 ceph-mon[62854]: from='osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:35:08.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:07 smithi007 ceph-mon[62854]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:35:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:07 smithi204 ceph-mon[66572]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:07 smithi204 ceph-mon[66572]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T00:35:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:07 smithi204 ceph-mon[66572]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T00:35:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:08.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:07 smithi204 ceph-mon[66572]: from='osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:35:08.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:07 smithi204 ceph-mon[66572]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-19T00:35:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 -- 172.21.15.7:0/473920995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270101240 msgr2=0x7f2270101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 --2- 172.21.15.7:0/473920995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270101240 0x7f2270101660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22640099c0 tx=0x7f2264009cd0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 -- 172.21.15.7:0/473920995 shutdown_connections 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 --2- 172.21.15.7:0/473920995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270102de0 0x7f2270109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 --2- 172.21.15.7:0/473920995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270101240 0x7f2270101660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 --2- 172.21.15.7:0/473920995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270102440 0x7f22701028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 -- 172.21.15.7:0/473920995 >> 172.21.15.7:0/473920995 conn(0x7f22700fc7e0 msgr2=0x7f22700fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 -- 172.21.15.7:0/473920995 shutdown_connections 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.496+0000 7f2276d06700 1 -- 172.21.15.7:0/473920995 wait complete. 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 Processor -- start 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 -- start start 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270101240 0x7f227006ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 0x7f227019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270102de0 0x7f227019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f227010ccd0 con 0x7f2270102440 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f227010cb50 con 0x7f2270101240 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2276d06700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f227010ce50 con 0x7f2270102de0 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 0x7f227019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 0x7f227019d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59740/0 (socket says 172.21.15.7:59740) 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 -- 172.21.15.7:0/2789788587 learned_addr learned my addr 172.21.15.7:0/2789788587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f22752a3700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270102de0 0x7f227019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2274aa2700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270101240 0x7f227006ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 -- 172.21.15.7:0/2789788587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270102de0 msgr2=0x7f227019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270102de0 0x7f227019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 -- 172.21.15.7:0/2789788587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270101240 msgr2=0x7f227006ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270101240 0x7f227006ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 -- 172.21.15.7:0/2789788587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701a0190 con 0x7f2270102440 2024-03-19T00:35:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f2274aa2700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270101240 0x7f227006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.497+0000 7f226ffff700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 0x7f227019d280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9c0 tx=0x7f225c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.498+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c00cbd0 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.498+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701a0480 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.498+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f225c00cd70 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.498+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c0056c0 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.498+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701a09c0 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.499+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270068fd0 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.500+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f225c004750 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.501+0000 7f226dffb700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22600a2890 0x7f22600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.501+0000 7f2274aa2700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22600a2890 0x7f22600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.501+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f225c0ce930 con 0x7f2270102440 2024-03-19T00:35:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.501+0000 7f2274aa2700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22600a2890 0x7f22600a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f2264005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.505+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f225c053740 con 0x7f2270102440 2024-03-19T00:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.711+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f227002dbb0 con 0x7f2270102440 2024-03-19T00:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.712+0000 7f226dffb700 1 -- 172.21.15.7:0/2789788587 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f225c084e20 con 0x7f2270102440 2024-03-19T00:35:08.713 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22600a2890 msgr2=0x7f22600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22600a2890 0x7f22600a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f2264005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 msgr2=0x7f227019d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 0x7f227019d280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9c0 tx=0x7f225c00ecd0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 shutdown_connections 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22600a2890 0x7f22600a4d50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2270102de0 0x7f227019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2270101240 0x7f227006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 --2- 172.21.15.7:0/2789788587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2270102440 0x7f227019d280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 >> 172.21.15.7:0/2789788587 conn(0x7f22700fc7e0 msgr2=0x7f2270105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 shutdown_connections 2024-03-19T00:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:08.714+0000 7f2276d06700 1 -- 172.21.15.7:0/2789788587 wait complete. 2024-03-19T00:35:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-mon[66572]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:35:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-mon[66572]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T00:35:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:09.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-mon[66572]: from='osd.11 ' entity='osd.11' 2024-03-19T00:35:09.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2789788587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T00:35:09.016 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:35:08.613+0000 7f2c319ea700 -1 osd.11 0 waiting for initial osdmap 2024-03-19T00:35:09.016 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:35:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:35:08.626+0000 7f2c297d5700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T00:35:09.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:08 smithi007 ceph-mon[62854]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:35:09.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:08 smithi007 ceph-mon[62854]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T00:35:09.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:09.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:08 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:09.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:08 smithi007 ceph-mon[62854]: from='osd.11 ' entity='osd.11' 2024-03-19T00:35:09.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:08 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2789788587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T00:35:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:08 smithi044 ceph-mon[66366]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-19T00:35:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:08 smithi044 ceph-mon[66366]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T00:35:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:08 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:08 smithi044 ceph-mon[66366]: from='osd.11 ' entity='osd.11' 2024-03-19T00:35:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:08 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2789788587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T00:35:09.516 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1710808482,"num_in_osds":12,"osd_in_since":1710808489,"num_remapped_pgs":0} 2024-03-19T00:35:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: purged_snaps scrub starts 2024-03-19T00:35:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: purged_snaps scrub ok 2024-03-19T00:35:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] boot 2024-03-19T00:35:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T00:35:10.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:09 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:10.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: purged_snaps scrub starts 2024-03-19T00:35:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: purged_snaps scrub ok 2024-03-19T00:35:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] boot 2024-03-19T00:35:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T00:35:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:09 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: purged_snaps scrub starts 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: purged_snaps scrub ok 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: osd.11 [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] boot 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T00:35:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:09 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T00:35:10.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-19T00:35:10.761 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:35:11.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:11.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: Detected new or changed devices on smithi204 2024-03-19T00:35:11.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi204 to 4272M 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:11.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:11 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: Detected new or changed devices on smithi204 2024-03-19T00:35:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi204 to 4272M 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:11.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:11 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: Detected new or changed devices on smithi204 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi204 to 4272M 2024-03-19T00:35:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:11.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:11 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:12.235 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:12 smithi007 ceph-mon[62854]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T00:35:12.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:12 smithi204 ceph-mon[66572]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T00:35:12.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 -- 172.21.15.7:0/387007963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58102e30 msgr2=0x7f4f58105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:12.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 --2- 172.21.15.7:0/387007963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58102e30 0x7f4f58105220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009990 tx=0x7f4f48009ca0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 -- 172.21.15.7:0/387007963 shutdown_connections 2024-03-19T00:35:12.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 --2- 172.21.15.7:0/387007963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58108090 0x7f4f5810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 --2- 172.21.15.7:0/387007963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58102e30 0x7f4f58105220 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 --2- 172.21.15.7:0/387007963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58105760 0x7f4f58107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 -- 172.21.15.7:0/387007963 >> 172.21.15.7:0/387007963 conn(0x7f4f580fc800 msgr2=0x7f4f580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 -- 172.21.15.7:0/387007963 shutdown_connections 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.594+0000 7f4f5e397700 1 -- 172.21.15.7:0/387007963 wait complete. 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 Processor -- start 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 -- start start 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102e30 0x7f4f58198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 0x7f4f58198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58108090 0x7f4f5819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5810de60 con 0x7f4f58105760 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5810dce0 con 0x7f4f58108090 2024-03-19T00:35:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f5e397700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5810dfe0 con 0x7f4f58102e30 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 0x7f4f58198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 0x7f4f58198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59752/0 (socket says 172.21.15.7:59752) 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.595+0000 7f4f577fe700 1 -- 172.21.15.7:0/39954540 learned_addr learned my addr 172.21.15.7:0/39954540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f57fff700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102e30 0x7f4f58198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f5c934700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58108090 0x7f4f5819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f577fe700 1 -- 172.21.15.7:0/39954540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102e30 msgr2=0x7f4f58198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f577fe700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102e30 0x7f4f58198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f577fe700 1 -- 172.21.15.7:0/39954540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58108090 msgr2=0x7f4f5819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f577fe700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58108090 0x7f4f5819fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f577fe700 1 -- 172.21.15.7:0/39954540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f581a0170 con 0x7f4f58105760 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f577fe700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 0x7f4f58198a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c9a0 tx=0x7f4f4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c007a60 con 0x7f4f58105760 2024-03-19T00:35:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f581a0460 con 0x7f4f58105760 2024-03-19T00:35:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f4c004ce0 con 0x7f4f58105760 2024-03-19T00:35:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c005560 con 0x7f4f58105760 2024-03-19T00:35:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.596+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f581a09d0 con 0x7f4f58105760 2024-03-19T00:35:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.597+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f58068fd0 con 0x7f4f58105760 2024-03-19T00:35:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.599+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f4f4c007c00 con 0x7f4f58105760 2024-03-19T00:35:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.599+0000 7f4f557fa700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2890 0x7f4f400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.600+0000 7f4f57fff700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2890 0x7f4f400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.600+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4f4c0ce6b0 con 0x7f4f58105760 2024-03-19T00:35:12.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.600+0000 7f4f57fff700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2890 0x7f4f400a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b5c0 tx=0x7f4f480038e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:12.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.603+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4f4c00de20 con 0x7f4f58105760 2024-03-19T00:35:12.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:12 smithi044 ceph-mon[66366]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T00:35:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.810+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4f5802dbe0 con 0x7f4f58105760 2024-03-19T00:35:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.811+0000 7f4f557fa700 1 -- 172.21.15.7:0/39954540 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f4f4c005700 con 0x7f4f58105760 2024-03-19T00:35:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2890 msgr2=0x7f4f400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2890 0x7f4f400a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b5c0 tx=0x7f4f480038e0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 msgr2=0x7f4f58198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 0x7f4f58198a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c9a0 tx=0x7f4f4c00ccb0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 shutdown_connections 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2890 0x7f4f400a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58108090 0x7f4f5819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102e30 0x7f4f58198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 --2- 172.21.15.7:0/39954540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58105760 0x7f4f58198a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 >> 172.21.15.7:0/39954540 conn(0x7f4f580fc800 msgr2=0x7f4f58103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 shutdown_connections 2024-03-19T00:35:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:12.813+0000 7f4f5e397700 1 -- 172.21.15.7:0/39954540 wait complete. 2024-03-19T00:35:13.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:13 smithi007 ceph-mon[62854]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:13.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:13 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/39954540' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T00:35:13.439 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1710808509,"num_in_osds":12,"osd_in_since":1710808489,"num_remapped_pgs":0} 2024-03-19T00:35:13.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T00:35:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:13 smithi204 ceph-mon[66572]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:13 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/39954540' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T00:35:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:13 smithi044 ceph-mon[66366]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:13 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/39954540' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T00:35:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:35:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1453427984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102d20 msgr2=0x7fe9ac10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1453427984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102d20 0x7fe9ac10f160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00b0d0 tx=0x7fe99c00b3e0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1453427984 shutdown_connections 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1453427984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102d20 0x7fe9ac10f160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1453427984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 0x7fe9ac108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1453427984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102360 0x7fe9ac1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1453427984 >> 172.21.15.7:0/1453427984 conn(0x7fe9ac079440 msgr2=0x7fe9ac079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1453427984 shutdown_connections 2024-03-19T00:35:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1453427984 wait complete. 2024-03-19T00:35:15.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:15 smithi007 ceph-mon[62854]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.402+0000 7fe9b12d0700 1 Processor -- start 2024-03-19T00:35:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 -- start start 2024-03-19T00:35:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 0x7fe9ac19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102d20 0x7fe9ac19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 0x7fe9ac198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac112470 con 0x7fe9ac102360 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac1122f0 con 0x7fe9ac108340 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9b12d0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac1125f0 con 0x7fe9ac102d20 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 0x7fe9ac19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 0x7fe9ac19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34852/0 (socket says 172.21.15.7:34852) 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 -- 172.21.15.7:0/1248857049 learned_addr learned my addr 172.21.15.7:0/1248857049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aa7fc700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102d20 0x7fe9ac19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9ab7fe700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 0x7fe9ac198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 -- 172.21.15.7:0/1248857049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102d20 msgr2=0x7fe9ac19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102d20 0x7fe9ac19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 -- 172.21.15.7:0/1248857049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 msgr2=0x7fe9ac198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 0x7fe9ac198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 -- 172.21.15.7:0/1248857049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac198f70 con 0x7fe9ac102360 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aa7fc700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102d20 0x7fe9ac19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9ab7fe700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 0x7fe9ac198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.403+0000 7fe9aaffd700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 0x7fe9ac19e610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e9a0 tx=0x7fe9a400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.404+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400cbd0 con 0x7fe9ac102360 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.404+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9a400cd70 con 0x7fe9ac102360 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.404+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac199260 con 0x7fe9ac102360 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.404+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4018750 con 0x7fe9ac102360 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.404+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac1a5470 con 0x7fe9ac102360 2024-03-19T00:35:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.405+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac1039f0 con 0x7fe9ac102360 2024-03-19T00:35:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.406+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fe9a4010430 con 0x7fe9ac102360 2024-03-19T00:35:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.407+0000 7fe993fff700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9940a2890 0x7fe9940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.407+0000 7fe9aa7fc700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9940a2890 0x7fe9940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.407+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe9a40cead0 con 0x7fe9ac102360 2024-03-19T00:35:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.407+0000 7fe9aa7fc700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9940a2890 0x7fe9940a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac079af0 tx=0x7fe9a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.411+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe9a409bf70 con 0x7fe9ac102360 2024-03-19T00:35:15.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:15 smithi204 ceph-mon[66572]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:15.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:15 smithi044 ceph-mon[66366]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.616+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9ac02dbb0 con 0x7fe9ac102360 2024-03-19T00:35:15.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.617+0000 7fe993fff700 1 -- 172.21.15.7:0/1248857049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18299 (secure 0 0 0) 0x7fe9a409b870 con 0x7fe9ac102360 2024-03-19T00:35:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:15.618 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:35:11.135052+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:35:15.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9940a2890 msgr2=0x7fe9940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:15.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9940a2890 0x7fe9940a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac079af0 tx=0x7fe9a0009260 comp rx=0 tx=0).stop 2024-03-19T00:35:15.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 msgr2=0x7fe9ac19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:15.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 0x7fe9ac19e610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e9a0 tx=0x7fe9a400ed60 comp rx=0 tx=0).stop 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 shutdown_connections 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9940a2890 0x7fe9940a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9ac108340 0x7fe9ac198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9ac102360 0x7fe9ac19e610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 --2- 172.21.15.7:0/1248857049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9ac102d20 0x7fe9ac19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 >> 172.21.15.7:0/1248857049 conn(0x7fe9ac079440 msgr2=0x7fe9ac101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 shutdown_connections 2024-03-19T00:35:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:15.619+0000 7fe9b12d0700 1 -- 172.21.15.7:0/1248857049 wait complete. 2024-03-19T00:35:16.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1248857049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:16.352 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T00:35:16.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-19T00:35:16.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1248857049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:35:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1248857049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:17.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:17 smithi007 ceph-mon[62854]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:17.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:17 smithi204 ceph-mon[66572]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:17 smithi044 ceph-mon[66366]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 -- 172.21.15.7:0/53859766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 msgr2=0x7f1d18108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 --2- 172.21.15.7:0/53859766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18108720 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0099c0 tx=0x7f1d0c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 -- 172.21.15.7:0/53859766 shutdown_connections 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 --2- 172.21.15.7:0/53859766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d18102d20 0x7f1d1810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 --2- 172.21.15.7:0/53859766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18108720 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 --2- 172.21.15.7:0/53859766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d18102360 0x7f1d181027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 -- 172.21.15.7:0/53859766 >> 172.21.15.7:0/53859766 conn(0x7f1d18079440 msgr2=0x7f1d18079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 -- 172.21.15.7:0/53859766 shutdown_connections 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.353+0000 7f1d1f017700 1 -- 172.21.15.7:0/53859766 wait complete. 2024-03-19T00:35:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.354+0000 7f1d1f017700 1 Processor -- start 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.354+0000 7f1d1f017700 1 -- start start 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.354+0000 7f1d1f017700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d18102360 0x7f1d1819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.354+0000 7f1d1f017700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d18102d20 0x7f1d1819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.354+0000 7f1d1f017700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1f017700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18112470 con 0x7f1d18108340 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1f017700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181122f0 con 0x7f1d18102360 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1f017700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181125f0 con 0x7f1d18102d20 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34874/0 (socket says 172.21.15.7:34874) 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 -- 172.21.15.7:0/2332540420 learned_addr learned my addr 172.21.15.7:0/2332540420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d17fff700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d18102d20 0x7f1d1819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1cdb3700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d18102360 0x7f1d1819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 -- 172.21.15.7:0/2332540420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d18102d20 msgr2=0x7f1d1819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d18102d20 0x7f1d1819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 -- 172.21.15.7:0/2332540420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d18102360 msgr2=0x7f1d1819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d18102360 0x7f1d1819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 -- 172.21.15.7:0/2332540420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d18198f70 con 0x7f1d18108340 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d1d5b4700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18198690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d10007d70 tx=0x7f1d1000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.355+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10016020 con 0x7f1d18108340 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.356+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d1000ad40 con 0x7f1d18108340 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.356+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d18199260 con 0x7f1d18108340 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.356+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d100115e0 con 0x7f1d18108340 2024-03-19T00:35:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.356+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181a5540 con 0x7f1d18108340 2024-03-19T00:35:18.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.357+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d181039f0 con 0x7f1d18108340 2024-03-19T00:35:18.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.358+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f1d10019020 con 0x7f1d18108340 2024-03-19T00:35:18.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.358+0000 7f1d15ffb700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1d080a2910 0x7f1d080a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:18.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.359+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1d100ceb10 con 0x7f1d18108340 2024-03-19T00:35:18.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.359+0000 7f1d1cdb3700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1d080a2910 0x7f1d080a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:18.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.359+0000 7f1d1cdb3700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1d080a2910 0x7f1d080a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009fd0 tx=0x7f1d0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:18.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.362+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1d10020020 con 0x7f1d18108340 2024-03-19T00:35:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.563+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d18199bf0 con 0x7f1d18108340 2024-03-19T00:35:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.563+0000 7f1d15ffb700 1 -- 172.21.15.7:0/2332540420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1d18199bf0 con 0x7f1d18108340 2024-03-19T00:35:18.564 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:35:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1d080a2910 msgr2=0x7f1d080a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1d080a2910 0x7f1d080a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009fd0 tx=0x7f1d0c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 msgr2=0x7f1d18198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18198690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d10007d70 tx=0x7f1d1000a340 comp rx=0 tx=0).stop 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 shutdown_connections 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1d080a2910 0x7f1d080a4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d18108340 0x7f1d18198690 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d18102360 0x7f1d1819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 --2- 172.21.15.7:0/2332540420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d18102d20 0x7f1d1819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 >> 172.21.15.7:0/2332540420 conn(0x7f1d18079440 msgr2=0x7f1d18101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 shutdown_connections 2024-03-19T00:35:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:18.565+0000 7f1d1f017700 1 -- 172.21.15.7:0/2332540420 wait complete. 2024-03-19T00:35:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:19 smithi007 ceph-mon[62854]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:19.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:19 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2332540420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.b on smithi044 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.e on smithi044 2024-03-19T00:35:19.342 INFO:tasks.cephadm:Adding mds.h on smithi044 2024-03-19T00:35:19.343 INFO:tasks.cephadm:Adding mds.k on smithi044 2024-03-19T00:35:19.343 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-03-19T00:35:19.343 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-03-19T00:35:19.343 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-03-19T00:35:19.343 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-03-19T00:35:19.343 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-03-19T00:35:19.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:19 smithi204 ceph-mon[66572]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:19.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:19 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2332540420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:19.595 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:35:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:19 smithi044 ceph-mon[66366]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:19 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2332540420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 -- 172.21.15.204:0/181224970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568102e10 msgr2=0x7f85681096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 --2- 172.21.15.204:0/181224970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568102e10 0x7f85681096c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f856000b0a0 tx=0x7f856000b3b0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 -- 172.21.15.204:0/181224970 shutdown_connections 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 --2- 172.21.15.204:0/181224970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568102e10 0x7f85681096c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 --2- 172.21.15.204:0/181224970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568101270 0x7f8568101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 --2- 172.21.15.204:0/181224970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102470 0x7f85681028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.095+0000 7f856dacb700 1 -- 172.21.15.204:0/181224970 >> 172.21.15.204:0/181224970 conn(0x7f85680fc830 msgr2=0x7f85680fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:21.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.096+0000 7f856dacb700 1 -- 172.21.15.204:0/181224970 shutdown_connections 2024-03-19T00:35:21.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.096+0000 7f856dacb700 1 -- 172.21.15.204:0/181224970 wait complete. 2024-03-19T00:35:21.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.096+0000 7f856dacb700 1 Processor -- start 2024-03-19T00:35:21.098 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.096+0000 7f856dacb700 1 -- start start 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.096+0000 7f856dacb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 0x7f85681973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568102470 0x7f85681978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102e10 0x7f856819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f856810cd90 con 0x7f8568102e10 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f856810cc10 con 0x7f8568102470 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f856810cf10 con 0x7f8568101270 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 0x7f85681973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 0x7f85681973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:58798/0 (socket says 172.21.15.204:58798) 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 -- 172.21.15.204:0/2342968902 learned_addr learned my addr 172.21.15.204:0/2342968902 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8567fff700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102e10 0x7f856819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 -- 172.21.15.204:0/2342968902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568102470 msgr2=0x7f85681978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:21.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8566ffd700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568102470 0x7f85681978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568102470 0x7f85681978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 -- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102e10 msgr2=0x7f856819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102e10 0x7f856819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 -- 172.21.15.204:0/2342968902 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856819f000 con 0x7f8568101270 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8567fff700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102e10 0x7f856819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8566ffd700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568102470 0x7f85681978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f85677fe700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 0x7f85681973a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f855800d9b0 tx=0x7f855800dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558017070 con 0x7f8568101270 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856819f2f0 con 0x7f8568101270 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8558010460 con 0x7f8568101270 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800f620 con 0x7f8568101270 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.097+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856819f8b0 con 0x7f8568101270 2024-03-19T00:35:21.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.098+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8568068fd0 con 0x7f8568101270 2024-03-19T00:35:21.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.099+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f8558010610 con 0x7f8568101270 2024-03-19T00:35:21.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.100+0000 7f8564ff9700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f85500a2890 0x7f85500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:21.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.100+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f85580ce1b0 con 0x7f8568101270 2024-03-19T00:35:21.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.100+0000 7f8566ffd700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f85500a2890 0x7f85500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:21.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.101+0000 7f8566ffd700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f85500a2890 0x7f85500a4d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f855c007900 tx=0x7f855c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:21.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.104+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f855809b6f0 con 0x7f8568101270 2024-03-19T00:35:21.313 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.312+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7f8568062850 con 0x7f85500a2890 2024-03-19T00:35:21.321 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.320+0000 7f8564ff9700 1 -- 172.21.15.204:0/2342968902 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8568062850 con 0x7f85500a2890 2024-03-19T00:35:21.321 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-03-19T00:35:21.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f85500a2890 msgr2=0x7f85500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f85500a2890 0x7f85500a4d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f855c007900 tx=0x7f855c008040 comp rx=0 tx=0).stop 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 msgr2=0x7f85681973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 0x7f85681973a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f855800d9b0 tx=0x7f855800dd70 comp rx=0 tx=0).stop 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 shutdown_connections 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f85500a2890 0x7f85500a4d50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8568102e10 0x7f856819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568101270 0x7f85681973a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 --2- 172.21.15.204:0/2342968902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8568102470 0x7f85681978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 >> 172.21.15.204:0/2342968902 conn(0x7f85680fc830 msgr2=0x7f8568105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 shutdown_connections 2024-03-19T00:35:21.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-19T00:35:21.322+0000 7f856dacb700 1 -- 172.21.15.204:0/2342968902 wait complete. 2024-03-19T00:35:21.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:21 smithi007 ceph-mon[62854]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:21.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:21 smithi204 ceph-mon[66572]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:21.605 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:21 smithi044 ceph-mon[66366]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:21.986 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.a.service 2024-03-19T00:35:21.989 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.d.service 2024-03-19T00:35:22.031 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.g.service 2024-03-19T00:35:22.074 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.j.service 2024-03-19T00:35:22.116 DEBUG:teuthology.orchestra.run.smithi044:mds.b> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.b.service 2024-03-19T00:35:22.119 DEBUG:teuthology.orchestra.run.smithi044:mds.e> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.e.service 2024-03-19T00:35:22.162 DEBUG:teuthology.orchestra.run.smithi044:mds.h> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.h.service 2024-03-19T00:35:22.205 DEBUG:teuthology.orchestra.run.smithi044:mds.k> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.k.service 2024-03-19T00:35:22.248 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.c.service 2024-03-19T00:35:22.250 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.f.service 2024-03-19T00:35:22.293 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.i.service 2024-03-19T00:35:22.336 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.l.service 2024-03-19T00:35:22.379 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-19T00:35:22.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:35:22.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 -- 172.21.15.7:0/4258003255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd530102370 msgr2=0x7fd530108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 --2- 172.21.15.7:0/4258003255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd530102370 0x7fd530108c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd520009ca0 tx=0x7fd52002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 -- 172.21.15.7:0/4258003255 shutdown_connections 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 --2- 172.21.15.7:0/4258003255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd530102370 0x7fd530108c00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 --2- 172.21.15.7:0/4258003255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5301019b0 0x7fd530101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 --2- 172.21.15.7:0/4258003255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5301007c0 0x7fd530100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 -- 172.21.15.7:0/4258003255 >> 172.21.15.7:0/4258003255 conn(0x7fd5300fbd30 msgr2=0x7fd5300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 -- 172.21.15.7:0/4258003255 shutdown_connections 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.509+0000 7fd534bf4640 1 -- 172.21.15.7:0/4258003255 wait complete. 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 Processor -- start 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 -- start start 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5301007c0 0x7fd530197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 0x7fd530197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd530102370 0x7fd53019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010d3a0 con 0x7fd5301019b0 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010d220 con 0x7fd5301007c0 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd534bf4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010d520 con 0x7fd530102370 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.510+0000 7fd52dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 0x7fd530197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 0x7fd530197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34894/0 (socket says 172.21.15.7:34894) 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 -- 172.21.15.7:0/2962140028 learned_addr learned my addr 172.21.15.7:0/2962140028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52ed9d640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd530102370 0x7fd53019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52e59c640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5301007c0 0x7fd530197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 -- 172.21.15.7:0/2962140028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd530102370 msgr2=0x7fd53019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd530102370 0x7fd53019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 -- 172.21.15.7:0/2962140028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5301007c0 msgr2=0x7fd530197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5301007c0 0x7fd530197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 -- 172.21.15.7:0/2962140028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53019f640 con 0x7fd5301019b0 2024-03-19T00:35:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd52dd9b640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 0x7fd530197f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e9f0 tx=0x7fd51800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:22.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51800cd80 con 0x7fd5301019b0 2024-03-19T00:35:22.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53019f930 con 0x7fd5301019b0 2024-03-19T00:35:22.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5180054d0 con 0x7fd5301019b0 2024-03-19T00:35:22.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518010490 con 0x7fd5301019b0 2024-03-19T00:35:22.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.511+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53019fe30 con 0x7fd5301019b0 2024-03-19T00:35:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.512+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd530107ae0 con 0x7fd5301019b0 2024-03-19T00:35:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.513+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fd518005020 con 0x7fd5301019b0 2024-03-19T00:35:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.514+0000 7fd5277fe640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd51c0a2800 0x7fd51c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.514+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd5180ce920 con 0x7fd5301019b0 2024-03-19T00:35:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.514+0000 7fd52e59c640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd51c0a2800 0x7fd51c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.514+0000 7fd52e59c640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd51c0a2800 0x7fd51c0a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5280095f0 tx=0x7fd528005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:22.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.518+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd51809be60 con 0x7fd5301019b0 2024-03-19T00:35:22.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:35:22.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-19T00:35:22.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:22.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:22 smithi204 ceph-mon[66572]: Deploying daemon mds.l on smithi204 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:22.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:22.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:22 smithi044 ceph-mon[66366]: Deploying daemon mds.l on smithi204 2024-03-19T00:35:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.704+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd53002d440 con 0x7fd5301019b0 2024-03-19T00:35:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.705+0000 7fd5277fe640 1 -- 172.21.15.7:0/2962140028 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18299 (secure 0 0 0) 0x7fd51809b760 con 0x7fd5301019b0 2024-03-19T00:35:22.706 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:22.706 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:35:11.135052+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd51c0a2800 msgr2=0x7fd51c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd51c0a2800 0x7fd51c0a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5280095f0 tx=0x7fd528005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 msgr2=0x7fd530197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 0x7fd530197f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e9f0 tx=0x7fd51800edf0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 shutdown_connections 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd51c0a2800 0x7fd51c0a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd530102370 0x7fd53019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5301019b0 0x7fd530197f60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 --2- 172.21.15.7:0/2962140028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5301007c0 0x7fd530197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 >> 172.21.15.7:0/2962140028 conn(0x7fd5300fbd30 msgr2=0x7fd530104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 shutdown_connections 2024-03-19T00:35:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.707+0000 7fd534bf4640 1 -- 172.21.15.7:0/2962140028 wait complete. 2024-03-19T00:35:22.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:22.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:22.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:22 smithi007 ceph-mon[62854]: Deploying daemon mds.l on smithi204 2024-03-19T00:35:22.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T00:35:22.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 -- 172.21.15.7:0/987871827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 msgr2=0x7f60e4100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 --2- 172.21.15.7:0/987871827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4100b60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009900 tx=0x7f60e002bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 -- 172.21.15.7:0/987871827 shutdown_connections 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 --2- 172.21.15.7:0/987871827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e41022f0 0x7f60e4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 --2- 172.21.15.7:0/987871827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4101930 0x7f60e4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 --2- 172.21.15.7:0/987871827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4100b60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 -- 172.21.15.7:0/987871827 >> 172.21.15.7:0/987871827 conn(0x7f60e40fbcb0 msgr2=0x7f60e40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 -- 172.21.15.7:0/987871827 shutdown_connections 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 -- 172.21.15.7:0/987871827 wait complete. 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 Processor -- start 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.825+0000 7f60eb53b640 1 -- start start 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e4101930 0x7f60e4196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41022f0 0x7f60e419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60e410c260 con 0x7f60e4100740 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60e410c0e0 con 0x7f60e4101930 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60e410c3e0 con 0x7f60e41022f0 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34910/0 (socket says 172.21.15.7:34910) 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 -- 172.21.15.7:0/1704353177 learned_addr learned my addr 172.21.15.7:0/1704353177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e9ad8640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41022f0 0x7f60e419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e8ad6640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e4101930 0x7f60e4196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 -- 172.21.15.7:0/1704353177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41022f0 msgr2=0x7f60e419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41022f0 0x7f60e419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 -- 172.21.15.7:0/1704353177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e4101930 msgr2=0x7f60e4196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e4101930 0x7f60e4196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 -- 172.21.15.7:0/1704353177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e419e5b0 con 0x7f60e4100740 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60e92d7640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4196990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000c00 tx=0x7f60e0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0043020 con 0x7f60e4100740 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e419e840 con 0x7f60e4100740 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60e0004350 con 0x7f60e4100740 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0003b80 con 0x7f60e4100740 2024-03-19T00:35:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.826+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e419ede0 con 0x7f60e4100740 2024-03-19T00:35:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.827+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e4107b30 con 0x7f60e4100740 2024-03-19T00:35:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.828+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f60e0046020 con 0x7f60e4100740 2024-03-19T00:35:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.829+0000 7f60da7fc640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f60d00a2800 0x7f60d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.829+0000 7f60e8ad6640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f60d00a2800 0x7f60d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.829+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f60e00f0760 con 0x7f60e4100740 2024-03-19T00:35:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.829+0000 7f60e8ad6640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f60d00a2800 0x7f60d00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60e4197ea0 tx=0x7f60d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:22.833+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f60e004d020 con 0x7f60e4100740 2024-03-19T00:35:23.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.017+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60e4062420 con 0x7f60e4100740 2024-03-19T00:35:23.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.017+0000 7f60da7fc640 1 -- 172.21.15.7:0/1704353177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f60e00bdc00 con 0x7f60e4100740 2024-03-19T00:35:23.018 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f60d00a2800 msgr2=0x7f60d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f60d00a2800 0x7f60d00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60e4197ea0 tx=0x7f60d4005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 msgr2=0x7f60e4196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4196990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000c00 tx=0x7f60e0005dc0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 shutdown_connections 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f60d00a2800 0x7f60d00a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41022f0 0x7f60e419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60e4101930 0x7f60e4196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 --2- 172.21.15.7:0/1704353177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60e4100740 0x7f60e4196990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 >> 172.21.15.7:0/1704353177 conn(0x7f60e40fbcb0 msgr2=0x7f60e4104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 shutdown_connections 2024-03-19T00:35:23.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.020+0000 7f60eb53b640 1 -- 172.21.15.7:0/1704353177 wait complete. 2024-03-19T00:35:23.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:23.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 -- 172.21.15.7:0/1580650255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 msgr2=0x7f5d28100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/1580650255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28100b70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009900 tx=0x7f5d1802bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 -- 172.21.15.7:0/1580650255 shutdown_connections 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/1580650255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d28102300 0x7f5d28108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/1580650255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d28101940 0x7f5d28101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/1580650255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28100b70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 -- 172.21.15.7:0/1580650255 >> 172.21.15.7:0/1580650255 conn(0x7f5d280fbce0 msgr2=0x7f5d280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 -- 172.21.15.7:0/1580650255 shutdown_connections 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.133+0000 7f5d2e33b640 1 -- 172.21.15.7:0/1580650255 wait complete. 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 Processor -- start 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 -- start start 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:23.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d28101940 0x7f5d2819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d28102300 0x7f5d281a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810c270 con 0x7f5d28100750 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810c0f0 con 0x7f5d28101940 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d2e33b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810c3f0 con 0x7f5d28102300 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34920/0 (socket says 172.21.15.7:34920) 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.134+0000 7f5d27fff640 1 -- 172.21.15.7:0/2885183892 learned_addr learned my addr 172.21.15.7:0/2885183892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d2c8d8640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d28102300 0x7f5d281a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d277fe640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d28101940 0x7f5d2819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d27fff640 1 -- 172.21.15.7:0/2885183892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d28102300 msgr2=0x7f5d281a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d27fff640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d28102300 0x7f5d281a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d27fff640 1 -- 172.21.15.7:0/2885183892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d28101940 msgr2=0x7f5d2819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d27fff640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d28101940 0x7f5d2819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d27fff640 1 -- 172.21.15.7:0/2885183892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d281a1910 con 0x7f5d28100750 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d27fff640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28199d00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000c00 tx=0x7f5d18005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18043020 con 0x7f5d28100750 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d18004350 con 0x7f5d28100750 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d281a1ba0 con 0x7f5d28100750 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18003b80 con 0x7f5d28100750 2024-03-19T00:35:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.135+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d281a20f0 con 0x7f5d28100750 2024-03-19T00:35:23.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.136+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d28068af0 con 0x7f5d28100750 2024-03-19T00:35:23.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.137+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f5d18046020 con 0x7f5d28100750 2024-03-19T00:35:23.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.138+0000 7f5d257fa640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5d140a2800 0x7f5d140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:23.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.138+0000 7f5d277fe640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5d140a2800 0x7f5d140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:23.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.138+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5d180f08c0 con 0x7f5d28100750 2024-03-19T00:35:23.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.138+0000 7f5d277fe640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5d140a2800 0x7f5d140a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009b60 tx=0x7f5d10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:23.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.142+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5d1804d020 con 0x7f5d28100750 2024-03-19T00:35:23.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.364+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d281a29d0 con 0x7f5d28100750 2024-03-19T00:35:23.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.364+0000 7f5d257fa640 1 -- 172.21.15.7:0/2885183892 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f5d180bdd60 con 0x7f5d28100750 2024-03-19T00:35:23.365 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:23.366 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-19T00:35:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 1 2024-03-19T00:35:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5d140a2800 msgr2=0x7f5d140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5d140a2800 0x7f5d140a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009b60 tx=0x7f5d10005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 msgr2=0x7f5d28199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28199d00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000c00 tx=0x7f5d18005dc0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 shutdown_connections 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5d140a2800 0x7f5d140a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d28102300 0x7f5d281a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d28101940 0x7f5d2819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 --2- 172.21.15.7:0/2885183892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d28100750 0x7f5d28199d00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 >> 172.21.15.7:0/2885183892 conn(0x7f5d280fbce0 msgr2=0x7f5d28069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.366+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 shutdown_connections 2024-03-19T00:35:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:23.367+0000 7f5d2e33b640 1 -- 172.21.15.7:0/2885183892 wait complete. 2024-03-19T00:35:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:23 smithi204 ceph-mon[66572]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2962140028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1704353177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:23 smithi044 ceph-mon[66366]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2962140028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1704353177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:23 smithi007 ceph-mon[62854]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2962140028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1704353177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:24.200 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:23 smithi204 systemd[1]: Starting Ceph mds.l for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:24.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2885183892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:24.515 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 podman[96524]: 2024-03-19 00:35:24.199552136 +0000 UTC m=+0.113110041 container create 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:35:24.515 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 podman[96524]: 2024-03-19 00:35:24.104832887 +0000 UTC m=+0.018390801 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:24.515 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 podman[96524]: 2024-03-19 00:35:24.419058386 +0000 UTC m=+0.332616290 container init 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:35:24.515 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 podman[96524]: 2024-03-19 00:35:24.421819791 +0000 UTC m=+0.335377696 container start 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-19T00:35:24.515 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 ceph-mds[96543]: -- 172.21.15.204:0/155450614 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3850e1a40 con 0x55b3850b4800 2024-03-19T00:35:24.516 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l[96539]: starting mds.l at 2024-03-19T00:35:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2885183892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2885183892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:25.015 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 bash[96524]: 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 2024-03-19T00:35:25.015 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:35:24 smithi204 systemd[1]: Started Ceph mds.l for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:25.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 -- 172.21.15.7:0/3105018952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241019b0 msgr2=0x7fb924101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 --2- 172.21.15.7:0/3105018952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241019b0 0x7fb924101e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009900 tx=0x7fb90c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 -- 172.21.15.7:0/3105018952 shutdown_connections 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 --2- 172.21.15.7:0/3105018952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 0x7fb924108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 --2- 172.21.15.7:0/3105018952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241019b0 0x7fb924101e30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 --2- 172.21.15.7:0/3105018952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9241007c0 0x7fb924100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 -- 172.21.15.7:0/3105018952 >> 172.21.15.7:0/3105018952 conn(0x7fb9240fbd50 msgr2=0x7fb9240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 -- 172.21.15.7:0/3105018952 shutdown_connections 2024-03-19T00:35:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.494+0000 7fb928cf8640 1 -- 172.21.15.7:0/3105018952 wait complete. 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 Processor -- start 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 -- start start 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 0x7fb92406aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9241019b0 0x7fb92419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 0x7fb92419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c260 con 0x7fb9241007c0 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c0e0 con 0x7fb924102370 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb928cf8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c3e0 con 0x7fb9241019b0 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 0x7fb92406aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 0x7fb92406aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52586/0 (socket says 172.21.15.7:52586) 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 -- 172.21.15.7:0/2820858452 learned_addr learned my addr 172.21.15.7:0/2820858452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb91bfff640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9241019b0 0x7fb92419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb922d9d640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 0x7fb92419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 -- 172.21.15.7:0/2820858452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9241019b0 msgr2=0x7fb92419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9241019b0 0x7fb92419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 -- 172.21.15.7:0/2820858452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 msgr2=0x7fb92419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 0x7fb92419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 -- 172.21.15.7:0/2820858452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419f6a0 con 0x7fb9241007c0 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb922d9d640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 0x7fb92419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.495+0000 7fb92259c640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 0x7fb92406aa90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb91400d770 tx=0x7fb91400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.496+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914017070 con 0x7fb9241007c0 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.496+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb914009c30 con 0x7fb9241007c0 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.496+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92419f990 con 0x7fb9241007c0 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.496+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914003d00 con 0x7fb9241007c0 2024-03-19T00:35:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.496+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92419feb0 con 0x7fb9241007c0 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6832/1644274763,v1:172.21.15.204:6833/1644274763] up:boot 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: fsmap 1 up:standby 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.500 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:25.501 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:25.501 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:25.501 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:25 smithi044 ceph-mon[66366]: Deploying daemon mds.k on smithi044 2024-03-19T00:35:25.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.497+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb924107b60 con 0x7fb9241007c0 2024-03-19T00:35:25.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.498+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fb914003ea0 con 0x7fb9241007c0 2024-03-19T00:35:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.499+0000 7fb91b7fe640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb9100a2800 0x7fb9100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.499+0000 7fb91bfff640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb9100a2800 0x7fb9100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.499+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb9140ce330 con 0x7fb9241007c0 2024-03-19T00:35:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.499+0000 7fb91bfff640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb9100a2800 0x7fb9100a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009230 tx=0x7fb90c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.503+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb91409b870 con 0x7fb9241007c0 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6832/1644274763,v1:172.21.15.204:6833/1644274763] up:boot 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: fsmap 1 up:standby 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:25 smithi007 ceph-mon[62854]: Deploying daemon mds.k on smithi044 2024-03-19T00:35:25.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.727+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb92402d1f0 con 0x7fb9241007c0 2024-03-19T00:35:25.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.728+0000 7fb91b7fe640 1 -- 172.21.15.7:0/2820858452 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb914009de0 con 0x7fb9241007c0 2024-03-19T00:35:25.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:25.729 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T00:35:25.729 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-03-19T00:35:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb9100a2800 msgr2=0x7fb9100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb9100a2800 0x7fb9100a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009230 tx=0x7fb90c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 msgr2=0x7fb92406aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 0x7fb92406aa90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb91400d770 tx=0x7fb91400dc30 comp rx=0 tx=0).stop 2024-03-19T00:35:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 shutdown_connections 2024-03-19T00:35:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb9100a2800 0x7fb9100a4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb924102370 0x7fb92419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9241019b0 0x7fb92419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 --2- 172.21.15.7:0/2820858452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9241007c0 0x7fb92406aa90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:25.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 >> 172.21.15.7:0/2820858452 conn(0x7fb9240fbd50 msgr2=0x7fb924104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:25.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 shutdown_connections 2024-03-19T00:35:25.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:25.730+0000 7fb928cf8640 1 -- 172.21.15.7:0/2820858452 wait complete. 2024-03-19T00:35:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6832/1644274763,v1:172.21.15.204:6833/1644274763] up:boot 2024-03-19T00:35:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: fsmap 1 up:standby 2024-03-19T00:35:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T00:35:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:25.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:25.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:25.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:25.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:25 smithi204 ceph-mon[66572]: Deploying daemon mds.k on smithi044 2024-03-19T00:35:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2820858452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2820858452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:26.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2820858452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:26.998 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:26 smithi044 systemd[1]: Starting Ceph mds.k for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:27.269 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:26 smithi044 podman[95606]: 2024-03-19 00:35:26.9014631 +0000 UTC m=+0.017403849 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:27.269 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 podman[95606]: 2024-03-19 00:35:27.023853641 +0000 UTC m=+0.139794421 container create 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:35:27.269 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 podman[95606]: 2024-03-19 00:35:27.242948924 +0000 UTC m=+0.358889663 container init 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-19T00:35:27.269 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 podman[95606]: 2024-03-19 00:35:27.245538808 +0000 UTC m=+0.361479550 container start 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-19T00:35:27.606 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mds[95625]: -- 172.21.15.44:0/4229092467 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590efde1a40 con 0x5590efdb5000 2024-03-19T00:35:27.606 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k[95621]: starting mds.k at 2024-03-19T00:35:27.606 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 bash[95606]: 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d 2024-03-19T00:35:27.606 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:35:27 smithi044 systemd[1]: Started Ceph mds.k for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:27 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:27.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:27 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:27.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:27.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:27.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:27.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:27.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:27 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 -- 172.21.15.7:0/3463402959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144102300 msgr2=0x7fd144108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/3463402959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144102300 0x7fd144108b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ca0 tx=0x7fd13c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 -- 172.21.15.7:0/3463402959 shutdown_connections 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/3463402959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144102300 0x7fd144108b90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/3463402959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd144101940 0x7fd144101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/3463402959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144100750 0x7fd144100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 -- 172.21.15.7:0/3463402959 >> 172.21.15.7:0/3463402959 conn(0x7fd1440fbcc0 msgr2=0x7fd1440fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 -- 172.21.15.7:0/3463402959 shutdown_connections 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.847+0000 7fd14ae7e640 1 -- 172.21.15.7:0/3463402959 wait complete. 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 Processor -- start 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 -- start start 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 0x7fd144197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144101940 0x7fd144197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd144102300 0x7fd14419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c260 con 0x7fd144100750 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c0e0 con 0x7fd144101940 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14ae7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410c3e0 con 0x7fd144102300 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 0x7fd144197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 0x7fd144197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52598/0 (socket says 172.21.15.7:52598) 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 -- 172.21.15.7:0/1679743937 learned_addr learned my addr 172.21.15.7:0/1679743937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd143fff640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144101940 0x7fd144197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd14941b640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd144102300 0x7fd14419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 -- 172.21.15.7:0/1679743937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd144102300 msgr2=0x7fd14419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd144102300 0x7fd14419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 -- 172.21.15.7:0/1679743937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144101940 msgr2=0x7fd144197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144101940 0x7fd144197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 -- 172.21.15.7:0/1679743937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14419f640 con 0x7fd144100750 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd143fff640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144101940 0x7fd144197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.848+0000 7fd148c1a640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 0x7fd144197a20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd13800d770 tx=0x7fd13800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.849+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138017070 con 0x7fd144100750 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.849+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd138009c30 con 0x7fd144100750 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.849+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14419f930 con 0x7fd144100750 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.849+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138003d00 con 0x7fd144100750 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.849+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14419fe80 con 0x7fd144100750 2024-03-19T00:35:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.850+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1441a00b0 con 0x7fd144100750 2024-03-19T00:35:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.851+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fd138003ea0 con 0x7fd144100750 2024-03-19T00:35:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.851+0000 7fd141ffb640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1340a2800 0x7fd1340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.851+0000 7fd143fff640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1340a2800 0x7fd1340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.851+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd1380ce240 con 0x7fd144100750 2024-03-19T00:35:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.852+0000 7fd143fff640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1340a2800 0x7fd1340a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd144198f30 tx=0x7fd13000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:27.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:27.856+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd138009de0 con 0x7fd144100750 2024-03-19T00:35:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.096+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1441a0280 con 0x7fd144100750 2024-03-19T00:35:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.097+0000 7fd141ffb640 1 -- 172.21.15.7:0/1679743937 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fd13809b130 con 0x7fd144100750 2024-03-19T00:35:28.098 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:28.098 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T00:35:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-03-19T00:35:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1340a2800 msgr2=0x7fd1340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1340a2800 0x7fd1340a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd144198f30 tx=0x7fd13000b410 comp rx=0 tx=0).stop 2024-03-19T00:35:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 msgr2=0x7fd144197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 0x7fd144197a20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd13800d770 tx=0x7fd13800dc30 comp rx=0 tx=0).stop 2024-03-19T00:35:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 shutdown_connections 2024-03-19T00:35:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1340a2800 0x7fd1340a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd144102300 0x7fd14419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd144101940 0x7fd144197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 --2- 172.21.15.7:0/1679743937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd144100750 0x7fd144197a20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 >> 172.21.15.7:0/1679743937 conn(0x7fd1440fbcc0 msgr2=0x7fd1440fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 shutdown_connections 2024-03-19T00:35:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:28.099+0000 7fd14ae7e640 1 -- 172.21.15.7:0/1679743937 wait complete. 2024-03-19T00:35:28.638 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:28 smithi044 ceph-mon[66366]: Deploying daemon mds.j on smithi007 2024-03-19T00:35:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:28 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6832/1889634498,v1:172.21.15.44:6833/1889634498] up:boot 2024-03-19T00:35:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:28 smithi044 ceph-mon[66366]: fsmap 2 up:standby 2024-03-19T00:35:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:28 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T00:35:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:28 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1679743937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:28.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:28 smithi007 ceph-mon[62854]: Deploying daemon mds.j on smithi007 2024-03-19T00:35:28.719 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:28 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6832/1889634498,v1:172.21.15.44:6833/1889634498] up:boot 2024-03-19T00:35:28.719 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:28 smithi007 ceph-mon[62854]: fsmap 2 up:standby 2024-03-19T00:35:28.719 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:28 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T00:35:28.719 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:28 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1679743937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:28 smithi204 ceph-mon[66572]: Deploying daemon mds.j on smithi007 2024-03-19T00:35:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:28 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6832/1889634498,v1:172.21.15.44:6833/1889634498] up:boot 2024-03-19T00:35:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:28 smithi204 ceph-mon[66572]: fsmap 2 up:standby 2024-03-19T00:35:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:28 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T00:35:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:28 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1679743937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:29.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:29 smithi007 ceph-mon[62854]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:29.715 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:29 smithi007 systemd[1]: Starting Ceph mds.j for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:29 smithi204 ceph-mon[66572]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:29.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:29 smithi044 ceph-mon[66366]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:30.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:30.133 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:29 smithi007 podman[100072]: 2024-03-19 00:35:29.847183977 +0000 UTC m=+0.113314445 container create c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T00:35:30.134 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:29 smithi007 podman[100072]: 2024-03-19 00:35:29.751537798 +0000 UTC m=+0.017668267 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:30.134 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:30 smithi007 podman[100072]: 2024-03-19 00:35:30.133576117 +0000 UTC m=+0.399706585 container init c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 -- 172.21.15.7:0/3975421250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57881018b0 msgr2=0x7f5788101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3975421250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57881018b0 0x7f5788101cd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5770009900 tx=0x7f577002bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 -- 172.21.15.7:0/3975421250 shutdown_connections 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3975421250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5788103460 0x7f5788109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3975421250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788102aa0 0x7f5788102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3975421250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57881018b0 0x7f5788101cd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 -- 172.21.15.7:0/3975421250 >> 172.21.15.7:0/3975421250 conn(0x7f57880fce40 msgr2=0x7f57880ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 -- 172.21.15.7:0/3975421250 shutdown_connections 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 -- 172.21.15.7:0/3975421250 wait complete. 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.221+0000 7f578d3b9640 1 Processor -- start 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 -- start start 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57881018b0 0x7f5788197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 0x7f5788197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5788103460 0x7f578819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f578810d360 con 0x7f5788102aa0 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f578810d1e0 con 0x7f5788103460 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f578810d4e0 con 0x7f57881018b0 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 0x7f5788197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 0x7f5788197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52646/0 (socket says 172.21.15.7:52646) 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 -- 172.21.15.7:0/3054557464 learned_addr learned my addr 172.21.15.7:0/3054557464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57877fe640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5788103460 0x7f578819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f5786ffd640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57881018b0 0x7f5788197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 -- 172.21.15.7:0/3054557464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57881018b0 msgr2=0x7f5788197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57881018b0 0x7f5788197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 -- 172.21.15.7:0/3054557464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5788103460 msgr2=0x7f578819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5788103460 0x7f578819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 -- 172.21.15.7:0/3054557464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578819f6a0 con 0x7f5788102aa0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f57867fc640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 0x7f5788197fc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f577800d770 tx=0x7f577800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778017070 con 0x7f5788102aa0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57780054d0 con 0x7f5788102aa0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578819f990 con 0x7f5788102aa0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.222+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778004cf0 con 0x7f5788102aa0 2024-03-19T00:35:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.223+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578819fe90 con 0x7f5788102aa0 2024-03-19T00:35:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.224+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788108ca0 con 0x7f5788102aa0 2024-03-19T00:35:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.225+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f5778004e90 con 0x7f5788102aa0 2024-03-19T00:35:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.225+0000 7f577ffff640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57740a2800 0x7f57740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.225+0000 7f5786ffd640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57740a2800 0x7f57740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.225+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f57780cdf70 con 0x7f5788102aa0 2024-03-19T00:35:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.226+0000 7f5786ffd640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57740a2800 0x7f57740a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5770005ab0 tx=0x7f5770005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:30.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.230+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f577809ae60 con 0x7f5788102aa0 2024-03-19T00:35:30.465 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:30 smithi007 podman[100072]: 2024-03-19 00:35:30.13719371 +0000 UTC m=+0.403324179 container start c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-19T00:35:30.465 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:30 smithi007 ceph-mds[100108]: -- 172.21.15.7:0/262395174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55871145ba40 con 0x55871142ec00 2024-03-19T00:35:30.465 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j[100087]: starting mds.j at 2024-03-19T00:35:30.465 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:30 smithi007 bash[100072]: c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 2024-03-19T00:35:30.465 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:35:30 smithi007 systemd[1]: Started Ceph mds.j for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:30.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.473+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578802d440 con 0x7f5788102aa0 2024-03-19T00:35:30.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.473+0000 7f577ffff640 1 -- 172.21.15.7:0/3054557464 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f577809ef00 con 0x7f5788102aa0 2024-03-19T00:35:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:30.474 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T00:35:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-03-19T00:35:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57740a2800 msgr2=0x7f57740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57740a2800 0x7f57740a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5770005ab0 tx=0x7f5770005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 msgr2=0x7f5788197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 0x7f5788197fc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f577800d770 tx=0x7f577800dc30 comp rx=0 tx=0).stop 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 shutdown_connections 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57740a2800 0x7f57740a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5788103460 0x7f578819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5788102aa0 0x7f5788197fc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 --2- 172.21.15.7:0/3054557464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57881018b0 0x7f5788197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.475+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 >> 172.21.15.7:0/3054557464 conn(0x7f57880fce40 msgr2=0x7f5788105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.476+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 shutdown_connections 2024-03-19T00:35:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:30.476+0000 7f578d3b9640 1 -- 172.21.15.7:0/3054557464 wait complete. 2024-03-19T00:35:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: Deploying daemon mds.d on smithi007 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3054557464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.7:6834/2506887195,v1:172.21.15.7:6835/2506887195] up:boot 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: fsmap 3 up:standby 2024-03-19T00:35:31.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:31 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T00:35:31.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:31.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:31.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:31.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:31.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: Deploying daemon mds.d on smithi007 2024-03-19T00:35:31.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3054557464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:31.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.7:6834/2506887195,v1:172.21.15.7:6835/2506887195] up:boot 2024-03-19T00:35:31.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: fsmap 3 up:standby 2024-03-19T00:35:31.694 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:31 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: Deploying daemon mds.d on smithi007 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3054557464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.7:6834/2506887195,v1:172.21.15.7:6835/2506887195] up:boot 2024-03-19T00:35:31.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: fsmap 3 up:standby 2024-03-19T00:35:31.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:31 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T00:35:32.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 -- 172.21.15.7:0/356369657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834102a60 msgr2=0x7fa834102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 --2- 172.21.15.7:0/356369657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834102a60 0x7fa834102ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa828009900 tx=0x7fa82802bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 -- 172.21.15.7:0/356369657 shutdown_connections 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 --2- 172.21.15.7:0/356369657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834103420 0x7fa834109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 --2- 172.21.15.7:0/356369657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834102a60 0x7fa834102ee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 --2- 172.21.15.7:0/356369657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834101870 0x7fa834101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 -- 172.21.15.7:0/356369657 >> 172.21.15.7:0/356369657 conn(0x7fa8340fcde0 msgr2=0x7fa8340ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 -- 172.21.15.7:0/356369657 shutdown_connections 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 -- 172.21.15.7:0/356369657 wait complete. 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.634+0000 7fa839552640 1 Processor -- start 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 -- start start 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 0x7fa834197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834102a60 0x7fa834197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834103420 0x7fa83419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa83410d380 con 0x7fa834101870 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa83410d200 con 0x7fa834103420 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa839552640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa83410d500 con 0x7fa834102a60 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 0x7fa834197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 0x7fa834197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52664/0 (socket says 172.21.15.7:52664) 2024-03-19T00:35:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 -- 172.21.15.7:0/2119153693 learned_addr learned my addr 172.21.15.7:0/2119153693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa8337fe640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834103420 0x7fa83419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa8327fc640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834102a60 0x7fa834197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 -- 172.21.15.7:0/2119153693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834102a60 msgr2=0x7fa834197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834102a60 0x7fa834197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 -- 172.21.15.7:0/2119153693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834103420 msgr2=0x7fa83419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834103420 0x7fa83419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 -- 172.21.15.7:0/2119153693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83419f680 con 0x7fa834101870 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.635+0000 7fa832ffd640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 0x7fa834197a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c970 tx=0x7fa82400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.636+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824004220 con 0x7fa834101870 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.636+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83419f970 con 0x7fa834101870 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.636+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8240043c0 con 0x7fa834101870 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.636+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82400f450 con 0x7fa834101870 2024-03-19T00:35:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.636+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83419fee0 con 0x7fa834101870 2024-03-19T00:35:32.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.637+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa834108c10 con 0x7fa834101870 2024-03-19T00:35:32.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.638+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fa8240079b0 con 0x7fa834101870 2024-03-19T00:35:32.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.639+0000 7fa81bfff640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa81c0a2800 0x7fa81c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.639+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa8240ce2b0 con 0x7fa834101870 2024-03-19T00:35:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.639+0000 7fa8327fc640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa81c0a2800 0x7fa81c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.640+0000 7fa8327fc640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa81c0a2800 0x7fa81c0a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa828009230 tx=0x7fa828005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.644+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa824014030 con 0x7fa834101870 2024-03-19T00:35:32.715 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:32 smithi007 systemd[1]: Starting Ceph mds.d for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.894+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa83402d1f0 con 0x7fa834101870 2024-03-19T00:35:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.894+0000 7fa81bfff640 1 -- 172.21.15.7:0/2119153693 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fa82409b1a0 con 0x7fa834101870 2024-03-19T00:35:32.895 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:32.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T00:35:32.896 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa81c0a2800 msgr2=0x7fa81c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa81c0a2800 0x7fa81c0a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa828009230 tx=0x7fa828005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 msgr2=0x7fa834197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 0x7fa834197a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c970 tx=0x7fa82400ce30 comp rx=0 tx=0).stop 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 shutdown_connections 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa81c0a2800 0x7fa81c0a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa834103420 0x7fa83419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834102a60 0x7fa834197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 --2- 172.21.15.7:0/2119153693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa834101870 0x7fa834197a60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.896+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 >> 172.21.15.7:0/2119153693 conn(0x7fa8340fcde0 msgr2=0x7fa834105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.897+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 shutdown_connections 2024-03-19T00:35:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:32.897+0000 7fa839552640 1 -- 172.21.15.7:0/2119153693 wait complete. 2024-03-19T00:35:33.068 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:32 smithi007 podman[100458]: 2024-03-19 00:35:32.705105718 +0000 UTC m=+0.026476856 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:33.068 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:32 smithi007 podman[100458]: 2024-03-19 00:35:32.830304598 +0000 UTC m=+0.151675735 container create 0912a57a2241f7a3737aa7842c5e41119ca882f367016b2339472d363cbc9bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-19T00:35:33.068 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:33 smithi007 podman[100458]: 2024-03-19 00:35:33.041763498 +0000 UTC m=+0.363134639 container init 0912a57a2241f7a3737aa7842c5e41119ca882f367016b2339472d363cbc9bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:35:33.068 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:33 smithi007 podman[100458]: 2024-03-19 00:35:33.044332501 +0000 UTC m=+0.365703637 container start 0912a57a2241f7a3737aa7842c5e41119ca882f367016b2339472d363cbc9bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-19T00:35:33.465 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mds[100479]: -- 172.21.15.7:0/2476368356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf3abf7a40 con 0x55bf3abcb000 2024-03-19T00:35:33.465 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d[100475]: starting mds.d at 2024-03-19T00:35:33.465 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:33 smithi007 bash[100458]: 0912a57a2241f7a3737aa7842c5e41119ca882f367016b2339472d363cbc9bbb 2024-03-19T00:35:33.465 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:35:33 smithi007 systemd[1]: Started Ceph mds.d for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2119153693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:33 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2119153693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:33 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2119153693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:33.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:33 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:34.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:34 smithi044 ceph-mon[66366]: Deploying daemon mds.e on smithi044 2024-03-19T00:35:34.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:34 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.7:6836/520010551,v1:172.21.15.7:6837/520010551] up:boot 2024-03-19T00:35:34.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:34 smithi044 ceph-mon[66366]: fsmap 4 up:standby 2024-03-19T00:35:34.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:34 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T00:35:34.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:34 smithi007 ceph-mon[62854]: Deploying daemon mds.e on smithi044 2024-03-19T00:35:34.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:34 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.7:6836/520010551,v1:172.21.15.7:6837/520010551] up:boot 2024-03-19T00:35:34.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:34 smithi007 ceph-mon[62854]: fsmap 4 up:standby 2024-03-19T00:35:34.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:34 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T00:35:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:34 smithi204 ceph-mon[66572]: Deploying daemon mds.e on smithi044 2024-03-19T00:35:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:34 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.7:6836/520010551,v1:172.21.15.7:6837/520010551] up:boot 2024-03-19T00:35:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:34 smithi204 ceph-mon[66572]: fsmap 4 up:standby 2024-03-19T00:35:34.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:34 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T00:35:34.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:35.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 -- 172.21.15.7:0/2579378374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174102310 msgr2=0x7fa174104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 --2- 172.21.15.7:0/2579378374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174102310 0x7fa174104700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009900 tx=0x7fa16c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 -- 172.21.15.7:0/2579378374 shutdown_connections 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 --2- 172.21.15.7:0/2579378374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174107570 0x7fa174109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 --2- 172.21.15.7:0/2579378374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa174104c40 0x7fa174107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 --2- 172.21.15.7:0/2579378374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174102310 0x7fa174104700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 -- 172.21.15.7:0/2579378374 >> 172.21.15.7:0/2579378374 conn(0x7fa1740fbcc0 msgr2=0x7fa1740fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 -- 172.21.15.7:0/2579378374 shutdown_connections 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.025+0000 7fa17936f640 1 -- 172.21.15.7:0/2579378374 wait complete. 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 Processor -- start 2024-03-19T00:35:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 -- start start 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174102310 0x7fa174197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 0x7fa174197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa174107570 0x7fa17419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa17410d440 con 0x7fa174104c40 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa17410d2c0 con 0x7fa174102310 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.026+0000 7fa17936f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa17410d5c0 con 0x7fa174107570 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 0x7fa174197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 0x7fa174197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57808/0 (socket says 172.21.15.7:57808) 2024-03-19T00:35:35.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 -- 172.21.15.7:0/3946989640 learned_addr learned my addr 172.21.15.7:0/3946989640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1737fe640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa174107570 0x7fa17419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa172ffd640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174102310 0x7fa174197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 -- 172.21.15.7:0/3946989640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa174107570 msgr2=0x7fa17419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa174107570 0x7fa17419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 -- 172.21.15.7:0/3946989640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174102310 msgr2=0x7fa174197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174102310 0x7fa174197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 -- 172.21.15.7:0/3946989640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17419f670 con 0x7fa174104c40 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa1727fc640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 0x7fa174197fe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c970 tx=0x7fa15c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c014070 con 0x7fa174104c40 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15c004440 con 0x7fa174104c40 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17419f960 con 0x7fa174104c40 2024-03-19T00:35:35.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c003e50 con 0x7fa174104c40 2024-03-19T00:35:35.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.027+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17419ff30 con 0x7fa174104c40 2024-03-19T00:35:35.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.028+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1740fd7b0 con 0x7fa174104c40 2024-03-19T00:35:35.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.029+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fa15c017070 con 0x7fa174104c40 2024-03-19T00:35:35.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.030+0000 7fa16bfff640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa1600a29a0 0x7fa1600a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:35.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.030+0000 7fa172ffd640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa1600a29a0 0x7fa1600a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:35.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.030+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa15c0ce190 con 0x7fa174104c40 2024-03-19T00:35:35.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.031+0000 7fa172ffd640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa1600a29a0 0x7fa1600a4e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa16c02c840 tx=0x7fa16c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.035+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa15c01e030 con 0x7fa174104c40 2024-03-19T00:35:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.257+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa174062420 con 0x7fa174104c40 2024-03-19T00:35:35.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.257+0000 7fa16bfff640 1 -- 172.21.15.7:0/3946989640 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7fa15c09b630 con 0x7fa174104c40 2024-03-19T00:35:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:35.259 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T00:35:35.259 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 5 2024-03-19T00:35:35.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa1600a29a0 msgr2=0x7fa1600a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa1600a29a0 0x7fa1600a4e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa16c02c840 tx=0x7fa16c000bc0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 msgr2=0x7fa174197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 0x7fa174197fe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c970 tx=0x7fa15c00ce30 comp rx=0 tx=0).stop 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 shutdown_connections 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa1600a29a0 0x7fa1600a4e60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa174107570 0x7fa17419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa174104c40 0x7fa174197fe0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 --2- 172.21.15.7:0/3946989640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174102310 0x7fa174197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.259+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 >> 172.21.15.7:0/3946989640 conn(0x7fa1740fbcc0 msgr2=0x7fa174102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.260+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 shutdown_connections 2024-03-19T00:35:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:35.260+0000 7fa17936f640 1 -- 172.21.15.7:0/3946989640 wait complete. 2024-03-19T00:35:35.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:35 smithi044 ceph-mon[66366]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:35.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:35 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3946989640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:35.607 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 systemd[1]: Starting Ceph mds.e for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:35 smithi007 ceph-mon[62854]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:35 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3946989640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:35 smithi204 ceph-mon[66572]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:35 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3946989640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:35.958 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 podman[95922]: 2024-03-19 00:35:35.58154402 +0000 UTC m=+0.017537308 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:35.959 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 podman[95922]: 2024-03-19 00:35:35.73075377 +0000 UTC m=+0.166747050 container create ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-03-19T00:35:35.959 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 podman[95922]: 2024-03-19 00:35:35.958405939 +0000 UTC m=+0.394399220 container init ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-19T00:35:36.356 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 podman[95922]: 2024-03-19 00:35:35.961347672 +0000 UTC m=+0.397340957 container start ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=) 2024-03-19T00:35:36.356 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 ceph-mds[95941]: -- 172.21.15.44:0/148474698 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560345539a40 con 0x56034550c800 2024-03-19T00:35:36.356 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e[95937]: starting mds.e at 2024-03-19T00:35:36.356 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:36 smithi044 bash[95922]: ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b 2024-03-19T00:35:36.356 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:35:36 smithi044 systemd[1]: Started Ceph mds.e for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:37.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/2424226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4102390 msgr2=0x7f3db4104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/2424226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4102390 0x7f3db4104780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009900 tx=0x7f3da802bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/2424226776 shutdown_connections 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/2424226776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db41075f0 0x7f3db41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/2424226776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3db4104cc0 0x7f3db41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/2424226776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4102390 0x7f3db4104780 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/2424226776 >> 172.21.15.7:0/2424226776 conn(0x7f3db40fbce0 msgr2=0x7f3db40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/2424226776 shutdown_connections 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.385+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/2424226776 wait complete. 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 Processor -- start 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 -- start start 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4102390 0x7f3db4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3db4104cc0 0x7f3db4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 0x7f3db419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3db410d3b0 con 0x7f3db41075f0 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3db410d230 con 0x7f3db4104cc0 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3dbb1b9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3db410d530 con 0x7f3db4102390 2024-03-19T00:35:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 0x7f3db419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 0x7f3db419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57830/0 (socket says 172.21.15.7:57830) 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 -- 172.21.15.7:0/1759276193 learned_addr learned my addr 172.21.15.7:0/1759276193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db8f55640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4102390 0x7f3db4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db3fff640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3db4104cc0 0x7f3db4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 -- 172.21.15.7:0/1759276193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4102390 msgr2=0x7f3db4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4102390 0x7f3db4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 -- 172.21.15.7:0/1759276193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3db4104cc0 msgr2=0x7f3db4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3db4104cc0 0x7f3db4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.386+0000 7f3db9756640 1 -- 172.21.15.7:0/1759276193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db419f700 con 0x7f3db41075f0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.387+0000 7f3db9756640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 0x7f3db419f090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00cad0 tx=0x7f3dac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.387+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00ae20 con 0x7f3db41075f0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.387+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db419f9f0 con 0x7f3db41075f0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.387+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dac005020 con 0x7f3db41075f0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.387+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00f6b0 con 0x7f3db41075f0 2024-03-19T00:35:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.387+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db419ff40 con 0x7f3db41075f0 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:37.413 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: Deploying daemon mds.f on smithi204 2024-03-19T00:35:37.414 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6834/1036080774,v1:172.21.15.44:6835/1036080774] up:boot 2024-03-19T00:35:37.414 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: fsmap 5 up:standby 2024-03-19T00:35:37.414 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:37 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.388+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db40fd7d0 con 0x7f3db41075f0 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.389+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f3dac0051d0 con 0x7f3db41075f0 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.390+0000 7f3db1ffb640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3da40a2800 0x7f3da40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.390+0000 7f3db8f55640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3da40a2800 0x7f3da40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.390+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3dac0ce5b0 con 0x7f3db41075f0 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.390+0000 7f3db8f55640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3da40a2800 0x7f3da40a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009230 tx=0x7f3da8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.394+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3dac014030 con 0x7f3db41075f0 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: Deploying daemon mds.f on smithi204 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6834/1036080774,v1:172.21.15.44:6835/1036080774] up:boot 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: fsmap 5 up:standby 2024-03-19T00:35:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:37 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T00:35:37.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:37.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: Deploying daemon mds.f on smithi204 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6834/1036080774,v1:172.21.15.44:6835/1036080774] up:boot 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: fsmap 5 up:standby 2024-03-19T00:35:37.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:37 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T00:35:37.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.616+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3db4062420 con 0x7f3db41075f0 2024-03-19T00:35:37.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.616+0000 7f3db1ffb640 1 -- 172.21.15.7:0/1759276193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7f3dac09b400 con 0x7f3db41075f0 2024-03-19T00:35:37.617 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:37.617 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-19T00:35:37.618 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-03-19T00:35:37.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3da40a2800 msgr2=0x7f3da40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:37.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3da40a2800 0x7f3da40a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009230 tx=0x7f3da8005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 msgr2=0x7f3db419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 0x7f3db419f090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00cad0 tx=0x7f3dac00cf90 comp rx=0 tx=0).stop 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 shutdown_connections 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3da40a2800 0x7f3da40a4cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db41075f0 0x7f3db419f090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3db4104cc0 0x7f3db4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 --2- 172.21.15.7:0/1759276193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db4102390 0x7f3db4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 >> 172.21.15.7:0/1759276193 conn(0x7f3db40fbce0 msgr2=0x7f3db4102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 shutdown_connections 2024-03-19T00:35:37.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:37.618+0000 7f3dbb1b9640 1 -- 172.21.15.7:0/1759276193 wait complete. 2024-03-19T00:35:38.305 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:38 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1759276193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:38.305 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 systemd[1]: Starting Ceph mds.f for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:38.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:38 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1759276193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:38.597 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 podman[96831]: 2024-03-19 00:35:38.371703612 +0000 UTC m=+0.018717874 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:38.597 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 podman[96831]: 2024-03-19 00:35:38.480168943 +0000 UTC m=+0.127183206 container create 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:35:38.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:38 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1759276193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:39.015 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 podman[96831]: 2024-03-19 00:35:38.707884396 +0000 UTC m=+0.354898666 container init 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:35:39.015 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 podman[96831]: 2024-03-19 00:35:38.710660026 +0000 UTC m=+0.357674296 container start 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0) 2024-03-19T00:35:39.015 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 ceph-mds[96858]: -- 172.21.15.204:0/4130077570 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574bef0ba40 con 0x5574beedf000 2024-03-19T00:35:39.015 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f[96854]: starting mds.f at 2024-03-19T00:35:39.015 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 bash[96831]: 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 2024-03-19T00:35:39.015 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:35:38 smithi204 systemd[1]: Started Ceph mds.f for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:39.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 -- 172.21.15.7:0/4021659404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 msgr2=0x7f62d4100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 --2- 172.21.15.7:0/4021659404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4100b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009900 tx=0x7f62c402bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 -- 172.21.15.7:0/4021659404 shutdown_connections 2024-03-19T00:35:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 --2- 172.21.15.7:0/4021659404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d4102300 0x7f62d4108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 --2- 172.21.15.7:0/4021659404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62d4101940 0x7f62d4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 --2- 172.21.15.7:0/4021659404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4100b70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 -- 172.21.15.7:0/4021659404 >> 172.21.15.7:0/4021659404 conn(0x7f62d40fbcc0 msgr2=0x7f62d40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 -- 172.21.15.7:0/4021659404 shutdown_connections 2024-03-19T00:35:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 -- 172.21.15.7:0/4021659404 wait complete. 2024-03-19T00:35:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 Processor -- start 2024-03-19T00:35:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.746+0000 7f62d9044640 1 -- start start 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d9044640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4069630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d9044640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62d4101940 0x7f62d419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d9044640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d4102300 0x7f62d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d9044640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62d410c250 con 0x7f62d4100750 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d9044640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62d410c0d0 con 0x7f62d4102300 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d9044640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62d410c3d0 con 0x7f62d4101940 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4069630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4069630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57848/0 (socket says 172.21.15.7:57848) 2024-03-19T00:35:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 -- 172.21.15.7:0/3593304340 learned_addr learned my addr 172.21.15.7:0/3593304340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d359e640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d4102300 0x7f62d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62cbfff640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62d4101940 0x7f62d419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 -- 172.21.15.7:0/3593304340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62d4101940 msgr2=0x7f62d419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62d4101940 0x7f62d419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 -- 172.21.15.7:0/3593304340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d4102300 msgr2=0x7f62d419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d4102300 0x7f62d419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 -- 172.21.15.7:0/3593304340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d419f630 con 0x7f62d4100750 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d2d9d640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4069630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62c4000c00 tx=0x7f62c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.747+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4043020 con 0x7f62d4100750 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.748+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d419f8c0 con 0x7f62d4100750 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.748+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c4004aa0 con 0x7f62d4100750 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.748+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4003d00 con 0x7f62d4100750 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.748+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d419fe60 con 0x7f62d4100750 2024-03-19T00:35:39.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.749+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d4107a90 con 0x7f62d4100750 2024-03-19T00:35:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.750+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f62c4046020 con 0x7f62d4100750 2024-03-19T00:35:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.750+0000 7f62d0d99640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.751+0000 7f62cbfff640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.751+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f62c40f0f50 con 0x7f62d4100750 2024-03-19T00:35:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.751+0000 7f62cbfff640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62c00a2800 0x7f62c00a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f62bc005e10 tx=0x7f62bc005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.754+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f62c404d020 con 0x7f62d4100750 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: Deploying daemon mds.c on smithi204 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6834/3301743833,v1:172.21.15.204:6835/3301743833] up:boot 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: fsmap 6 up:standby 2024-03-19T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:39 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T00:35:39.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.977+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62d402d1f0 con 0x7f62d4100750 2024-03-19T00:35:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.978+0000 7f62d0d99640 1 -- 172.21.15.7:0/3593304340 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f62c40be490 con 0x7f62d4100750 2024-03-19T00:35:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:39.979 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-19T00:35:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-03-19T00:35:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62c00a2800 msgr2=0x7f62c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62c00a2800 0x7f62c00a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f62bc005e10 tx=0x7f62bc005da0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 msgr2=0x7f62d4069630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4069630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62c4000c00 tx=0x7f62c4004270 comp rx=0 tx=0).stop 2024-03-19T00:35:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 shutdown_connections 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62d4102300 0x7f62d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62d4101940 0x7f62d419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 --2- 172.21.15.7:0/3593304340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d4100750 0x7f62d4069630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 >> 172.21.15.7:0/3593304340 conn(0x7f62d40fbcc0 msgr2=0x7f62d4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 shutdown_connections 2024-03-19T00:35:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:39.980+0000 7f62d9044640 1 -- 172.21.15.7:0/3593304340 wait complete. 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: Deploying daemon mds.c on smithi204 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6834/3301743833,v1:172.21.15.204:6835/3301743833] up:boot 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: fsmap 6 up:standby 2024-03-19T00:35:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:39 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T00:35:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: Deploying daemon mds.c on smithi204 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6834/3301743833,v1:172.21.15.204:6835/3301743833] up:boot 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: fsmap 6 up:standby 2024-03-19T00:35:40.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:39 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T00:35:41.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:40 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3593304340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:41.223 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:40 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3593304340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:41.223 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 systemd[1]: Starting Ceph mds.c for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:41.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:40 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3593304340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:41.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 podman[97148]: 2024-03-19 00:35:41.125650541 +0000 UTC m=+0.017940967 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:41.515 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 podman[97148]: 2024-03-19 00:35:41.235716621 +0000 UTC m=+0.128007041 container create 8af9b6feaac41ae7d05c302840b87bb4f683c13b76c34239caada13fd3ed5818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-19T00:35:41.872 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 podman[97148]: 2024-03-19 00:35:41.580678351 +0000 UTC m=+0.472968773 container init 8af9b6feaac41ae7d05c302840b87bb4f683c13b76c34239caada13fd3ed5818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-19T00:35:41.872 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 podman[97148]: 2024-03-19 00:35:41.583328952 +0000 UTC m=+0.475619369 container start 8af9b6feaac41ae7d05c302840b87bb4f683c13b76c34239caada13fd3ed5818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:35:41.872 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mds[97167]: -- 172.21.15.204:0/398830821 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b62fe07a40 con 0x55b62fdda800 2024-03-19T00:35:41.872 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c[97163]: starting mds.c at 2024-03-19T00:35:41.872 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 bash[97148]: 8af9b6feaac41ae7d05c302840b87bb4f683c13b76c34239caada13fd3ed5818 2024-03-19T00:35:41.872 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:35:41 smithi204 systemd[1]: Started Ceph mds.c for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:41.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 -- 172.21.15.7:0/751340155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 msgr2=0x7f76e8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/751340155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8101e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009900 tx=0x7f76d802bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 -- 172.21.15.7:0/751340155 shutdown_connections 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/751340155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e8102370 0x7f76e8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/751340155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8101e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/751340155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e81007c0 0x7f76e8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 -- 172.21.15.7:0/751340155 >> 172.21.15.7:0/751340155 conn(0x7f76e80fbd50 msgr2=0x7f76e80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 -- 172.21.15.7:0/751340155 shutdown_connections 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.108+0000 7f76ef6c6640 1 -- 172.21.15.7:0/751340155 wait complete. 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 Processor -- start 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 -- start start 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e81007c0 0x7f76e8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e8102370 0x7f76e819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810c280 con 0x7f76e81019b0 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810c100 con 0x7f76e81007c0 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ef6c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810c400 con 0x7f76e8102370 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ecc61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ecc61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57872/0 (socket says 172.21.15.7:57872) 2024-03-19T00:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ecc61640 1 -- 172.21.15.7:0/1304819556 learned_addr learned my addr 172.21.15.7:0/1304819556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.109+0000 7f76ed462640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e81007c0 0x7f76e8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76edc63640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e8102370 0x7f76e819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ecc61640 1 -- 172.21.15.7:0/1304819556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e8102370 msgr2=0x7f76e819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ecc61640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e8102370 0x7f76e819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ecc61640 1 -- 172.21.15.7:0/1304819556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e81007c0 msgr2=0x7f76e8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ecc61640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e81007c0 0x7f76e8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ecc61640 1 -- 172.21.15.7:0/1304819556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e819f6e0 con 0x7f76e81019b0 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76edc63640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e8102370 0x7f76e819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ed462640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e81007c0 0x7f76e8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ecc61640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8198000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8043020 con 0x7f76e81019b0 2024-03-19T00:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d8004440 con 0x7f76e81019b0 2024-03-19T00:35:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e819f970 con 0x7f76e81019b0 2024-03-19T00:35:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003bf0 con 0x7f76e81019b0 2024-03-19T00:35:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.110+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e819ff10 con 0x7f76e81019b0 2024-03-19T00:35:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.111+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8068af0 con 0x7f76e81019b0 2024-03-19T00:35:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.112+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f76d8046020 con 0x7f76e81019b0 2024-03-19T00:35:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.113+0000 7f76de7fc640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76d40a2880 0x7f76d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.113+0000 7f76ed462640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76d40a2880 0x7f76d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.113+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f76d80f0fb0 con 0x7f76e81019b0 2024-03-19T00:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.113+0000 7f76ed462640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76d40a2880 0x7f76d40a4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005fd0 tx=0x7f76e400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.116+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76d804d020 con 0x7f76e81019b0 2024-03-19T00:35:42.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.338+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76e81a0330 con 0x7f76e81019b0 2024-03-19T00:35:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.339+0000 7f76de7fc640 1 -- 172.21.15.7:0/1304819556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f76d80be450 con 0x7f76e81019b0 2024-03-19T00:35:42.340 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:42.340 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T00:35:42.340 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-03-19T00:35:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76d40a2880 msgr2=0x7f76d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76d40a2880 0x7f76d40a4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005fd0 tx=0x7f76e400b390 comp rx=0 tx=0).stop 2024-03-19T00:35:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 msgr2=0x7f76e8198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8198000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d8004040 comp rx=0 tx=0).stop 2024-03-19T00:35:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 shutdown_connections 2024-03-19T00:35:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76d40a2880 0x7f76d40a4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e8102370 0x7f76e819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76e81019b0 0x7f76e8198000 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 --2- 172.21.15.7:0/1304819556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76e81007c0 0x7f76e8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 >> 172.21.15.7:0/1304819556 conn(0x7f76e80fbd50 msgr2=0x7f76e8069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 shutdown_connections 2024-03-19T00:35:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:42.341+0000 7f76ef6c6640 1 -- 172.21.15.7:0/1304819556 wait complete. 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:43.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:42 smithi044 ceph-mon[66366]: Deploying daemon mds.h on smithi044 2024-03-19T00:35:43.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:42 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6836/2630641732,v1:172.21.15.204:6837/2630641732] up:boot 2024-03-19T00:35:43.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:42 smithi044 ceph-mon[66366]: fsmap 7 up:standby 2024-03-19T00:35:43.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T00:35:43.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:42 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1304819556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:43.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:42 smithi007 ceph-mon[62854]: Deploying daemon mds.h on smithi044 2024-03-19T00:35:43.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:42 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6836/2630641732,v1:172.21.15.204:6837/2630641732] up:boot 2024-03-19T00:35:43.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:42 smithi007 ceph-mon[62854]: fsmap 7 up:standby 2024-03-19T00:35:43.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T00:35:43.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:42 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1304819556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:43.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:42 smithi204 ceph-mon[66572]: Deploying daemon mds.h on smithi044 2024-03-19T00:35:43.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:42 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6836/2630641732,v1:172.21.15.204:6837/2630641732] up:boot 2024-03-19T00:35:43.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:42 smithi204 ceph-mon[66572]: fsmap 7 up:standby 2024-03-19T00:35:43.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T00:35:43.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:42 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1304819556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:43 smithi044 ceph-mon[66366]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:44.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:43 smithi007 ceph-mon[62854]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:44.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:43 smithi204 ceph-mon[66572]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:44.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 -- 172.21.15.7:0/3290341486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41019b0 msgr2=0x7febd4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 --2- 172.21.15.7:0/3290341486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41019b0 0x7febd4101e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febc4009900 tx=0x7febc402bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 -- 172.21.15.7:0/3290341486 shutdown_connections 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 --2- 172.21.15.7:0/3290341486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd4102370 0x7febd4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 --2- 172.21.15.7:0/3290341486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41019b0 0x7febd4101e30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 --2- 172.21.15.7:0/3290341486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd41007c0 0x7febd4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.467+0000 7febda3d4640 1 -- 172.21.15.7:0/3290341486 >> 172.21.15.7:0/3290341486 conn(0x7febd40fbd50 msgr2=0x7febd40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 -- 172.21.15.7:0/3290341486 shutdown_connections 2024-03-19T00:35:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 -- 172.21.15.7:0/3290341486 wait complete. 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 Processor -- start 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 -- start start 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 0x7febd4104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41019b0 0x7febd4102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102370 0x7febd41032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7febd410c280 con 0x7febd41007c0 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febd410c100 con 0x7febd41019b0 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.468+0000 7febda3d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febd410c400 con 0x7febd4102370 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 0x7febd4104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 0x7febd4104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57894/0 (socket says 172.21.15.7:57894) 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 -- 172.21.15.7:0/2420391203 learned_addr learned my addr 172.21.15.7:0/2420391203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd37fe640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41019b0 0x7febd4102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd8971640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102370 0x7febd41032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 -- 172.21.15.7:0/2420391203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102370 msgr2=0x7febd41032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102370 0x7febd41032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 -- 172.21.15.7:0/2420391203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41019b0 msgr2=0x7febd4102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41019b0 0x7febd4102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 -- 172.21.15.7:0/2420391203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd4103b90 con 0x7febd41007c0 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd37fe640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41019b0 0x7febd4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd8971640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102370 0x7febd41032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd3fff640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 0x7febd4104700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9c0 tx=0x7febbc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00cd80 con 0x7febd41007c0 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febbc0035f0 con 0x7febd41007c0 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd41a1c20 con 0x7febd41007c0 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc010690 con 0x7febd41007c0 2024-03-19T00:35:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.469+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41a20f0 con 0x7febd41007c0 2024-03-19T00:35:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.471+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd41a22c0 con 0x7febd41007c0 2024-03-19T00:35:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.472+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7febbc0037a0 con 0x7febd41007c0 2024-03-19T00:35:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.472+0000 7febd17fa640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7febc00a2800 0x7febc00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.472+0000 7febd37fe640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7febc00a2800 0x7febc00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.473+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7febbc0ceb70 con 0x7febd41007c0 2024-03-19T00:35:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.473+0000 7febd37fe640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7febc00a2800 0x7febc00a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7febc4009230 tx=0x7febc4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:44.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.477+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7febbc09c0b0 con 0x7febd41007c0 2024-03-19T00:35:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.700+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd41a25b0 con 0x7febd41007c0 2024-03-19T00:35:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.700+0000 7febd17fa640 1 -- 172.21.15.7:0/2420391203 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7febbc09b9b0 con 0x7febd41007c0 2024-03-19T00:35:44.701 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:44.702 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T00:35:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-03-19T00:35:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7febc00a2800 msgr2=0x7febc00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7febc00a2800 0x7febc00a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7febc4009230 tx=0x7febc4005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 msgr2=0x7febd4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 0x7febd4104700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9c0 tx=0x7febbc00ee80 comp rx=0 tx=0).stop 2024-03-19T00:35:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 shutdown_connections 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7febc00a2800 0x7febc00a4cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102370 0x7febd41032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41019b0 0x7febd4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 --2- 172.21.15.7:0/2420391203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41007c0 0x7febd4104700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 >> 172.21.15.7:0/2420391203 conn(0x7febd40fbd50 msgr2=0x7febd40fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 shutdown_connections 2024-03-19T00:35:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:44.702+0000 7febda3d4640 1 -- 172.21.15.7:0/2420391203 wait complete. 2024-03-19T00:35:44.856 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:35:44 smithi044 systemd[1]: Started Ceph mds.h for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2420391203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2420391203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:45.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:45.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2420391203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:45.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:45.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:45.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:45.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:45.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:45.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:45 smithi007 ceph-mon[62854]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:45 smithi007 ceph-mon[62854]: Deploying daemon mds.i on smithi204 2024-03-19T00:35:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:45 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] up:boot 2024-03-19T00:35:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:45 smithi007 ceph-mon[62854]: fsmap 8 up:standby 2024-03-19T00:35:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T00:35:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:45 smithi204 ceph-mon[66572]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:45 smithi204 ceph-mon[66572]: Deploying daemon mds.i on smithi204 2024-03-19T00:35:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:45 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] up:boot 2024-03-19T00:35:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:45 smithi204 ceph-mon[66572]: fsmap 8 up:standby 2024-03-19T00:35:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T00:35:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:45 smithi044 ceph-mon[66366]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:45 smithi044 ceph-mon[66366]: Deploying daemon mds.i on smithi204 2024-03-19T00:35:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:45 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] up:boot 2024-03-19T00:35:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:45 smithi044 ceph-mon[66366]: fsmap 8 up:standby 2024-03-19T00:35:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T00:35:46.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 -- 172.21.15.7:0/3626389739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec101940 msgr2=0x7f18ec101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/3626389739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec101940 0x7f18ec101dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009900 tx=0x7f18d402bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 -- 172.21.15.7:0/3626389739 shutdown_connections 2024-03-19T00:35:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/3626389739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 0x7f18ec108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/3626389739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec101940 0x7f18ec101dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/3626389739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18ec100750 0x7f18ec100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 -- 172.21.15.7:0/3626389739 >> 172.21.15.7:0/3626389739 conn(0x7f18ec0fbcc0 msgr2=0x7f18ec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 -- 172.21.15.7:0/3626389739 shutdown_connections 2024-03-19T00:35:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.830+0000 7f18f0eb2640 1 -- 172.21.15.7:0/3626389739 wait complete. 2024-03-19T00:35:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 Processor -- start 2024-03-19T00:35:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 -- start start 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 0x7f18ec199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18ec101940 0x7f18ec19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 0x7f18ec1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec10c280 con 0x7f18ec100750 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec10c100 con 0x7f18ec102300 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18f0eb2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec10c400 con 0x7f18ec101940 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 0x7f18ec199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 0x7f18ec199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38430/0 (socket says 172.21.15.7:38430) 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 -- 172.21.15.7:0/1354303712 learned_addr learned my addr 172.21.15.7:0/1354303712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18e3fff640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18ec101940 0x7f18ec19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ead9d640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 0x7f18ec1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 -- 172.21.15.7:0/1354303712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18ec101940 msgr2=0x7f18ec19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18ec101940 0x7f18ec19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 -- 172.21.15.7:0/1354303712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 msgr2=0x7f18ec1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 0x7f18ec1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ea59c640 1 -- 172.21.15.7:0/1354303712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec1a1900 con 0x7f18ec100750 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.831+0000 7f18ead9d640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 0x7f18ec1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.832+0000 7f18ea59c640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 0x7f18ec199d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e9c0 tx=0x7f18dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.832+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00cd80 con 0x7f18ec100750 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.832+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18dc0035f0 con 0x7f18ec100750 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.832+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc010690 con 0x7f18ec100750 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.832+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec1a1bf0 con 0x7f18ec100750 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.832+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a2110 con 0x7f18ec100750 2024-03-19T00:35:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.833+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec1a22e0 con 0x7f18ec100750 2024-03-19T00:35:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.834+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f18dc0037a0 con 0x7f18ec100750 2024-03-19T00:35:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.835+0000 7f18e37fe640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18d80a2880 0x7f18d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.835+0000 7f18e3fff640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18d80a2880 0x7f18d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.835+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f18dc0cecf0 con 0x7f18ec100750 2024-03-19T00:35:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.835+0000 7f18e3fff640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18d80a2880 0x7f18d80a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009230 tx=0x7f18d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:46.838+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f18dc09c190 con 0x7f18ec100750 2024-03-19T00:35:47.015 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:46 smithi204 systemd[1]: Starting Ceph mds.i for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.060+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18ec1a24b0 con 0x7f18ec100750 2024-03-19T00:35:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.060+0000 7f18e37fe640 1 -- 172.21.15.7:0/1354303712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f18dc09ba90 con 0x7f18ec100750 2024-03-19T00:35:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:47.062 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-19T00:35:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-03-19T00:35:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18d80a2880 msgr2=0x7f18d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18d80a2880 0x7f18d80a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009230 tx=0x7f18d4005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 msgr2=0x7f18ec199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 0x7f18ec199d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e9c0 tx=0x7f18dc00ee80 comp rx=0 tx=0).stop 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 shutdown_connections 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18d80a2880 0x7f18d80a4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18ec102300 0x7f18ec1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.062+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18ec101940 0x7f18ec19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.063+0000 7f18f0eb2640 1 --2- 172.21.15.7:0/1354303712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18ec100750 0x7f18ec199d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.063+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 >> 172.21.15.7:0/1354303712 conn(0x7f18ec0fbcc0 msgr2=0x7f18ec068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.063+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 shutdown_connections 2024-03-19T00:35:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:47.063+0000 7f18f0eb2640 1 -- 172.21.15.7:0/1354303712 wait complete. 2024-03-19T00:35:47.327 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 podman[97456]: 2024-03-19 00:35:46.976578602 +0000 UTC m=+0.017758981 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:47.327 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 podman[97456]: 2024-03-19 00:35:47.099545281 +0000 UTC m=+0.140725644 container create 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-19T00:35:47.328 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 podman[97456]: 2024-03-19 00:35:47.327667068 +0000 UTC m=+0.368847443 container init 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-19T00:35:47.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 podman[97456]: 2024-03-19 00:35:47.330358963 +0000 UTC m=+0.371539337 container start 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-19T00:35:47.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mds[97475]: -- 172.21.15.204:0/3675841059 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558088549a40 con 0x55808851d000 2024-03-19T00:35:47.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i[97471]: starting mds.i at 2024-03-19T00:35:47.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 bash[97456]: 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 2024-03-19T00:35:47.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:35:47 smithi204 systemd[1]: Started Ceph mds.i for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:48.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:48.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1354303712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:48.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:48.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:48.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1354303712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:48.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:48.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1354303712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:49.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:49.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 -- 172.21.15.7:0/2139366524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1007c0 msgr2=0x7f585c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 --2- 172.21.15.7:0/2139366524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1007c0 0x7f585c100be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 -- 172.21.15.7:0/2139366524 shutdown_connections 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 --2- 172.21.15.7:0/2139366524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 0x7f585c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 --2- 172.21.15.7:0/2139366524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585c1019b0 0x7f585c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 --2- 172.21.15.7:0/2139366524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1007c0 0x7f585c100be0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 -- 172.21.15.7:0/2139366524 >> 172.21.15.7:0/2139366524 conn(0x7f585c0fbd50 msgr2=0x7f585c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 -- 172.21.15.7:0/2139366524 shutdown_connections 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 -- 172.21.15.7:0/2139366524 wait complete. 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 Processor -- start 2024-03-19T00:35:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.194+0000 7f5864461640 1 -- start start 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585c1007c0 0x7f585c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 0x7f585c196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 0x7f585c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10c2c0 con 0x7f585c1019b0 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10c140 con 0x7f585c102370 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10c440 con 0x7f585c1007c0 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 0x7f585c196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 0x7f585c196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38446/0 (socket says 172.21.15.7:38446) 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 -- 172.21.15.7:0/700527330 learned_addr learned my addr 172.21.15.7:0/700527330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58621fd640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585c1007c0 0x7f585c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58629fe640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 0x7f585c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 -- 172.21.15.7:0/700527330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585c1007c0 msgr2=0x7f585c1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585c1007c0 0x7f585c1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 -- 172.21.15.7:0/700527330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 msgr2=0x7f585c19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 0x7f585c19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 -- 172.21.15.7:0/700527330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c19e5e0 con 0x7f585c1019b0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58629fe640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 0x7f585c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58619fc640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 0x7f585c196f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f584c009f40 tx=0x7f584c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c00cd80 con 0x7f585c1019b0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c069f60 con 0x7f585c1019b0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f584c0054d0 con 0x7f585c1019b0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.195+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0115d0 con 0x7f585c1019b0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.196+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c06a460 con 0x7f585c1019b0 2024-03-19T00:35:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.196+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c068af0 con 0x7f585c1019b0 2024-03-19T00:35:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.198+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f584c005000 con 0x7f585c1019b0 2024-03-19T00:35:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.198+0000 7f58537fe640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58480a2800 0x7f58480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.198+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f584c0d0d40 con 0x7f585c1019b0 2024-03-19T00:35:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.198+0000 7f58621fd640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58480a2800 0x7f58480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.199+0000 7f58621fd640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58480a2800 0x7f58480a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f585802c240 tx=0x7f585802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.202+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f584c0052f0 con 0x7f585c1019b0 2024-03-19T00:35:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:48 smithi007 ceph-mon[62854]: Deploying daemon mds.g on smithi007 2024-03-19T00:35:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:48 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] up:boot 2024-03-19T00:35:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:48 smithi007 ceph-mon[62854]: fsmap 9 up:standby 2024-03-19T00:35:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T00:35:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:48 smithi204 ceph-mon[66572]: Deploying daemon mds.g on smithi007 2024-03-19T00:35:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:48 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] up:boot 2024-03-19T00:35:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:48 smithi204 ceph-mon[66572]: fsmap 9 up:standby 2024-03-19T00:35:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T00:35:49.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:48 smithi044 ceph-mon[66366]: Deploying daemon mds.g on smithi007 2024-03-19T00:35:49.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:48 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] up:boot 2024-03-19T00:35:49.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:48 smithi044 ceph-mon[66366]: fsmap 9 up:standby 2024-03-19T00:35:49.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T00:35:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.444+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f585c107620 con 0x7f585c1019b0 2024-03-19T00:35:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.445+0000 7f58537fe640 1 -- 172.21.15.7:0/700527330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7995 (secure 0 0 0) 0x7f585c107620 con 0x7f585c1019b0 2024-03-19T00:35:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:49.447 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-19T00:35:49.447 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-03-19T00:35:49.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58480a2800 msgr2=0x7f58480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:49.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58480a2800 0x7f58480a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f585802c240 tx=0x7f585802c7e0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 msgr2=0x7f585c196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 0x7f585c196f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f584c009f40 tx=0x7f584c00ee80 comp rx=0 tx=0).stop 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 shutdown_connections 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58480a2800 0x7f58480a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102370 0x7f585c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c1019b0 0x7f585c196f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 --2- 172.21.15.7:0/700527330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585c1007c0 0x7f585c1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 >> 172.21.15.7:0/700527330 conn(0x7f585c0fbd50 msgr2=0x7f585c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 shutdown_connections 2024-03-19T00:35:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:49.447+0000 7f5864461640 1 -- 172.21.15.7:0/700527330 wait complete. 2024-03-19T00:35:49.897 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:49 smithi007 systemd[1]: Starting Ceph mds.g for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:50.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:49 smithi007 ceph-mon[62854]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:50.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:49 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/700527330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:50.201 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:49 smithi007 podman[101054]: 2024-03-19 00:35:49.809459984 +0000 UTC m=+0.018839341 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:50.201 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:49 smithi007 podman[101054]: 2024-03-19 00:35:49.912156531 +0000 UTC m=+0.121535916 container create 3e4143e76ce259631282ecaf9f2995752309fcea469cb88221440faa9bd49c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:35:50.201 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:50 smithi007 podman[101054]: 2024-03-19 00:35:50.198876123 +0000 UTC m=+0.408255499 container init 3e4143e76ce259631282ecaf9f2995752309fcea469cb88221440faa9bd49c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:35:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:49 smithi204 ceph-mon[66572]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:49 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/700527330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:50.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:49 smithi044 ceph-mon[66366]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:50.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:49 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/700527330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:50.465 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:50 smithi007 podman[101054]: 2024-03-19 00:35:50.202610838 +0000 UTC m=+0.411990194 container start 3e4143e76ce259631282ecaf9f2995752309fcea469cb88221440faa9bd49c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-19T00:35:50.465 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:50 smithi007 ceph-mds[101074]: -- 172.21.15.7:0/2224916762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574e1121a40 con 0x5574e10f4c00 2024-03-19T00:35:50.465 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g[101069]: starting mds.g at 2024-03-19T00:35:50.465 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:50 smithi007 bash[101054]: 3e4143e76ce259631282ecaf9f2995752309fcea469cb88221440faa9bd49c7f 2024-03-19T00:35:50.465 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:35:50 smithi007 systemd[1]: Started Ceph mds.g for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:51.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 -- 172.21.15.7:0/1534315884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 msgr2=0x7fc23c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 --2- 172.21.15.7:0/1534315884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c101dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc228009900 tx=0x7fc22802bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 -- 172.21.15.7:0/1534315884 shutdown_connections 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 --2- 172.21.15.7:0/1534315884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 0x7fc23c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 --2- 172.21.15.7:0/1534315884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c101dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 --2- 172.21.15.7:0/1534315884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 0x7fc23c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 -- 172.21.15.7:0/1534315884 >> 172.21.15.7:0/1534315884 conn(0x7fc23c0fbcc0 msgr2=0x7fc23c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 -- 172.21.15.7:0/1534315884 shutdown_connections 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 -- 172.21.15.7:0/1534315884 wait complete. 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 Processor -- start 2024-03-19T00:35:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.567+0000 7fc243137640 1 -- start start 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 0x7fc23c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 0x7fc23c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10c2f0 con 0x7fc23c101940 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10c170 con 0x7fc23c100750 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10c470 con 0x7fc23c102300 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38478/0 (socket says 172.21.15.7:38478) 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 -- 172.21.15.7:0/3521200732 learned_addr learned my addr 172.21.15.7:0/3521200732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc2416d4640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 0x7fc23c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc240ed3640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 0x7fc23c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 -- 172.21.15.7:0/3521200732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 msgr2=0x7fc23c19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 0x7fc23c19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 -- 172.21.15.7:0/3521200732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 msgr2=0x7fc23c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 0x7fc23c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 -- 172.21.15.7:0/3521200732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c19e580 con 0x7fc23c101940 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc2416d4640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 0x7fc23c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc240ed3640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 0x7fc23c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc23bfff640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c196ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc228036040 tx=0x7fc2280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228043020 con 0x7fc23c101940 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c19e810 con 0x7fc23c101940 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2280054c0 con 0x7fc23c101940 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.568+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228003d70 con 0x7fc23c101940 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.569+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c19ee30 con 0x7fc23c101940 2024-03-19T00:35:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.570+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c0686a0 con 0x7fc23c101940 2024-03-19T00:35:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.571+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fc228046020 con 0x7fc23c101940 2024-03-19T00:35:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.571+0000 7fc239ffb640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc22c0a2800 0x7fc22c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.571+0000 7fc240ed3640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc22c0a2800 0x7fc22c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.571+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc2280f0bd0 con 0x7fc23c101940 2024-03-19T00:35:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.572+0000 7fc240ed3640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc22c0a2800 0x7fc22c0a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc230006fd0 tx=0x7fc230008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:51.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.576+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc22804d020 con 0x7fc23c101940 2024-03-19T00:35:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: Deploying daemon mds.a on smithi007 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.7:6838/1114335301,v1:172.21.15.7:6839/1114335301] up:boot 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: fsmap 10 up:standby 2024-03-19T00:35:51.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T00:35:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:51.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:51.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:51.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: Deploying daemon mds.a on smithi007 2024-03-19T00:35:51.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.7:6838/1114335301,v1:172.21.15.7:6839/1114335301] up:boot 2024-03-19T00:35:51.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: fsmap 10 up:standby 2024-03-19T00:35:51.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T00:35:51.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.804+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc23c02d1f0 con 0x7fc23c101940 2024-03-19T00:35:51.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.804+0000 7fc239ffb640 1 -- 172.21.15.7:0/3521200732 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8818 (secure 0 0 0) 0x7fc2280be110 con 0x7fc23c101940 2024-03-19T00:35:51.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:51.805 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-19T00:35:51.805 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 11 2024-03-19T00:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc22c0a2800 msgr2=0x7fc22c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc22c0a2800 0x7fc22c0a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc230006fd0 tx=0x7fc230008040 comp rx=0 tx=0).stop 2024-03-19T00:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 msgr2=0x7fc23c196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c196ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc228036040 tx=0x7fc2280042f0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 shutdown_connections 2024-03-19T00:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc22c0a2800 0x7fc22c0a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc23c102300 0x7fc23c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c101940 0x7fc23c196ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 --2- 172.21.15.7:0/3521200732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23c100750 0x7fc23c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 >> 172.21.15.7:0/3521200732 conn(0x7fc23c0fbcc0 msgr2=0x7fc23c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 shutdown_connections 2024-03-19T00:35:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:51.807+0000 7fc243137640 1 -- 172.21.15.7:0/3521200732 wait complete. 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: Deploying daemon mds.a on smithi007 2024-03-19T00:35:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.7:6838/1114335301,v1:172.21.15.7:6839/1114335301] up:boot 2024-03-19T00:35:51.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: fsmap 10 up:standby 2024-03-19T00:35:51.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T00:35:52.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:52 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3521200732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:52.715 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:52 smithi007 systemd[1]: Starting Ceph mds.a for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:52.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:52 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3521200732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:52.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:52 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3521200732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:53.082 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:52 smithi007 podman[101415]: 2024-03-19 00:35:52.708519001 +0000 UTC m=+0.019075506 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:53.082 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:52 smithi007 podman[101415]: 2024-03-19 00:35:52.820328115 +0000 UTC m=+0.130884595 container create 0b271c0ad50e6970e5eed27cb8d357abcf66bcf2b5e320e78bded2b178db1ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T00:35:53.082 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:53 smithi007 podman[101415]: 2024-03-19 00:35:53.081618664 +0000 UTC m=+0.392175158 container init 0b271c0ad50e6970e5eed27cb8d357abcf66bcf2b5e320e78bded2b178db1ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-19T00:35:53.389 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:53 smithi007 podman[101415]: 2024-03-19 00:35:53.084268186 +0000 UTC m=+0.394824667 container start 0b271c0ad50e6970e5eed27cb8d357abcf66bcf2b5e320e78bded2b178db1ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:35:53.389 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mds[101434]: -- 172.21.15.7:0/3462101749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cb71dda40 con 0x564cb71b0800 2024-03-19T00:35:53.389 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a[101430]: starting mds.a at 2024-03-19T00:35:53.389 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:53 smithi007 bash[101415]: 0b271c0ad50e6970e5eed27cb8d357abcf66bcf2b5e320e78bded2b178db1ede 2024-03-19T00:35:53.389 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:35:53 smithi007 systemd[1]: Started Ceph mds.a for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:53.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T00:35:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 -- 172.21.15.7:0/3486204351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 msgr2=0x7f6e0c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 --2- 172.21.15.7:0/3486204351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c1099f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009ca0 tx=0x7f6e0402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:35:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 -- 172.21.15.7:0/3486204351 shutdown_connections 2024-03-19T00:35:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 --2- 172.21.15.7:0/3486204351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c1099f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 --2- 172.21.15.7:0/3486204351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e0c104cd0 0x7f6e0c1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:53.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 --2- 172.21.15.7:0/3486204351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e0c1023a0 0x7f6e0c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 -- 172.21.15.7:0/3486204351 >> 172.21.15.7:0/3486204351 conn(0x7f6e0c0fbd30 msgr2=0x7f6e0c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 -- 172.21.15.7:0/3486204351 shutdown_connections 2024-03-19T00:35:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.930+0000 7f6e141a9640 1 -- 172.21.15.7:0/3486204351 wait complete. 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 Processor -- start 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 -- start start 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e0c1023a0 0x7f6e0c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e0c104cd0 0x7f6e0c100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c10d380 con 0x7f6e0c107600 2024-03-19T00:35:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c10d200 con 0x7f6e0c1023a0 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e141a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c10d500 con 0x7f6e0c104cd0 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38524/0 (socket says 172.21.15.7:38524) 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 -- 172.21.15.7:0/733257819 learned_addr learned my addr 172.21.15.7:0/733257819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e11f45640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e0c1023a0 0x7f6e0c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e11744640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e0c104cd0 0x7f6e0c100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 -- 172.21.15.7:0/733257819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e0c104cd0 msgr2=0x7f6e0c100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e0c104cd0 0x7f6e0c100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 -- 172.21.15.7:0/733257819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e0c1023a0 msgr2=0x7f6e0c101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e0c1023a0 0x7f6e0c101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.931+0000 7f6e12746640 1 -- 172.21.15.7:0/733257819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c101120 con 0x7f6e0c107600 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.932+0000 7f6e12746640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c100890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e0402c700 tx=0x7f6e040328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.932+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04041020 con 0x7f6e0c107600 2024-03-19T00:35:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.932+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e04003e20 con 0x7f6e0c107600 2024-03-19T00:35:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.932+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c1a1ac0 con 0x7f6e0c107600 2024-03-19T00:35:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.932+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04039040 con 0x7f6e0c107600 2024-03-19T00:35:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.932+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c1a2060 con 0x7f6e0c107600 2024-03-19T00:35:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.933+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c0fd820 con 0x7f6e0c107600 2024-03-19T00:35:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.934+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f6e04044020 con 0x7f6e0c107600 2024-03-19T00:35:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.934+0000 7f6e02ffd640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6df80a2880 0x7f6df80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.935+0000 7f6e11f45640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6df80a2880 0x7f6df80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.935+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6e040f0bf0 con 0x7f6e0c107600 2024-03-19T00:35:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.935+0000 7f6e11f45640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6df80a2880 0x7f6df80a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009bd0 tx=0x7f6e08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:53.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:53.938+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e0404b020 con 0x7f6e0c107600 2024-03-19T00:35:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.164+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e0c02d1f0 con 0x7f6e0c107600 2024-03-19T00:35:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.165+0000 7f6e02ffd640 1 -- 172.21.15.7:0/733257819 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f6e040be090 con 0x7f6e0c107600 2024-03-19T00:35:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:54.166 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-19T00:35:54.166 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-03-19T00:35:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6df80a2880 msgr2=0x7f6df80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6df80a2880 0x7f6df80a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009bd0 tx=0x7f6e08005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 msgr2=0x7f6e0c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c100890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e0402c700 tx=0x7f6e040328a0 comp rx=0 tx=0).stop 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 shutdown_connections 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6df80a2880 0x7f6df80a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e0c107600 0x7f6e0c100890 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e0c104cd0 0x7f6e0c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 --2- 172.21.15.7:0/733257819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e0c1023a0 0x7f6e0c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 >> 172.21.15.7:0/733257819 conn(0x7f6e0c0fbd30 msgr2=0x7f6e0c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 shutdown_connections 2024-03-19T00:35:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:54.167+0000 7f6e141a9640 1 -- 172.21.15.7:0/733257819 wait complete. 2024-03-19T00:35:54.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:54 smithi007 ceph-mon[62854]: Deploying daemon mds.b on smithi044 2024-03-19T00:35:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:54 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.7:6840/3178899604,v1:172.21.15.7:6841/3178899604] up:boot 2024-03-19T00:35:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:54 smithi007 ceph-mon[62854]: fsmap 11 up:standby 2024-03-19T00:35:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:54 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T00:35:54.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:54 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/733257819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:54 smithi044 ceph-mon[66366]: Deploying daemon mds.b on smithi044 2024-03-19T00:35:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:54 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.7:6840/3178899604,v1:172.21.15.7:6841/3178899604] up:boot 2024-03-19T00:35:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:54 smithi044 ceph-mon[66366]: fsmap 11 up:standby 2024-03-19T00:35:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:54 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T00:35:54.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:54 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/733257819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:54.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:54 smithi204 ceph-mon[66572]: Deploying daemon mds.b on smithi044 2024-03-19T00:35:54.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:54 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.7:6840/3178899604,v1:172.21.15.7:6841/3178899604] up:boot 2024-03-19T00:35:54.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:54 smithi204 ceph-mon[66572]: fsmap 11 up:standby 2024-03-19T00:35:54.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:54 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T00:35:54.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:54 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/733257819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:55 smithi044 ceph-mon[66366]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:55.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:55 smithi044 systemd[1]: Starting Ceph mds.b for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:35:55.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:55 smithi007 ceph-mon[62854]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:55.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:55 smithi204 ceph-mon[66572]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:56.005 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:55 smithi044 podman[96542]: 2024-03-19 00:35:55.638819572 +0000 UTC m=+0.018602508 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:35:56.005 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:55 smithi044 podman[96542]: 2024-03-19 00:35:55.760354532 +0000 UTC m=+0.140137467 container create 0e16aa255c9a720acd9f14537483eab50d66a49cb18fbf1ca3d5aa3fc33f0d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-19T00:35:56.005 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:56 smithi044 podman[96542]: 2024-03-19 00:35:56.005062122 +0000 UTC m=+0.384845058 container init 0e16aa255c9a720acd9f14537483eab50d66a49cb18fbf1ca3d5aa3fc33f0d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-19T00:35:56.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 -- 172.21.15.7:0/2042811689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1053e0 msgr2=0x7f0b6c105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 --2- 172.21.15.7:0/2042811689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1053e0 0x7f0b6c105860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009ca0 tx=0x7f0b6402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 -- 172.21.15.7:0/2042811689 shutdown_connections 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 --2- 172.21.15.7:0/2042811689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1053e0 0x7f0b6c105860 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 --2- 172.21.15.7:0/2042811689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c102a30 0x7f0b6c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 --2- 172.21.15.7:0/2042811689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.295+0000 7f0b738e8640 1 -- 172.21.15.7:0/2042811689 >> 172.21.15.7:0/2042811689 conn(0x7f0b6c0fbcc0 msgr2=0x7f0b6c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 -- 172.21.15.7:0/2042811689 shutdown_connections 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 -- 172.21.15.7:0/2042811689 wait complete. 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 Processor -- start 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 -- start start 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b6c102a30 0x7f0b6c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c1053e0 0x7f0b6c1a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c113950 con 0x7f0b6c1000c0 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c1137d0 con 0x7f0b6c1053e0 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b738e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c113ad0 con 0x7f0b6c102a30 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b71684640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b71684640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34166/0 (socket says 172.21.15.7:34166) 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.296+0000 7f0b71684640 1 -- 172.21.15.7:0/1921078309 learned_addr learned my addr 172.21.15.7:0/1921078309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71e85640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c1053e0 0x7f0b6c1a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b70e83640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b6c102a30 0x7f0b6c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71684640 1 -- 172.21.15.7:0/1921078309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b6c102a30 msgr2=0x7f0b6c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71684640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b6c102a30 0x7f0b6c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71684640 1 -- 172.21.15.7:0/1921078309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c1053e0 msgr2=0x7f0b6c1a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71684640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c1053e0 0x7f0b6c1a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71684640 1 -- 172.21.15.7:0/1921078309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c1a5b40 con 0x7f0b6c1000c0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b71684640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c19df20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c920 tx=0x7f0b6800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007c70 con 0x7f0b6c1000c0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b68007e10 con 0x7f0b6c1000c0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c1a5e30 con 0x7f0b6c1000c0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68018820 con 0x7f0b6c1000c0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.297+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c1a62f0 con 0x7f0b6c1000c0 2024-03-19T00:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.298+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c0fd7b0 con 0x7f0b6c1000c0 2024-03-19T00:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.299+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f0b680189c0 con 0x7f0b6c1000c0 2024-03-19T00:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.300+0000 7f0b627fc640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b580a2880 0x7f0b580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.300+0000 7f0b70e83640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b580a2880 0x7f0b580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.300+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0b680ce690 con 0x7f0b6c1000c0 2024-03-19T00:35:56.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.300+0000 7f0b70e83640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b580a2880 0x7f0b580a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c19f430 tx=0x7f0b5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:56.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.305+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0b68014030 con 0x7f0b6c1000c0 2024-03-19T00:35:56.356 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:56 smithi044 podman[96542]: 2024-03-19 00:35:56.007827505 +0000 UTC m=+0.387610450 container start 0e16aa255c9a720acd9f14537483eab50d66a49cb18fbf1ca3d5aa3fc33f0d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-19T00:35:56.356 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:56 smithi044 ceph-mds[96561]: -- 172.21.15.44:0/3617532204 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eaee395a40 con 0x55eaee368800 2024-03-19T00:35:56.356 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:56 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: starting mds.b at 2024-03-19T00:35:56.356 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:56 smithi044 bash[96542]: 0e16aa255c9a720acd9f14537483eab50d66a49cb18fbf1ca3d5aa3fc33f0d2e 2024-03-19T00:35:56.356 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:35:56 smithi044 systemd[1]: Started Ceph mds.b for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:35:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.570+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b6c02d610 con 0x7f0b6c1000c0 2024-03-19T00:35:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.571+0000 7f0b627fc640 1 -- 172.21.15.7:0/1921078309 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7f0b6809bbd0 con 0x7f0b6c1000c0 2024-03-19T00:35:56.573 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:56.573 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-19T00:35:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b580a2880 msgr2=0x7f0b580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b580a2880 0x7f0b580a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c19f430 tx=0x7f0b5c005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 msgr2=0x7f0b6c19df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c19df20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c920 tx=0x7f0b6800cde0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 shutdown_connections 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b580a2880 0x7f0b580a4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b6c1053e0 0x7f0b6c1a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b6c102a30 0x7f0b6c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 --2- 172.21.15.7:0/1921078309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c1000c0 0x7f0b6c19df20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 >> 172.21.15.7:0/1921078309 conn(0x7f0b6c0fbcc0 msgr2=0x7f0b6c100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 shutdown_connections 2024-03-19T00:35:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.573+0000 7f0b738e8640 1 -- 172.21.15.7:0/1921078309 wait complete. 2024-03-19T00:35:56.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/533424054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8102300 msgr2=0x7ff8c8108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/533424054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8102300 0x7ff8c8108b90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009ca0 tx=0x7ff8c002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/533424054 shutdown_connections 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/533424054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8102300 0x7ff8c8108b90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/533424054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8101940 0x7ff8c8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/533424054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c8100750 0x7ff8c8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/533424054 >> 172.21.15.7:0/533424054 conn(0x7ff8c80fbcc0 msgr2=0x7ff8c80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:56.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/533424054 shutdown_connections 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.688+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/533424054 wait complete. 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 Processor -- start 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 -- start start 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 0x7ff8c8069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c8101940 0x7ff8c819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8102300 0x7ff8c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810c280 con 0x7ff8c8100750 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810c100 con 0x7ff8c8102300 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8cc9d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810c400 con 0x7ff8c8101940 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 0x7ff8c8069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 0x7ff8c8069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34182/0 (socket says 172.21.15.7:34182) 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 -- 172.21.15.7:0/1359781461 learned_addr learned my addr 172.21.15.7:0/1359781461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c6d9d640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8102300 0x7ff8c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8bffff640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c8101940 0x7ff8c819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 -- 172.21.15.7:0/1359781461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c8101940 msgr2=0x7ff8c819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c8101940 0x7ff8c819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 -- 172.21.15.7:0/1359781461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8102300 msgr2=0x7ff8c819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8102300 0x7ff8c819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.689+0000 7ff8c659c640 1 -- 172.21.15.7:0/1359781461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c819f620 con 0x7ff8c8100750 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.690+0000 7ff8c659c640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 0x7ff8c8069680 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9c0 tx=0x7ff8b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.690+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800cd80 con 0x7ff8c8100750 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.690+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8b80035f0 con 0x7ff8c8100750 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.690+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c819f910 con 0x7ff8c8100750 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.690+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8010690 con 0x7ff8c8100750 2024-03-19T00:35:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.690+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c819fe10 con 0x7ff8c8100750 2024-03-19T00:35:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.691+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c8107a70 con 0x7ff8c8100750 2024-03-19T00:35:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.692+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7ff8b80037a0 con 0x7ff8c8100750 2024-03-19T00:35:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.692+0000 7ff8bf7fe640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8b40a2800 0x7ff8b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.692+0000 7ff8bffff640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8b40a2800 0x7ff8b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.692+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff8b80ceac0 con 0x7ff8c8100750 2024-03-19T00:35:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.693+0000 7ff8bffff640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8b40a2800 0x7ff8b40a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80693f0 tx=0x7ff8b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:56.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.697+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff8b809c000 con 0x7ff8c8100750 2024-03-19T00:35:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.892+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8c802d430 con 0x7ff8c8100750 2024-03-19T00:35:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.894+0000 7ff8bf7fe640 1 -- 172.21.15.7:0/1359781461 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18299 (secure 0 0 0) 0x7ff8b809b900 con 0x7ff8c8100750 2024-03-19T00:35:56.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:35:11.135052+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8b40a2800 msgr2=0x7ff8b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8b40a2800 0x7ff8b40a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80693f0 tx=0x7ff8b0005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 msgr2=0x7ff8c8069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 0x7ff8c8069680 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9c0 tx=0x7ff8b800ee80 comp rx=0 tx=0).stop 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 shutdown_connections 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8b40a2800 0x7ff8b40a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c8102300 0x7ff8c819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c8101940 0x7ff8c819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 --2- 172.21.15.7:0/1359781461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c8100750 0x7ff8c8069680 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.896+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 >> 172.21.15.7:0/1359781461 conn(0x7ff8c80fbcc0 msgr2=0x7ff8c8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.897+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 shutdown_connections 2024-03-19T00:35:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:56.897+0000 7ff8cc9d4640 1 -- 172.21.15.7:0/1359781461 wait complete. 2024-03-19T00:35:56.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T00:35:56.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 -- 172.21.15.7:0/2667495848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120100740 msgr2=0x7f9120100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 --2- 172.21.15.7:0/2667495848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120100740 0x7f9120100b60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9110009900 tx=0x7f911002bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 -- 172.21.15.7:0/2667495848 shutdown_connections 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 --2- 172.21.15.7:0/2667495848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91201022f0 0x7f9120108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 --2- 172.21.15.7:0/2667495848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9120101930 0x7f9120101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 --2- 172.21.15.7:0/2667495848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120100740 0x7f9120100b60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 -- 172.21.15.7:0/2667495848 >> 172.21.15.7:0/2667495848 conn(0x7f91200fbcb0 msgr2=0x7f91200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 -- 172.21.15.7:0/2667495848 shutdown_connections 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.025+0000 7f91268bf640 1 -- 172.21.15.7:0/2667495848 wait complete. 2024-03-19T00:35:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.026+0000 7f91268bf640 1 Processor -- start 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.026+0000 7f91268bf640 1 -- start start 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9120100740 0x7f9120197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 0x7f9120198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91201022f0 0x7f912019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f912010d3d0 con 0x7f9120101930 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f912010d250 con 0x7f9120100740 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f912010d550 con 0x7f91201022f0 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 0x7f9120198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 0x7f9120198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34198/0 (socket says 172.21.15.7:34198) 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 -- 172.21.15.7:0/2942685761 learned_addr learned my addr 172.21.15.7:0/2942685761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911ffff640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9120100740 0x7f9120197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f9124e5c640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91201022f0 0x7f912019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 -- 172.21.15.7:0/2942685761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91201022f0 msgr2=0x7f912019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91201022f0 0x7f912019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 -- 172.21.15.7:0/2942685761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9120100740 msgr2=0x7f9120197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9120100740 0x7f9120197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 -- 172.21.15.7:0/2942685761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912019f740 con 0x7f9120101930 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911f7fe640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 0x7f9120198060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f911400d950 tx=0x7f911400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114017070 con 0x7f9120101930 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9114009c30 con 0x7f9120101930 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912019fa30 con 0x7f9120101930 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.027+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911400f620 con 0x7f9120101930 2024-03-19T00:35:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.028+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912019ff80 con 0x7f9120101930 2024-03-19T00:35:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.029+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9120107b30 con 0x7f9120101930 2024-03-19T00:35:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.030+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f91140109a0 con 0x7f9120101930 2024-03-19T00:35:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.030+0000 7f911d7fa640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f91080a2800 0x7f91080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.030+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f91140ce160 con 0x7f9120101930 2024-03-19T00:35:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.031+0000 7f911ffff640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f91080a2800 0x7f91080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.031+0000 7f911ffff640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f91080a2800 0x7f91080a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9110005ab0 tx=0x7f9110005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.035+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9114009de0 con 0x7f9120101930 2024-03-19T00:35:57.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.223+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f912002d410 con 0x7f9120101930 2024-03-19T00:35:57.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.223+0000 7f911d7fa640 1 -- 172.21.15.7:0/2942685761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f911409b050 con 0x7f9120101930 2024-03-19T00:35:57.225 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:35:57.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f91080a2800 msgr2=0x7f91080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f91080a2800 0x7f91080a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9110005ab0 tx=0x7f9110005fd0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 msgr2=0x7f9120198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 0x7f9120198060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f911400d950 tx=0x7f911400de10 comp rx=0 tx=0).stop 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 shutdown_connections 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f91080a2800 0x7f91080a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91201022f0 0x7f912019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9120101930 0x7f9120198060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 --2- 172.21.15.7:0/2942685761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9120100740 0x7f9120197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 >> 172.21.15.7:0/2942685761 conn(0x7f91200fbcb0 msgr2=0x7f9120104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 shutdown_connections 2024-03-19T00:35:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.225+0000 7f91268bf640 1 -- 172.21.15.7:0/2942685761 wait complete. 2024-03-19T00:35:57.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-19T00:35:57.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 -- 172.21.15.7:0/1123501289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 msgr2=0x7f5898102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 --2- 172.21.15.7:0/1123501289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898102ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5894009900 tx=0x7f589402bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 -- 172.21.15.7:0/1123501289 shutdown_connections 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 --2- 172.21.15.7:0/1123501289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898103420 0x7f5898109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 --2- 172.21.15.7:0/1123501289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898102ee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 --2- 172.21.15.7:0/1123501289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 0x7f5898101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.341+0000 7f589f80b640 1 -- 172.21.15.7:0/1123501289 >> 172.21.15.7:0/1123501289 conn(0x7f58980fcde0 msgr2=0x7f58980ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 -- 172.21.15.7:0/1123501289 shutdown_connections 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 -- 172.21.15.7:0/1123501289 wait complete. 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 Processor -- start 2024-03-19T00:35:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 -- start start 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 0x7f5898197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898103420 0x7f589819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f589810d3a0 con 0x7f5898102a60 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f589810d220 con 0x7f5898103420 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589f80b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f589810d520 con 0x7f5898101870 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.342+0000 7f589cda6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589d5a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 0x7f5898197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34220/0 (socket says 172.21.15.7:34220) 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 -- 172.21.15.7:0/1515639895 learned_addr learned my addr 172.21.15.7:0/1515639895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589dda8640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898103420 0x7f589819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 -- 172.21.15.7:0/1515639895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 msgr2=0x7f5898197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 0x7f5898197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 -- 172.21.15.7:0/1515639895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898103420 msgr2=0x7f589819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898103420 0x7f589819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 -- 172.21.15.7:0/1515639895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589819f690 con 0x7f5898102a60 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589d5a7640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 0x7f5898197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589cda6640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898197fb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5894006010 tx=0x7f5894004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894043020 con 0x7f5898102a60 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589819f920 con 0x7f5898102a60 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5894004440 con 0x7f5898102a60 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894003bf0 con 0x7f5898102a60 2024-03-19T00:35:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.343+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589819fe70 con 0x7f5898102a60 2024-03-19T00:35:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.344+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5898108c60 con 0x7f5898102a60 2024-03-19T00:35:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.345+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f5894046020 con 0x7f5898102a60 2024-03-19T00:35:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.346+0000 7f588a7fc640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58840a2800 0x7f58840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.346+0000 7f589d5a7640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58840a2800 0x7f58840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.346+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f58940f08d0 con 0x7f5898102a60 2024-03-19T00:35:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.346+0000 7f589d5a7640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58840a2800 0x7f58840a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58981028c0 tx=0x7f588c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.350+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f589404d020 con 0x7f5898102a60 2024-03-19T00:35:57.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] up:boot 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: fsmap 12 up:standby 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1921078309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1359781461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] up:boot 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: fsmap 12 up:standby 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T00:35:57.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1921078309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1359781461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.549+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f589802d3d0 con 0x7f5898102a60 2024-03-19T00:35:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.550+0000 7f588a7fc640 1 -- 172.21.15.7:0/1515639895 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f58940bde10 con 0x7f5898102a60 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:35:57.552 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58840a2800 msgr2=0x7f58840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58840a2800 0x7f58840a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58981028c0 tx=0x7f588c005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 msgr2=0x7f5898197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898197fb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5894006010 tx=0x7f5894004040 comp rx=0 tx=0).stop 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 shutdown_connections 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58840a2800 0x7f58840a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5898103420 0x7f589819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5898102a60 0x7f5898197fb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 --2- 172.21.15.7:0/1515639895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5898101870 0x7f5898197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 >> 172.21.15.7:0/1515639895 conn(0x7f58980fcde0 msgr2=0x7f5898105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 shutdown_connections 2024-03-19T00:35:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.552+0000 7f589f80b640 1 -- 172.21.15.7:0/1515639895 wait complete. 2024-03-19T00:35:57.564 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-19T00:35:57.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-19T00:35:57.567 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:57.567 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.567 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.567 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] up:boot 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: fsmap 12 up:standby 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1921078309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.568 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1359781461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:35:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 -- 172.21.15.7:0/422944633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 msgr2=0x7f95f4101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 --2- 172.21.15.7:0/422944633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4101db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009900 tx=0x7f95e402bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 -- 172.21.15.7:0/422944633 shutdown_connections 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 --2- 172.21.15.7:0/422944633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f41022f0 0x7f95f4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 --2- 172.21.15.7:0/422944633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4101db0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 --2- 172.21.15.7:0/422944633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95f4100740 0x7f95f4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 -- 172.21.15.7:0/422944633 >> 172.21.15.7:0/422944633 conn(0x7f95f40fbcd0 msgr2=0x7f95f40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 -- 172.21.15.7:0/422944633 shutdown_connections 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.669+0000 7f95fae7e640 1 -- 172.21.15.7:0/422944633 wait complete. 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 Processor -- start 2024-03-19T00:35:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 -- start start 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95f4100740 0x7f95f4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f41022f0 0x7f95f419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c270 con 0x7f95f4101930 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c0f0 con 0x7f95f41022f0 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95fae7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c3f0 con 0x7f95f4100740 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34234/0 (socket says 172.21.15.7:34234) 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 -- 172.21.15.7:0/4006680287 learned_addr learned my addr 172.21.15.7:0/4006680287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95f8c1a640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95f4100740 0x7f95f4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95f941b640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f41022f0 0x7f95f419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 -- 172.21.15.7:0/4006680287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95f4100740 msgr2=0x7f95f4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95f4100740 0x7f95f4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 -- 172.21.15.7:0/4006680287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f41022f0 msgr2=0x7f95f419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f41022f0 0x7f95f419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.670+0000 7f95ebfff640 1 -- 172.21.15.7:0/4006680287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f419f6c0 con 0x7f95f4101930 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.671+0000 7f95ebfff640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4198030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95e4006010 tx=0x7f95e4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.671+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4043020 con 0x7f95f4101930 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.671+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95e40043d0 con 0x7f95f4101930 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.671+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f419f950 con 0x7f95f4101930 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.671+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4003b80 con 0x7f95f4101930 2024-03-19T00:35:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.671+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f419fef0 con 0x7f95f4101930 2024-03-19T00:35:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.672+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f41a00c0 con 0x7f95f4101930 2024-03-19T00:35:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.673+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f95e4046020 con 0x7f95f4101930 2024-03-19T00:35:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.673+0000 7f95e9ffb640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95dc0a2880 0x7f95dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.674+0000 7f95f8c1a640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95dc0a2880 0x7f95dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.674+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f95e40f0df0 con 0x7f95f4101930 2024-03-19T00:35:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.674+0000 7f95f8c1a640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95dc0a2880 0x7f95dc0a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95f4101790 tx=0x7f95f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.677+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f95e404d020 con 0x7f95f4101930 2024-03-19T00:35:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:57.873+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f95f41a0290 con 0x7f95f4101930 2024-03-19T00:35:58.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.181+0000 7f95e9ffb640 1 -- 172.21.15.7:0/4006680287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f95e40f0020 con 0x7f95f4101930 2024-03-19T00:35:58.183 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-03-19T00:35:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95dc0a2880 msgr2=0x7f95dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95dc0a2880 0x7f95dc0a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95f4101790 tx=0x7f95f0005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 msgr2=0x7f95f4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4198030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95e4006010 tx=0x7f95e4005e40 comp rx=0 tx=0).stop 2024-03-19T00:35:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 shutdown_connections 2024-03-19T00:35:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95dc0a2880 0x7f95dc0a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95f41022f0 0x7f95f419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95f4101930 0x7f95f4198030 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 --2- 172.21.15.7:0/4006680287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95f4100740 0x7f95f4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 >> 172.21.15.7:0/4006680287 conn(0x7f95f40fbcd0 msgr2=0x7f95f4069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 shutdown_connections 2024-03-19T00:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.184+0000 7f95fae7e640 1 -- 172.21.15.7:0/4006680287 wait complete. 2024-03-19T00:35:58.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-19T00:35:58.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 -- 172.21.15.7:0/115833021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 msgr2=0x7f7270101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 --2- 172.21.15.7:0/115833021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f7270101870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7260009ca0 tx=0x7f726002c140 comp rx=0 tx=0).stop 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 -- 172.21.15.7:0/115833021 shutdown_connections 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 --2- 172.21.15.7:0/115833021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f7270101870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 --2- 172.21.15.7:0/115833021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7270100a50 0x7f7270100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 --2- 172.21.15.7:0/115833021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72701000f0 0x7f7270100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 -- 172.21.15.7:0/115833021 >> 172.21.15.7:0/115833021 conn(0x7f72700fbcb0 msgr2=0x7f72700fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 -- 172.21.15.7:0/115833021 shutdown_connections 2024-03-19T00:35:58.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.308+0000 7f72748bf640 1 -- 172.21.15.7:0/115833021 wait complete. 2024-03-19T00:35:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 Processor -- start 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 -- start start 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72701000f0 0x7f7270199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7270100a50 0x7f727019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f72701a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f727010f550 con 0x7f72701013f0 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f727010f3d0 con 0x7f72701000f0 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f72748bf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f727010f6d0 con 0x7f7270100a50 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f726ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f72701a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f726ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f72701a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34264/0 (socket says 172.21.15.7:34264) 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.309+0000 7f726ed9d640 1 -- 172.21.15.7:0/710086648 learned_addr learned my addr 172.21.15.7:0/710086648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726dd9b640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7270100a50 0x7f727019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726e59c640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72701000f0 0x7f7270199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726ed9d640 1 -- 172.21.15.7:0/710086648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7270100a50 msgr2=0x7f727019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726ed9d640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7270100a50 0x7f727019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726ed9d640 1 -- 172.21.15.7:0/710086648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72701000f0 msgr2=0x7f7270199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726ed9d640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72701000f0 0x7f7270199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726ed9d640 1 -- 172.21.15.7:0/710086648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72701a1870 con 0x7f72701013f0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f726ed9d640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f72701a11d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f726002c750 tx=0x7f7260035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260039040 con 0x7f72701013f0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7260044070 con 0x7f72701013f0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72701a1b00 con 0x7f72701013f0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260004420 con 0x7f72701013f0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.310+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72701a2040 con 0x7f72701013f0 2024-03-19T00:35:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.311+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727010ae40 con 0x7f72701013f0 2024-03-19T00:35:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.312+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f72600045c0 con 0x7f72701013f0 2024-03-19T00:35:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.313+0000 7f72677fe640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f725c0a2880 0x7f725c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.313+0000 7f726e59c640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f725c0a2880 0x7f725c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.313+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f72600f0de0 con 0x7f72701013f0 2024-03-19T00:35:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.313+0000 7f726e59c640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f725c0a2880 0x7f725c0a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f727019b130 tx=0x7f7268005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:58.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.317+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f72600be1b0 con 0x7f72701013f0 2024-03-19T00:35:58.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2942685761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:58.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1515639895' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T00:35:58.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4006680287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T00:35:58.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:58.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:58 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:58.501+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f727002d1f0 con 0x7f72701013f0 2024-03-19T00:35:58.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2942685761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:58.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1515639895' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T00:35:58.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4006680287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T00:35:58.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:58.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:58 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:58.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2942685761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:35:58.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1515639895' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T00:35:58.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4006680287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T00:35:58.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:58.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:58 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.190+0000 7f72677fe640 1 -- 172.21.15.7:0/710086648 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f72600bdab0 con 0x7f72701013f0 2024-03-19T00:35:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-03-19T00:35:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f725c0a2880 msgr2=0x7f725c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f725c0a2880 0x7f725c0a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f727019b130 tx=0x7f7268005f90 comp rx=0 tx=0).stop 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 msgr2=0x7f72701a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f72701a11d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f726002c750 tx=0x7f7260035d60 comp rx=0 tx=0).stop 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 shutdown_connections 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f725c0a2880 0x7f725c0a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72701013f0 0x7f72701a11d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7270100a50 0x7f727019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 --2- 172.21.15.7:0/710086648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72701000f0 0x7f7270199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.195+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 >> 172.21.15.7:0/710086648 conn(0x7f72700fbcb0 msgr2=0x7f72700fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.196+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 shutdown_connections 2024-03-19T00:35:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.196+0000 7f72748bf640 1 -- 172.21.15.7:0/710086648 wait complete. 2024-03-19T00:35:59.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 -- 172.21.15.7:0/2617263604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 msgr2=0x7fd9c0101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 --2- 172.21.15.7:0/2617263604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c0101c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009900 tx=0x7fd9b002bc30 comp rx=0 tx=0).stop 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 -- 172.21.15.7:0/2617263604 shutdown_connections 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 --2- 172.21.15.7:0/2617263604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c01033e0 0x7fd9c0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 --2- 172.21.15.7:0/2617263604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 0x7fd9c0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 --2- 172.21.15.7:0/2617263604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c0101c50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 -- 172.21.15.7:0/2617263604 >> 172.21.15.7:0/2617263604 conn(0x7fd9c00fcda0 msgr2=0x7fd9c00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 -- 172.21.15.7:0/2617263604 shutdown_connections 2024-03-19T00:35:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.323+0000 7fd9c579d640 1 -- 172.21.15.7:0/2617263604 wait complete. 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 Processor -- start 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 -- start start 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c00755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 0x7fd9c0073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c01033e0 0x7fd9c0074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c010d360 con 0x7fd9c0101830 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c010d1e0 con 0x7fd9c0102a20 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9c579d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c010d4e0 con 0x7fd9c01033e0 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c00755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c00755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34282/0 (socket says 172.21.15.7:34282) 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9beffd640 1 -- 172.21.15.7:0/1604884466 learned_addr learned my addr 172.21.15.7:0/1604884466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9be7fc640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 0x7fd9c0073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.324+0000 7fd9bf7fe640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c01033e0 0x7fd9c0074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9beffd640 1 -- 172.21.15.7:0/1604884466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c01033e0 msgr2=0x7fd9c0074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9beffd640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c01033e0 0x7fd9c0074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9beffd640 1 -- 172.21.15.7:0/1604884466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 msgr2=0x7fd9c0073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9beffd640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 0x7fd9c0073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9beffd640 1 -- 172.21.15.7:0/1604884466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c00749e0 con 0x7fd9c0101830 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9be7fc640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 0x7fd9c0073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9beffd640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c00755d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0000c00 tx=0x7fd9b0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0043020 con 0x7fd9c0101830 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c01a1bb0 con 0x7fd9c0101830 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9b0004350 con 0x7fd9c0101830 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0003b80 con 0x7fd9c0101830 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.325+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c01a2150 con 0x7fd9c0101830 2024-03-19T00:35:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.326+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c0108bc0 con 0x7fd9c0101830 2024-03-19T00:35:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.327+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fd9b0046020 con 0x7fd9c0101830 2024-03-19T00:35:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.328+0000 7fd9a7fff640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9a80a2800 0x7fd9a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:35:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.328+0000 7fd9be7fc640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9a80a2800 0x7fd9a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:35:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.328+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fd9b00f0d90 con 0x7fd9c0101830 2024-03-19T00:35:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.328+0000 7fd9be7fc640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9a80a2800 0x7fd9a80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40095f0 tx=0x7fd9b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:35:59.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.335+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd9b004d020 con 0x7fd9c0101830 2024-03-19T00:35:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4006680287' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T00:35:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T00:35:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/710086648' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T00:35:59.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:59.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:59.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:35:59 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4006680287' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/710086648' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:35:59 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:35:59.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:35:59.574+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd9c002d1f0 con 0x7fd9c0101830 2024-03-19T00:35:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:35:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4006680287' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T00:35:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T00:35:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/710086648' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T00:35:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:35:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:35:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:35:59.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:35:59 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:36:00.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.206+0000 7fd9a7fff640 1 -- 172.21.15.7:0/1604884466 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd9b00bdfe0 con 0x7fd9c0101830 2024-03-19T00:36:00.209 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-03-19T00:36:00.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.211+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9a80a2800 msgr2=0x7fd9a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:00.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.211+0000 7fd9c579d640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9a80a2800 0x7fd9a80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40095f0 tx=0x7fd9b4005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:00.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.211+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 msgr2=0x7fd9c00755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:00.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.211+0000 7fd9c579d640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c00755d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0000c00 tx=0x7fd9b0005dc0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.215+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 shutdown_connections 2024-03-19T00:36:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.215+0000 7fd9c579d640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9a80a2800 0x7fd9a80a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.215+0000 7fd9c579d640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c01033e0 0x7fd9c0074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.215+0000 7fd9c579d640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c0102a20 0x7fd9c0073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.215+0000 7fd9c579d640 1 --2- 172.21.15.7:0/1604884466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9c0101830 0x7fd9c00755d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.215+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 >> 172.21.15.7:0/1604884466 conn(0x7fd9c00fcda0 msgr2=0x7fd9c0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:00.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.217+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 shutdown_connections 2024-03-19T00:36:00.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.217+0000 7fd9c579d640 1 -- 172.21.15.7:0/1604884466 wait complete. 2024-03-19T00:36:00.238 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-19T00:36:00.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-19T00:36:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:00 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/710086648' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-19T00:36:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:00 smithi044 ceph-mon[66366]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T00:36:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:00 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1604884466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T00:36:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:00 smithi044 ceph-mon[66366]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-19T00:36:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 -- 172.21.15.7:0/2805526245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd301075c0 msgr2=0x7fbd301099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2805526245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd301075c0 0x7fbd301099b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009ca0 tx=0x7fbd2802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 -- 172.21.15.7:0/2805526245 shutdown_connections 2024-03-19T00:36:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2805526245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd301075c0 0x7fbd301099b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2805526245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd30104c90 0x7fbd30107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2805526245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd30102360 0x7fbd30104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 -- 172.21.15.7:0/2805526245 >> 172.21.15.7:0/2805526245 conn(0x7fbd300fbcf0 msgr2=0x7fbd300fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 -- 172.21.15.7:0/2805526245 shutdown_connections 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.357+0000 7fbd3767e640 1 -- 172.21.15.7:0/2805526245 wait complete. 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 Processor -- start 2024-03-19T00:36:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 -- start start 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 0x7fbd30197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd30104c90 0x7fbd30197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd301075c0 0x7fbd3019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010d3b0 con 0x7fbd30102360 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010d230 con 0x7fbd301075c0 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3767e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010d530 con 0x7fbd30104c90 2024-03-19T00:36:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3541a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 0x7fbd30197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3541a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 0x7fbd30197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34292/0 (socket says 172.21.15.7:34292) 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3541a640 1 -- 172.21.15.7:0/2175467008 learned_addr learned my addr 172.21.15.7:0/2175467008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd35c1b640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd301075c0 0x7fbd3019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd34c19640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd30104c90 0x7fbd30197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd35c1b640 1 -- 172.21.15.7:0/2175467008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd30104c90 msgr2=0x7fbd30197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd35c1b640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd30104c90 0x7fbd30197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd35c1b640 1 -- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 msgr2=0x7fbd30197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd35c1b640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 0x7fbd30197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd35c1b640 1 -- 172.21.15.7:0/2175467008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3019f6a0 con 0x7fbd301075c0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.358+0000 7fbd3541a640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 0x7fbd30197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.359+0000 7fbd35c1b640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd301075c0 0x7fbd3019f030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd2802c630 tx=0x7fbd280328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.359+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28041020 con 0x7fbd301075c0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.359+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd28003e20 con 0x7fbd301075c0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.359+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3019f930 con 0x7fbd301075c0 2024-03-19T00:36:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.359+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28039040 con 0x7fbd301075c0 2024-03-19T00:36:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.359+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3019fe20 con 0x7fbd301075c0 2024-03-19T00:36:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.360+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd300fd7e0 con 0x7fbd301075c0 2024-03-19T00:36:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.361+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fbd28044020 con 0x7fbd301075c0 2024-03-19T00:36:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.362+0000 7fbd267fc640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd1c0a2800 0x7fbd1c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.362+0000 7fbd3541a640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd1c0a2800 0x7fbd1c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.362+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fbd280f0eb0 con 0x7fbd301075c0 2024-03-19T00:36:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.362+0000 7fbd3541a640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd1c0a2800 0x7fbd1c0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd20011500 tx=0x7fbd20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:00.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.367+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd2804b020 con 0x7fbd301075c0 2024-03-19T00:36:00.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:00 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/710086648' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-19T00:36:00.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:00 smithi007 ceph-mon[62854]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T00:36:00.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:00 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1604884466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T00:36:00.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:00 smithi007 ceph-mon[62854]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-19T00:36:00.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:00 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/710086648' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-19T00:36:00.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:00 smithi204 ceph-mon[66572]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T00:36:00.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:00 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1604884466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T00:36:00.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:00 smithi204 ceph-mon[66572]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-19T00:36:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:00.579+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fbd3002d3f0 con 0x7fbd301075c0 2024-03-19T00:36:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.210+0000 7fbd267fc640 1 -- 172.21.15.7:0/2175467008 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7fbd280be000 con 0x7fbd301075c0 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd1c0a2800 msgr2=0x7fbd1c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd1c0a2800 0x7fbd1c0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd20011500 tx=0x7fbd20005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd301075c0 msgr2=0x7fbd3019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd301075c0 0x7fbd3019f030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd2802c630 tx=0x7fbd280328a0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 shutdown_connections 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd1c0a2800 0x7fbd1c0a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd301075c0 0x7fbd3019f030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd30104c90 0x7fbd30197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 --2- 172.21.15.7:0/2175467008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd30102360 0x7fbd30197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 >> 172.21.15.7:0/2175467008 conn(0x7fbd300fbcf0 msgr2=0x7fbd30102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 shutdown_connections 2024-03-19T00:36:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.213+0000 7fbd3767e640 1 -- 172.21.15.7:0/2175467008 wait complete. 2024-03-19T00:36:01.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-19T00:36:01.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 -- 172.21.15.7:0/1247476736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00102a60 msgr2=0x7f6b00102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:01.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 --2- 172.21.15.7:0/1247476736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00102a60 0x7f6b00102ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae802bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:01.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 -- 172.21.15.7:0/1247476736 shutdown_connections 2024-03-19T00:36:01.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 --2- 172.21.15.7:0/1247476736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00103420 0x7f6b00109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 --2- 172.21.15.7:0/1247476736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00102a60 0x7f6b00102ee0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 --2- 172.21.15.7:0/1247476736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00101870 0x7f6b00101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 -- 172.21.15.7:0/1247476736 >> 172.21.15.7:0/1247476736 conn(0x7f6b000fcde0 msgr2=0x7f6b000ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 -- 172.21.15.7:0/1247476736 shutdown_connections 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.348+0000 7f6b051f4640 1 -- 172.21.15.7:0/1247476736 wait complete. 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 Processor -- start 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 -- start start 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00101870 0x7f6b00199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00102a60 0x7f6b0019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 0x7f6b001a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0010d380 con 0x7f6b00103420 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0010d200 con 0x7f6b00102a60 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6b051f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0010d500 con 0x7f6b00101870 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6aff59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 0x7f6b001a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6aff59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 0x7f6b001a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34308/0 (socket says 172.21.15.7:34308) 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6aff59e640 1 -- 172.21.15.7:0/3485355666 learned_addr learned my addr 172.21.15.7:0/3485355666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.349+0000 7f6af7fff640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00102a60 0x7f6b0019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6afed9d640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00101870 0x7f6b00199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6aff59e640 1 -- 172.21.15.7:0/3485355666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00101870 msgr2=0x7f6b00199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6aff59e640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00101870 0x7f6b00199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6aff59e640 1 -- 172.21.15.7:0/3485355666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00102a60 msgr2=0x7f6b0019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6aff59e640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00102a60 0x7f6b0019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6aff59e640 1 -- 172.21.15.7:0/3485355666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b001a18d0 con 0x7f6b00103420 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6af7fff640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00102a60 0x7f6b0019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6aff59e640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 0x7f6b001a1230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009fd0 tx=0x7f6af800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8017070 con 0x7f6b00103420 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6af8003e20 con 0x7f6b00103420 2024-03-19T00:36:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b001a1bc0 con 0x7f6b00103420 2024-03-19T00:36:01.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8010460 con 0x7f6b00103420 2024-03-19T00:36:01.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.350+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b001a2090 con 0x7f6b00103420 2024-03-19T00:36:01.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.351+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b001a2260 con 0x7f6b00103420 2024-03-19T00:36:01.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.353+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f6af8003970 con 0x7f6b00103420 2024-03-19T00:36:01.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.353+0000 7f6afcd99640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6aec0a2800 0x7f6aec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:01.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.354+0000 7f6afed9d640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6aec0a2800 0x7f6aec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:01.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.354+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f6af80ce7e0 con 0x7f6b00103420 2024-03-19T00:36:01.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.354+0000 7f6afed9d640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6aec0a2800 0x7f6aec0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b0019ab60 tx=0x7f6af0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:01.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.357+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6af809b1e0 con 0x7f6b00103420 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: pgmap v285: 97 pgs: 7 active+clean, 45 creating+peering, 45 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1604884466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2175467008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:01 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: pgmap v285: 97 pgs: 7 active+clean, 45 creating+peering, 45 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1604884466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2175467008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T00:36:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:01 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:01.545+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6b001a2a10 con 0x7f6b00103420 2024-03-19T00:36:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: pgmap v285: 97 pgs: 7 active+clean, 45 creating+peering, 45 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T00:36:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:36:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T00:36:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1604884466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T00:36:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2175467008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:01.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:01 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:36:02.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:02 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] up:active 2024-03-19T00:36:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:02 smithi007 ceph-mon[62854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:02 smithi007 ceph-mon[62854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-19T00:36:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:02 smithi007 ceph-mon[62854]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T00:36:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:02 smithi007 ceph-mon[62854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:02 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3485355666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T00:36:02.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:02 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] up:active 2024-03-19T00:36:02.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:02 smithi204 ceph-mon[66572]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:02.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:02 smithi204 ceph-mon[66572]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-19T00:36:02.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:02 smithi204 ceph-mon[66572]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T00:36:02.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:02 smithi204 ceph-mon[66572]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:02.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:02 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3485355666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: 2024-03-19T00:36:02.408+0000 7f85dccc4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] up:active 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-mon[66366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-mon[66366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-mon[66366]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-mon[66366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:02.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:02 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3485355666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T00:36:03.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.219+0000 7f6afcd99640 1 -- 172.21.15.7:0/3485355666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f6af809f280 con 0x7f6b00103420 2024-03-19T00:36:03.221 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2024-03-19T00:36:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6aec0a2800 msgr2=0x7f6aec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6aec0a2800 0x7f6aec0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b0019ab60 tx=0x7f6af0005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 msgr2=0x7f6b001a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 0x7f6b001a1230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009fd0 tx=0x7f6af800b260 comp rx=0 tx=0).stop 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 shutdown_connections 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6aec0a2800 0x7f6aec0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b00103420 0x7f6b001a1230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b00102a60 0x7f6b0019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 --2- 172.21.15.7:0/3485355666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b00101870 0x7f6b00199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 >> 172.21.15.7:0/3485355666 conn(0x7f6b000fcde0 msgr2=0x7f6b00108200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 shutdown_connections 2024-03-19T00:36:03.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.222+0000 7f6b051f4640 1 -- 172.21.15.7:0/3485355666 wait complete. 2024-03-19T00:36:03.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.354+0000 7f43d7da9640 1 -- 172.21.15.7:0/420316175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d01000b0 msgr2=0x7f43d01004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.354+0000 7f43d7da9640 1 --2- 172.21.15.7:0/420316175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d01000b0 0x7f43d01004d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009900 tx=0x7f43cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 -- 172.21.15.7:0/420316175 shutdown_connections 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 --2- 172.21.15.7:0/420316175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d01013b0 0x7f43d0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 --2- 172.21.15.7:0/420316175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0100a10 0x7f43d0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 --2- 172.21.15.7:0/420316175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d01000b0 0x7f43d01004d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 -- 172.21.15.7:0/420316175 >> 172.21.15.7:0/420316175 conn(0x7f43d00fbcb0 msgr2=0x7f43d00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 -- 172.21.15.7:0/420316175 shutdown_connections 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 -- 172.21.15.7:0/420316175 wait complete. 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 Processor -- start 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.355+0000 7f43d7da9640 1 -- start start 2024-03-19T00:36:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01000b0 0x7f43d0199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 0x7f43d019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d01013b0 0x7f43d01a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010f530 con 0x7f43d0100a10 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010f3b0 con 0x7f43d01000b0 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010f6b0 con 0x7f43d01013b0 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 0x7f43d019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 0x7f43d019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34326/0 (socket says 172.21.15.7:34326) 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 -- 172.21.15.7:0/3434888780 learned_addr learned my addr 172.21.15.7:0/3434888780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d6346640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d01013b0 0x7f43d01a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5b45640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01000b0 0x7f43d0199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 -- 172.21.15.7:0/3434888780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d01013b0 msgr2=0x7f43d01a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d01013b0 0x7f43d01a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 -- 172.21.15.7:0/3434888780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01000b0 msgr2=0x7f43d0199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01000b0 0x7f43d0199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 -- 172.21.15.7:0/3434888780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d01a18d0 con 0x7f43d0100a10 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d5344640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 0x7f43d019a1d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c000cd80 con 0x7f43d0100a10 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43c00108e0 con 0x7f43d0100a10 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d01a1bc0 con 0x7f43d0100a10 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0018920 con 0x7f43d0100a10 2024-03-19T00:36:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.356+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01a2110 con 0x7f43d0100a10 2024-03-19T00:36:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.357+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d010b090 con 0x7f43d0100a10 2024-03-19T00:36:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.359+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f43c0010a90 con 0x7f43d0100a10 2024-03-19T00:36:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.359+0000 7f43c6ffd640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f43bc0a2800 0x7f43bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.359+0000 7f43d5b45640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f43bc0a2800 0x7f43bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.359+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f43c00cf470 con 0x7f43d0100a10 2024-03-19T00:36:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.360+0000 7f43d5b45640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f43bc0a2800 0x7f43bc0a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43d019ab60 tx=0x7f43cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:03.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.363+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43c009bc40 con 0x7f43d0100a10 2024-03-19T00:36:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:03 smithi204 ceph-mon[66572]: pgmap v288: 97 pgs: 24 active+clean, 45 creating+peering, 28 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-03-19T00:36:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:03 smithi204 ceph-mon[66572]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-19T00:36:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:03 smithi204 ceph-mon[66572]: Cluster is now healthy 2024-03-19T00:36:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:03 smithi204 ceph-mon[66572]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T00:36:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:03.546+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f43d0062530 con 0x7f43d0100a10 2024-03-19T00:36:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:03 smithi044 ceph-mon[66366]: pgmap v288: 97 pgs: 24 active+clean, 45 creating+peering, 28 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-03-19T00:36:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:03 smithi044 ceph-mon[66366]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-19T00:36:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:03 smithi044 ceph-mon[66366]: Cluster is now healthy 2024-03-19T00:36:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:03 smithi044 ceph-mon[66366]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T00:36:03.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:03 smithi007 ceph-mon[62854]: pgmap v288: 97 pgs: 24 active+clean, 45 creating+peering, 28 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-03-19T00:36:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:03 smithi007 ceph-mon[62854]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-19T00:36:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:03 smithi007 ceph-mon[62854]: Cluster is now healthy 2024-03-19T00:36:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:03 smithi007 ceph-mon[62854]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T00:36:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.220+0000 7f43c6ffd640 1 -- 172.21.15.7:0/3434888780 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f43c009fce0 con 0x7f43d0100a10 2024-03-19T00:36:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 4 allow_ec_overwrites to true 2024-03-19T00:36:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f43bc0a2800 msgr2=0x7f43bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f43bc0a2800 0x7f43bc0a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43d019ab60 tx=0x7f43cc005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 msgr2=0x7f43d019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 0x7f43d019a1d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 shutdown_connections 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f43bc0a2800 0x7f43bc0a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d01013b0 0x7f43d01a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d0100a10 0x7f43d019a1d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 --2- 172.21.15.7:0/3434888780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01000b0 0x7f43d0199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 >> 172.21.15.7:0/3434888780 conn(0x7f43d00fbcb0 msgr2=0x7f43d00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 shutdown_connections 2024-03-19T00:36:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.224+0000 7f43d7da9640 1 -- 172.21.15.7:0/3434888780 wait complete. 2024-03-19T00:36:04.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-19T00:36:04.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 -- 172.21.15.7:0/1721428495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e4100730 msgr2=0x7f61e4100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/1721428495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e4100730 0x7f61e4100b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009900 tx=0x7f61e002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 -- 172.21.15.7:0/1721428495 shutdown_connections 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/1721428495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e41022e0 0x7f61e4108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/1721428495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4101920 0x7f61e4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/1721428495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e4100730 0x7f61e4100b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 -- 172.21.15.7:0/1721428495 >> 172.21.15.7:0/1721428495 conn(0x7f61e40fbc80 msgr2=0x7f61e40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 -- 172.21.15.7:0/1721428495 shutdown_connections 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.360+0000 7f61ebb7e640 1 -- 172.21.15.7:0/1721428495 wait complete. 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 Processor -- start 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 -- start start 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4100730 0x7f61e41046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e4101920 0x7f61e4102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 0x7f61e4103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:04.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410c260 con 0x7f61e41022e0 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410c0e0 con 0x7f61e4100730 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ebb7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410c3e0 con 0x7f61e4101920 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 0x7f61e4103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61e991a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4100730 0x7f61e41046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 0x7f61e4103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34340/0 (socket says 172.21.15.7:34340) 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 -- 172.21.15.7:0/46625392 learned_addr learned my addr 172.21.15.7:0/46625392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61e9119640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e4101920 0x7f61e4102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 -- 172.21.15.7:0/46625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e4101920 msgr2=0x7f61e4102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e4101920 0x7f61e4102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 -- 172.21.15.7:0/46625392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4100730 msgr2=0x7f61e41046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4100730 0x7f61e41046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 -- 172.21.15.7:0/46625392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e4103ad0 con 0x7f61e41022e0 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61e991a640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4100730 0x7f61e41046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.361+0000 7f61ea11b640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 0x7f61e4103240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00eaa0 tx=0x7f61dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.362+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00cdb0 con 0x7f61e41022e0 2024-03-19T00:36:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.362+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61dc005020 con 0x7f61e41022e0 2024-03-19T00:36:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.362+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e41a1ba0 con 0x7f61e41022e0 2024-03-19T00:36:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.362+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0116d0 con 0x7f61e41022e0 2024-03-19T00:36:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.362+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a2140 con 0x7f61e41022e0 2024-03-19T00:36:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.363+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e41a2370 con 0x7f61e41022e0 2024-03-19T00:36:04.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.364+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f61dc0051d0 con 0x7f61e41022e0 2024-03-19T00:36:04.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.364+0000 7f61daffd640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61d00a2880 0x7f61d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:04.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.364+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f61dc0cf1f0 con 0x7f61e41022e0 2024-03-19T00:36:04.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.364+0000 7f61e991a640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61d00a2880 0x7f61d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:04.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.365+0000 7f61e991a640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61d00a2880 0x7f61d00a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005ab0 tx=0x7f61e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.368+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f61dc016030 con 0x7f61e41022e0 2024-03-19T00:36:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:04 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3485355666' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T00:36:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:04 smithi204 ceph-mon[66572]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T00:36:04.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:04 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3434888780' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T00:36:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:04.595+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f61e41a2540 con 0x7f61e41022e0 2024-03-19T00:36:04.605 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:04 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3485355666' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T00:36:04.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:04 smithi044 ceph-mon[66366]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T00:36:04.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:04 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3434888780' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T00:36:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:04 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3485355666' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T00:36:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:04 smithi007 ceph-mon[62854]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T00:36:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:04 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3434888780' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T00:36:05.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.342+0000 7f61daffd640 1 -- 172.21.15.7:0/46625392 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f61dc09c090 con 0x7f61e41022e0 2024-03-19T00:36:05.343 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 4 to fsmap 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.344+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61d00a2880 msgr2=0x7f61d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.344+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61d00a2880 0x7f61d00a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005ab0 tx=0x7f61e0005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.344+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 msgr2=0x7f61e4103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.344+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 0x7f61e4103240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00eaa0 tx=0x7f61dc00ef60 comp rx=0 tx=0).stop 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 shutdown_connections 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61d00a2880 0x7f61d00a4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61e41022e0 0x7f61e4103240 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e4101920 0x7f61e4102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 --2- 172.21.15.7:0/46625392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e4100730 0x7f61e41046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 >> 172.21.15.7:0/46625392 conn(0x7f61e40fbc80 msgr2=0x7f61e40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 shutdown_connections 2024-03-19T00:36:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.345+0000 7f61ebb7e640 1 -- 172.21.15.7:0/46625392 wait complete. 2024-03-19T00:36:05.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 -- 172.21.15.7:0/1734138248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e641018b0 msgr2=0x7f1e64101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 --2- 172.21.15.7:0/1734138248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e641018b0 0x7f1e64101cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009900 tx=0x7f1e5402bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 -- 172.21.15.7:0/1734138248 shutdown_connections 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 --2- 172.21.15.7:0/1734138248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64103460 0x7f1e64109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 --2- 172.21.15.7:0/1734138248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e64102aa0 0x7f1e64102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 --2- 172.21.15.7:0/1734138248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e641018b0 0x7f1e64101cd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 -- 172.21.15.7:0/1734138248 >> 172.21.15.7:0/1734138248 conn(0x7f1e640fce40 msgr2=0x7f1e640ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 -- 172.21.15.7:0/1734138248 shutdown_connections 2024-03-19T00:36:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 -- 172.21.15.7:0/1734138248 wait complete. 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.467+0000 7f1e69a97640 1 Processor -- start 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 -- start start 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e641018b0 0x7f1e64197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64102aa0 0x7f1e64198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 0x7f1e6419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6410d3c0 con 0x7f1e64103460 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6410d240 con 0x7f1e64102aa0 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e69a97640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6410d540 con 0x7f1e641018b0 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 0x7f1e6419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 0x7f1e6419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43550/0 (socket says 172.21.15.7:43550) 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 -- 172.21.15.7:0/2789821548 learned_addr learned my addr 172.21.15.7:0/2789821548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e62ffd640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64102aa0 0x7f1e64198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 -- 172.21.15.7:0/2789821548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e641018b0 msgr2=0x7f1e64197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e637fe640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e641018b0 0x7f1e64197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e641018b0 0x7f1e64197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 -- 172.21.15.7:0/2789821548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64102aa0 msgr2=0x7f1e64198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64102aa0 0x7f1e64198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 -- 172.21.15.7:0/2789821548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6419f6e0 con 0x7f1e64103460 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e62ffd640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64102aa0 0x7f1e64198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.468+0000 7f1e63fff640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 0x7f1e6419f070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00eb10 tx=0x7f1e5c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.469+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00ce20 con 0x7f1e64103460 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.469+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e5c005020 con 0x7f1e64103460 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.469+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6419f9d0 con 0x7f1e64103460 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.469+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c010690 con 0x7f1e64103460 2024-03-19T00:36:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.469+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6419ff50 con 0x7f1e64103460 2024-03-19T00:36:05.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.470+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e64108c70 con 0x7f1e64103460 2024-03-19T00:36:05.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.471+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f1e5c0051d0 con 0x7f1e64103460 2024-03-19T00:36:05.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.471+0000 7f1e60ff9640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e4c0a2800 0x7f1e4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.471+0000 7f1e637fe640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e4c0a2800 0x7f1e4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.471+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1e5c0cf320 con 0x7f1e64103460 2024-03-19T00:36:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.472+0000 7f1e637fe640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e4c0a2800 0x7f1e4c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e54036040 tx=0x7f1e54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.476+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1e5c09c120 con 0x7f1e64103460 2024-03-19T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:05 smithi044 ceph-mon[66366]: pgmap v291: 129 pgs: 68 active+clean, 46 creating+peering, 15 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-19T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:05 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3434888780' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-19T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:05 smithi044 ceph-mon[66366]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:05 smithi044 ceph-mon[66366]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:05 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/46625392' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T00:36:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:05 smithi204 ceph-mon[66572]: pgmap v291: 129 pgs: 68 active+clean, 46 creating+peering, 15 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-19T00:36:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:05 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3434888780' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-19T00:36:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:05 smithi204 ceph-mon[66572]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T00:36:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:05 smithi204 ceph-mon[66572]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:36:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:05 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/46625392' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T00:36:05.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.697+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e6402d1f0 con 0x7f1e64103460 2024-03-19T00:36:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.697+0000 7f1e60ff9640 1 -- 172.21.15.7:0/2789821548 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11782 (secure 0 0 0) 0x7f1e5c09ba20 con 0x7f1e64103460 2024-03-19T00:36:05.698 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:05.698 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:04.597762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-03-19T00:36:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-03-19T00:36:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e4c0a2800 msgr2=0x7f1e4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e4c0a2800 0x7f1e4c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e54036040 tx=0x7f1e54000bc0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 msgr2=0x7f1e6419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 0x7f1e6419f070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00eb10 tx=0x7f1e5c00ef10 comp rx=0 tx=0).stop 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 shutdown_connections 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e4c0a2800 0x7f1e4c0a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e64103460 0x7f1e6419f070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e64102aa0 0x7f1e64198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 --2- 172.21.15.7:0/2789821548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e641018b0 0x7f1e64197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 >> 172.21.15.7:0/2789821548 conn(0x7f1e640fce40 msgr2=0x7f1e64105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 shutdown_connections 2024-03-19T00:36:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.699+0000 7f1e69a97640 1 -- 172.21.15.7:0/2789821548 wait complete. 2024-03-19T00:36:05.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:05 smithi007 ceph-mon[62854]: pgmap v291: 129 pgs: 68 active+clean, 46 creating+peering, 15 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-19T00:36:05.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:05 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3434888780' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-19T00:36:05.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:05 smithi007 ceph-mon[62854]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T00:36:05.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:05 smithi007 ceph-mon[62854]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T00:36:05.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:05 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/46625392' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T00:36:05.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 -- 172.21.15.7:0/2210565213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 msgr2=0x7efd68108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/2210565213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd68108c00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd60009900 tx=0x7efd6002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 -- 172.21.15.7:0/2210565213 shutdown_connections 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/2210565213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd68108c00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/2210565213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd681019b0 0x7efd68101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/2210565213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 0x7efd68100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.814+0000 7efd6d6c0640 1 -- 172.21.15.7:0/2210565213 >> 172.21.15.7:0/2210565213 conn(0x7efd680fbd30 msgr2=0x7efd680fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 -- 172.21.15.7:0/2210565213 shutdown_connections 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 -- 172.21.15.7:0/2210565213 wait complete. 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 Processor -- start 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 -- start start 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 0x7efd68104820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd681019b0 0x7efd68102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd681033a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810c260 con 0x7efd68102370 2024-03-19T00:36:05.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810c0e0 con 0x7efd681019b0 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd6d6c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810c3e0 con 0x7efd681007c0 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd681033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd681033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43564/0 (socket says 172.21.15.7:43564) 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd677fe640 1 -- 172.21.15.7:0/1530640443 learned_addr learned my addr 172.21.15.7:0/1530640443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.815+0000 7efd667fc640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd681019b0 0x7efd68102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd66ffd640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 0x7efd68104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd677fe640 1 -- 172.21.15.7:0/1530640443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 msgr2=0x7efd68104820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd677fe640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 0x7efd68104820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd677fe640 1 -- 172.21.15.7:0/1530640443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd681019b0 msgr2=0x7efd68102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd677fe640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd681019b0 0x7efd68102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd677fe640 1 -- 172.21.15.7:0/1530640443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd68103c30 con 0x7efd68102370 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd66ffd640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 0x7efd68104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:36:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd677fe640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd681033a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd60009230 tx=0x7efd600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:05.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60043020 con 0x7efd68102370 2024-03-19T00:36:05.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd600044c0 con 0x7efd68102370 2024-03-19T00:36:05.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a1b50 con 0x7efd68102370 2024-03-19T00:36:05.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60003bc0 con 0x7efd68102370 2024-03-19T00:36:05.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.816+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a20a0 con 0x7efd68102370 2024-03-19T00:36:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.817+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd681a2270 con 0x7efd68102370 2024-03-19T00:36:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.818+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7efd60046020 con 0x7efd68102370 2024-03-19T00:36:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.819+0000 7efd5ffff640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd540a2800 0x7efd540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.819+0000 7efd66ffd640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd540a2800 0x7efd540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.819+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efd600f1000 con 0x7efd68102370 2024-03-19T00:36:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.819+0000 7efd66ffd640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd540a2800 0x7efd540a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efd68101810 tx=0x7efd58009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:05.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:05.823+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd6004d020 con 0x7efd68102370 2024-03-19T00:36:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.007+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd681a2930 con 0x7efd68102370 2024-03-19T00:36:06.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.008+0000 7efd5ffff640 1 -- 172.21.15.7:0/1530640443 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7efd600bde60 con 0x7efd68102370 2024-03-19T00:36:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:06.009 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd540a2800 msgr2=0x7efd540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd540a2800 0x7efd540a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efd68101810 tx=0x7efd58009400 comp rx=0 tx=0).stop 2024-03-19T00:36:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 msgr2=0x7efd681033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd681033a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd60009230 tx=0x7efd600040c0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 shutdown_connections 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd540a2800 0x7efd540a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd68102370 0x7efd681033a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd681019b0 0x7efd68102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 --2- 172.21.15.7:0/1530640443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd681007c0 0x7efd68104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 >> 172.21.15.7:0/1530640443 conn(0x7efd680fbd30 msgr2=0x7efd68069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 shutdown_connections 2024-03-19T00:36:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.010+0000 7efd6d6c0640 1 -- 172.21.15.7:0/1530640443 wait complete. 2024-03-19T00:36:06.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 -- 172.21.15.7:0/1802742896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8102aa0 msgr2=0x7f03b8102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 --2- 172.21.15.7:0/1802742896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8102aa0 0x7f03b8102f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009860 tx=0x7f03a802bb70 comp rx=0 tx=0).stop 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 -- 172.21.15.7:0/1802742896 shutdown_connections 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 --2- 172.21.15.7:0/1802742896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8103460 0x7f03b8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 --2- 172.21.15.7:0/1802742896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8102aa0 0x7f03b8102f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 --2- 172.21.15.7:0/1802742896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b81018b0 0x7f03b8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 -- 172.21.15.7:0/1802742896 >> 172.21.15.7:0/1802742896 conn(0x7f03b80fce40 msgr2=0x7f03b80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 -- 172.21.15.7:0/1802742896 shutdown_connections 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 -- 172.21.15.7:0/1802742896 wait complete. 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 Processor -- start 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.125+0000 7f03bfa87640 1 -- start start 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b81018b0 0x7f03b8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 0x7f03b8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8103460 0x7f03b819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03b810d370 con 0x7f03b8102aa0 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03b810d1f0 con 0x7f03b8103460 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03b810d4f0 con 0x7f03b81018b0 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 0x7f03b8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 0x7f03b8197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43574/0 (socket says 172.21.15.7:43574) 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 -- 172.21.15.7:0/3730099276 learned_addr learned my addr 172.21.15.7:0/3730099276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03be024640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8103460 0x7f03b819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd823640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b81018b0 0x7f03b8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 -- 172.21.15.7:0/3730099276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b81018b0 msgr2=0x7f03b8197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b81018b0 0x7f03b8197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 -- 172.21.15.7:0/3730099276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8103460 msgr2=0x7f03b819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8103460 0x7f03b819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 -- 172.21.15.7:0/3730099276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b819f6c0 con 0x7f03b8102aa0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bd022640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 0x7f03b8197fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005940 tx=0x7f03a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8041020 con 0x7f03b8102aa0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b819f950 con 0x7f03b8102aa0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03a802c9c0 con 0x7f03b8102aa0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8039040 con 0x7f03b8102aa0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.126+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b819fe40 con 0x7f03b8102aa0 2024-03-19T00:36:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.127+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b8108cf0 con 0x7f03b8102aa0 2024-03-19T00:36:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.128+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f03a8044020 con 0x7f03b8102aa0 2024-03-19T00:36:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.129+0000 7f03aeffd640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03a40a2800 0x7f03a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.129+0000 7f03bd823640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03a40a2800 0x7f03a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.129+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f03a80f0690 con 0x7f03b8102aa0 2024-03-19T00:36:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.129+0000 7f03bd823640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03a40a2800 0x7f03a40a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005fd0 tx=0x7f03b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:06.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.133+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03a804b020 con 0x7f03b8102aa0 2024-03-19T00:36:06.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.320+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03b8062420 con 0x7f03b8102aa0 2024-03-19T00:36:06.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.321+0000 7f03aeffd640 1 -- 172.21.15.7:0/3730099276 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f03a80bd4f0 con 0x7f03b8102aa0 2024-03-19T00:36:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:06.323 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:06.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03a40a2800 msgr2=0x7f03a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03a40a2800 0x7f03a40a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005fd0 tx=0x7f03b4005850 comp rx=0 tx=0).stop 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 msgr2=0x7f03b8197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 0x7f03b8197fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005940 tx=0x7f03a8004270 comp rx=0 tx=0).stop 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 shutdown_connections 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03a40a2800 0x7f03a40a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03b8103460 0x7f03b819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b8102aa0 0x7f03b8197fe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 --2- 172.21.15.7:0/3730099276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b81018b0 0x7f03b8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 >> 172.21.15.7:0/3730099276 conn(0x7f03b80fce40 msgr2=0x7f03b8105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 shutdown_connections 2024-03-19T00:36:06.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.323+0000 7f03bfa87640 1 -- 172.21.15.7:0/3730099276 wait complete. 2024-03-19T00:36:06.334 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:06 smithi007 ceph-mon[62854]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T00:36:06.334 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/46625392' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-19T00:36:06.334 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:06 smithi007 ceph-mon[62854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:06.334 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2789821548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:06.334 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1530640443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:06.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 -- 172.21.15.7:0/1192025544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc828104c90 msgr2=0x7fc828107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 --2- 172.21.15.7:0/1192025544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc828104c90 0x7fc828107080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 -- 172.21.15.7:0/1192025544 shutdown_connections 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 --2- 172.21.15.7:0/1192025544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8281075c0 0x7fc828109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 --2- 172.21.15.7:0/1192025544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc828104c90 0x7fc828107080 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 --2- 172.21.15.7:0/1192025544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 0x7fc828104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 -- 172.21.15.7:0/1192025544 >> 172.21.15.7:0/1192025544 conn(0x7fc8280fbd50 msgr2=0x7fc8280fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 -- 172.21.15.7:0/1192025544 shutdown_connections 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.451+0000 7fc82fb22640 1 -- 172.21.15.7:0/1192025544 wait complete. 2024-03-19T00:36:06.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 Processor -- start 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 -- start start 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 0x7fc828101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc828104c90 0x7fc828100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 0x7fc828100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc82810d440 con 0x7fc8281075c0 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc82810d2c0 con 0x7fc828102360 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82fb22640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc82810d5c0 con 0x7fc828104c90 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 0x7fc828100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82d8be640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 0x7fc828101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 0x7fc828100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43590/0 (socket says 172.21.15.7:43590) 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 -- 172.21.15.7:0/3137050879 learned_addr learned my addr 172.21.15.7:0/3137050879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82d0bd640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc828104c90 0x7fc828100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 -- 172.21.15.7:0/3137050879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc828104c90 msgr2=0x7fc828100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc828104c90 0x7fc828100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 -- 172.21.15.7:0/3137050879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 msgr2=0x7fc828101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 0x7fc828101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 -- 172.21.15.7:0/3137050879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8281010e0 con 0x7fc8281075c0 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82d8be640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 0x7fc828101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.452+0000 7fc82e0bf640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 0x7fc828100850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc820009fd0 tx=0x7fc82000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.453+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820011830 con 0x7fc8281075c0 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.453+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281a1c00 con 0x7fc8281075c0 2024-03-19T00:36:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.453+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc820016070 con 0x7fc8281075c0 2024-03-19T00:36:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.453+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820010de0 con 0x7fc8281075c0 2024-03-19T00:36:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.453+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a20d0 con 0x7fc8281075c0 2024-03-19T00:36:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.454+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc828068af0 con 0x7fc8281075c0 2024-03-19T00:36:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.455+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fc8200119d0 con 0x7fc8281075c0 2024-03-19T00:36:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.455+0000 7fc81effd640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc8140a2800 0x7fc8140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.455+0000 7fc82d8be640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc8140a2800 0x7fc8140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.455+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc8200ce430 con 0x7fc8281075c0 2024-03-19T00:36:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.456+0000 7fc82d8be640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc8140a2800 0x7fc8140a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8240099a0 tx=0x7fc824009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.460+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc82009b290 con 0x7fc8281075c0 2024-03-19T00:36:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:06 smithi204 ceph-mon[66572]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T00:36:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/46625392' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-19T00:36:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:06 smithi204 ceph-mon[66572]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2789821548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1530640443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:06.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:06 smithi044 ceph-mon[66366]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T00:36:06.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/46625392' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-19T00:36:06.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:06 smithi044 ceph-mon[66366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:06.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2789821548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:06.521 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1530640443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.683+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc82802c830 con 0x7fc8281075c0 2024-03-19T00:36:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.684+0000 7fc81effd640 1 -- 172.21.15.7:0/3137050879 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11782 (secure 0 0 0) 0x7fc82009ab90 con 0x7fc8281075c0 2024-03-19T00:36:06.685 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:04.597762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-03-19T00:36:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-03-19T00:36:06.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc8140a2800 msgr2=0x7fc8140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc8140a2800 0x7fc8140a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8240099a0 tx=0x7fc824009500 comp rx=0 tx=0).stop 2024-03-19T00:36:06.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 msgr2=0x7fc828100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 0x7fc828100850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc820009fd0 tx=0x7fc82000b8e0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 shutdown_connections 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc8140a2800 0x7fc8140a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8281075c0 0x7fc828100850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc828104c90 0x7fc828100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 --2- 172.21.15.7:0/3137050879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc828102360 0x7fc828101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 >> 172.21.15.7:0/3137050879 conn(0x7fc8280fbd50 msgr2=0x7fc828102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 shutdown_connections 2024-03-19T00:36:06.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.686+0000 7fc82fb22640 1 -- 172.21.15.7:0/3137050879 wait complete. 2024-03-19T00:36:06.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 -- 172.21.15.7:0/1914520412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0224102ed0 msgr2=0x7f02241032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 --2- 172.21.15.7:0/1914520412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0224102ed0 0x7f02241032f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0220009900 tx=0x7f022002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 -- 172.21.15.7:0/1914520412 shutdown_connections 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 --2- 172.21.15.7:0/1914520412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224100300 0x7f0224109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 --2- 172.21.15.7:0/1914520412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02240ff940 0x7f02240ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 --2- 172.21.15.7:0/1914520412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0224102ed0 0x7f02241032f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 -- 172.21.15.7:0/1914520412 >> 172.21.15.7:0/1914520412 conn(0x7f02240fb4a0 msgr2=0x7f02240fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 -- 172.21.15.7:0/1914520412 shutdown_connections 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.799+0000 7f022c5e8640 1 -- 172.21.15.7:0/1914520412 wait complete. 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 Processor -- start 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 -- start start 2024-03-19T00:36:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 0x7f0224197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0224100300 0x7f0224197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224102ed0 0x7f022419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f022410d390 con 0x7f02240ff940 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f022410d210 con 0x7f0224100300 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022c5e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f022410d510 con 0x7f0224102ed0 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 0x7f0224197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 0x7f0224197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43614/0 (socket says 172.21.15.7:43614) 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 -- 172.21.15.7:0/959000091 learned_addr learned my addr 172.21.15.7:0/959000091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f0229b83640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0224100300 0x7f0224197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022ab85640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224102ed0 0x7f022419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 -- 172.21.15.7:0/959000091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224102ed0 msgr2=0x7f022419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224102ed0 0x7f022419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 -- 172.21.15.7:0/959000091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0224100300 msgr2=0x7f0224197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0224100300 0x7f0224197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 -- 172.21.15.7:0/959000091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022419f690 con 0x7f02240ff940 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022ab85640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224102ed0 0x7f022419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.800+0000 7f022a384640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 0x7f0224197a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0220000c00 tx=0x7f0220004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.801+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220043020 con 0x7f02240ff940 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.801+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0220004aa0 con 0x7f02240ff940 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.801+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022419f920 con 0x7f02240ff940 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.801+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220003d00 con 0x7f02240ff940 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.801+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022419fec0 con 0x7f02240ff940 2024-03-19T00:36:06.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.802+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0224108db0 con 0x7f02240ff940 2024-03-19T00:36:06.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.803+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f0220046020 con 0x7f02240ff940 2024-03-19T00:36:06.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.803+0000 7f021b7fe640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02100a2880 0x7f02100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:06.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.803+0000 7f0229b83640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02100a2880 0x7f02100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:06.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.804+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f02200f1570 con 0x7f02240ff940 2024-03-19T00:36:06.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.804+0000 7f0229b83640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02100a2880 0x7f02100a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0224198f80 tx=0x7f021400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:06.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.807+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f022004d020 con 0x7f02240ff940 2024-03-19T00:36:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.992+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f022402d440 con 0x7f02240ff940 2024-03-19T00:36:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.994+0000 7f021b7fe640 1 -- 172.21.15.7:0/959000091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f02200be3d0 con 0x7f02240ff940 2024-03-19T00:36:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:06.995 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02100a2880 msgr2=0x7f02100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02100a2880 0x7f02100a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0224198f80 tx=0x7f021400b410 comp rx=0 tx=0).stop 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 msgr2=0x7f0224197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 0x7f0224197a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0220000c00 tx=0x7f0220004270 comp rx=0 tx=0).stop 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 shutdown_connections 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02100a2880 0x7f02100a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0224102ed0 0x7f022419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0224100300 0x7f0224197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 --2- 172.21.15.7:0/959000091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02240ff940 0x7f0224197a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 >> 172.21.15.7:0/959000091 conn(0x7f02240fb4a0 msgr2=0x7f0224105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 shutdown_connections 2024-03-19T00:36:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:06.996+0000 7f022c5e8640 1 -- 172.21.15.7:0/959000091 wait complete. 2024-03-19T00:36:07.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-19T00:35:59.576911+0000', 'modified': '2024-03-19T00:36:04.597762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/617428839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 617428839}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 617428839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24521, 'qdb_cluster': [24521]} 2024-03-19T00:36:07.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-19T00:36:07.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/3123086710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b381022f0 msgr2=0x7f2b38108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/3123086710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b381022f0 0x7f2b38108b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009ca0 tx=0x7f2b3002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/3123086710 shutdown_connections 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/3123086710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b381022f0 0x7f2b38108b80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/3123086710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b38101930 0x7f2b38101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/3123086710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b38100740 0x7f2b38100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/3123086710 >> 172.21.15.7:0/3123086710 conn(0x7f2b380fbcb0 msgr2=0x7f2b380fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.112+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/3123086710 shutdown_connections 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/3123086710 wait complete. 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 Processor -- start 2024-03-19T00:36:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 -- start start 2024-03-19T00:36:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b38100740 0x7f2b38197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 0x7f2b38197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b381022f0 0x7f2b3819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3810d390 con 0x7f2b38101930 2024-03-19T00:36:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3810d210 con 0x7f2b38100740 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b3e4dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3810d510 con 0x7f2b381022f0 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 0x7f2b38197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.113+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 0x7f2b38197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43638/0 (socket says 172.21.15.7:43638) 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 -- 172.21.15.7:0/995013030 learned_addr learned my addr 172.21.15.7:0/995013030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b37fff640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b38100740 0x7f2b38197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 -- 172.21.15.7:0/995013030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b381022f0 msgr2=0x7f2b3819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b381022f0 0x7f2b3819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 -- 172.21.15.7:0/995013030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b38100740 msgr2=0x7f2b38197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b38100740 0x7f2b38197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 -- 172.21.15.7:0/995013030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3819f640 con 0x7f2b38101930 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b377fe640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 0x7f2b38197f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00d770 tx=0x7f2b2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b357fa640 1 -- 172.21.15.7:0/995013030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c017070 con 0x7f2b38101930 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3819f930 con 0x7f2b38101930 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b357fa640 1 -- 172.21.15.7:0/995013030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b2c0054d0 con 0x7f2b38101930 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b357fa640 1 -- 172.21.15.7:0/995013030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c004cf0 con 0x7f2b38101930 2024-03-19T00:36:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.114+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3819fe80 con 0x7f2b38101930 2024-03-19T00:36:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.116+0000 7f2b357fa640 1 -- 172.21.15.7:0/995013030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f2b2c004e90 con 0x7f2b38101930 2024-03-19T00:36:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 Processor -- start 2024-03-19T00:36:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 -- start start 2024-03-19T00:36:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da08410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da0d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55877d96cde0 con 0x55877da0db50 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55877d915c80 con 0x55877da07ff0 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b411a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55877d968e80 con 0x55877da089e0 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b357fa640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b200a2800 0x7f2b200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b3ca79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da08410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b3ca79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da08410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53096/0 (socket says 172.21.15.7:53096) 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da0ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43646/0 (socket says 172.21.15.7:43646) 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b3ca79640 1 -- 172.21.15.7:0/230480979 learned_addr learned my addr 172.21.15.7:0/230480979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b200a2800 0x7f2b200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b357fa640 1 -- 172.21.15.7:0/995013030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2b2c0ce6c0 con 0x7f2b38101930 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b37fff640 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da0d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 -- 172.21.15.7:0/230480979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 msgr2=0x55877da0d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da0d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 -- 172.21.15.7:0/230480979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 msgr2=0x55877da08410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da08410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b377fe640 1 -- 172.21.15.7:0/230480979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877d927fb0 con 0x55877da0db50 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.117+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da08410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b377fe640 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da0ffe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c09aac0 tx=0x7f2b2c06daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d4201373bf0a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b200a2800 0x7f2b200a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b30032420 tx=0x7f2b30032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b1e7fc640 1 -- 172.21.15.7:0/230480979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c020070 con 0x55877da0db50 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b1e7fc640 1 -- 172.21.15.7:0/230480979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b2c009c30 con 0x55877da0db50 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b1e7fc640 1 -- 172.21.15.7:0/230480979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0a3560 con 0x55877da0db50 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 -- 172.21.15.7:0/230480979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 msgr2=0x55877da0ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da0ffe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c09aac0 tx=0x7f2b2c06daf0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 -- 172.21.15.7:0/230480979 shutdown_connections 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da0ffe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da0d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 --2- 172.21.15.7:0/230480979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da08410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 -- 172.21.15.7:0/230480979 >> 172.21.15.7:0/230480979 conn(0x55877da03560 msgr2=0x55877da03960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 -- 172.21.15.7:0/230480979 shutdown_connections 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.118+0000 7f2b411a0740 1 -- 172.21.15.7:0/230480979 wait complete. 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 -- start start 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 Processor -- start 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 0x55877da12590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da089e0 0x55877da12ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da19b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55877da10b00 con 0x55877da07ff0 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55877d928fc0 con 0x55877da089e0 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55877da10c80 con 0x55877da0db50 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 0x55877da12590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b411a0740 10 client.0 _ll_register_callbacks cb 0x55877d964a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 0x55877da12590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43656/0 (socket says 172.21.15.7:43656) 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 -- 172.21.15.7:0/4049963127 learned_addr learned my addr 172.21.15.7:0/4049963127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b377fe640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da089e0 0x55877da12ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da19b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 -- 172.21.15.7:0/4049963127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 msgr2=0x55877da19b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da19b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 -- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da089e0 msgr2=0x55877da12ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.119+0000 7f2b37fff640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da089e0 0x55877da12ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b37fff640 1 -- 172.21.15.7:0/4049963127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877d6812d0 con 0x55877da07ff0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da19b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b377fe640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da089e0 0x55877da12ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b37fff640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 0x55877da12590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b38101790 tx=0x7f2b2800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3776a9a0411ae616 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b177fe640 1 -- 172.21.15.7:0/4049963127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28005020 con 0x55877da07ff0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b177fe640 1 -- 172.21.15.7:0/4049963127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2b280051c0 con 0x55877da07ff0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55877d97a320 con 0x55877da07ff0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b177fe640 1 -- 172.21.15.7:0/4049963127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2b280053c0 con 0x55877da07ff0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 10 client.14772 fetch_fsmap learned FSMap version 17 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55877d74b6b0 con 0x55877da07ff0 2024-03-19T00:36:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b177fe640 1 -- 172.21.15.7:0/4049963127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55877d97a320 con 0x55877da07ff0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b177fe640 1 -- 172.21.15.7:0/4049963127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2b28019030 con 0x55877da07ff0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 10 client.14772 fetch_fsmap finished waiting for FSMap version 17 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 10 client.14772 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 20 client.14772 populate_metadata read hostname 'smithi007' 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da1f4c0 0x55877da20930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 4 client.14772 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.120+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55877d974670 con 0x55877da1f4c0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.121+0000 7f2b377fe640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da1f4c0 0x55877da20930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.121+0000 7f2b377fe640 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da1f4c0 0x55877da20930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feac9c4e71e5318f server_cookie=6e6e326e4faad8f6 in_seq=0 out_seq=0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.121+0000 7f2b177fe640 10 client.14772 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b177fe640 1 -- 172.21.15.7:0/4049963127 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55877d974670 con 0x55877da1f4c0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b177fe640 10 client.14772 handle_command_reply: tid=0 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 client.14772 shutdown 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 msgr2=0x55877da12590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 0x55877da12590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b38101790 tx=0x7f2b2800be60 comp rx=0 tx=0).stop 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 shutdown_connections 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da19b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da089e0 0x55877da12ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da1f4c0 0x55877da20930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 --2- 172.21.15.7:0/4049963127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da07ff0 0x55877da12590 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 >> 172.21.15.7:0/4049963127 conn(0x55877da03560 msgr2=0x55877da027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b177fe640 0 client.14772 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 shutdown_connections 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 1 -- 172.21.15.7:0/4049963127 wait complete. 2024-03-19T00:36:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.122+0000 7f2b411a0740 20 client.14772 trim_cache size 0 max 16384 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 Processor -- start 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 -- start start 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da09ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 0x55877da19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da0a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55877da1ec70 con 0x55877da089e0 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55877da1f340 con 0x55877da07ff0 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b411a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55877da1fa10 con 0x55877da0db50 2024-03-19T00:36:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 0x55877da19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 0x55877da19ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43668/0 (socket says 172.21.15.7:43668) 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 -- 172.21.15.7:0/403776985 learned_addr learned my addr 172.21.15.7:0/403776985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b37fff640 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da09ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 -- 172.21.15.7:0/403776985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 msgr2=0x55877da0a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da0a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 -- 172.21.15.7:0/403776985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 msgr2=0x55877da09ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da09ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.140+0000 7f2b377fe640 1 -- 172.21.15.7:0/403776985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877da1fec0 con 0x55877da089e0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b377fe640 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 0x55877da19ad0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c097790 tx=0x7f2b2c09aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc9bd0b4e115d42 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b1dffb640 1 -- 172.21.15.7:0/403776985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c020040 con 0x55877da089e0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b1dffb640 1 -- 172.21.15.7:0/403776985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b2c09b4c0 con 0x55877da089e0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b1dffb640 1 -- 172.21.15.7:0/403776985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c097430 con 0x55877da089e0 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 -- 172.21.15.7:0/403776985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 msgr2=0x55877da19ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 0x55877da19ad0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c097790 tx=0x7f2b2c09aa00 comp rx=0 tx=0).stop 2024-03-19T00:36:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 -- 172.21.15.7:0/403776985 shutdown_connections 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da0db50 0x55877da0a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da089e0 0x55877da19ad0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 --2- 172.21.15.7:0/403776985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da09ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 -- 172.21.15.7:0/403776985 >> 172.21.15.7:0/403776985 conn(0x55877da03560 msgr2=0x55877da03940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 -- 172.21.15.7:0/403776985 shutdown_connections 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.141+0000 7f2b411a0740 1 -- 172.21.15.7:0/403776985 wait complete. 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 -- start start 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 Processor -- start 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da16770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55877da207b0 con 0x55877da0db50 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55877da200e0 con 0x55877da07ff0 2024-03-19T00:36:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55877da20e80 con 0x55877da089e0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b411a0740 10 client.0 _ll_register_callbacks cb 0x7f2b28005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da83220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43684/0 (socket says 172.21.15.7:43684) 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da16770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 -- 172.21.15.7:0/2769847775 learned_addr learned my addr 172.21.15.7:0/2769847775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b377fe640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 -- 172.21.15.7:0/2769847775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 msgr2=0x55877da16cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da16cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 -- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 msgr2=0x55877da16770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.142+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da16770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b3ca79640 1 -- 172.21.15.7:0/2769847775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877da172a0 con 0x55877da0db50 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b37fff640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da16770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b3ca79640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da83220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2b3002c630 tx=0x7f2b30009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafd0dd5eb3d3e71 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:36:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b177fe640 1 -- 172.21.15.7:0/2769847775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30039040 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b177fe640 1 -- 172.21.15.7:0/2769847775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2b30004250 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b2c09b4c0 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b177fe640 1 -- 172.21.15.7:0/2769847775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2b30059050 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 10 client.14778 fetch_fsmap learned FSMap version 17 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55877da84840 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b177fe640 1 -- 172.21.15.7:0/2769847775 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b2c09b4c0 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b177fe640 1 -- 172.21.15.7:0/2769847775 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2b3003f040 con 0x55877da0db50 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 10 client.14778 fetch_fsmap finished waiting for FSMap version 17 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 10 client.14778 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 20 client.14778 populate_metadata read hostname 'smithi007' 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da894a0 0x55877da85fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 4 client.14778 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.143+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "status"}) -- 0x55877da10460 con 0x55877da894a0 2024-03-19T00:36:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b37fff640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da894a0 0x55877da85fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b37fff640 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da894a0 0x55877da85fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66839f74fca6a838 server_cookie=4460ca6f4b929dd2 in_seq=0 out_seq=0 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b177fe640 10 client.14778 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b177fe640 1 -- 172.21.15.7:0/2769847775 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55877da10460 con 0x55877da894a0 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b177fe640 10 client.14778 handle_command_reply: tid=0 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 client.14778 shutdown 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 msgr2=0x55877da83220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da83220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2b3002c630 tx=0x7f2b30009360 comp rx=0 tx=0).stop 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 shutdown_connections 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55877da894a0 0x55877da85fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877da0db50 0x55877da83220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b177fe640 0 client.14778 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55877da089e0 0x55877da16cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 --2- 172.21.15.7:0/2769847775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55877da07ff0 0x55877da16770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.144+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 >> 172.21.15.7:0/2769847775 conn(0x55877da03560 msgr2=0x55877da1e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 shutdown_connections 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b411a0740 1 -- 172.21.15.7:0/2769847775 wait complete. 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b411a0740 20 client.14778 trim_cache size 0 max 16384 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24521, 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.9416417040000002, 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 17, 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 89, 2024-03-19T00:36:07.152 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:36:07.153 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.112435778 2024-03-19T00:36:07.153 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:36:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b200a2800 msgr2=0x7f2b200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b200a2800 0x7f2b200a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b30032420 tx=0x7f2b30032340 comp rx=0 tx=0).stop 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 msgr2=0x7f2b38197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 0x7f2b38197f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00d770 tx=0x7f2b2c00dc30 comp rx=0 tx=0).stop 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 shutdown_connections 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2b200a2800 0x7f2b200a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b381022f0 0x7f2b3819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b38101930 0x7f2b38197f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 --2- 172.21.15.7:0/995013030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b38100740 0x7f2b38197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 >> 172.21.15.7:0/995013030 conn(0x7f2b380fbcb0 msgr2=0x7f2b38104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 shutdown_connections 2024-03-19T00:36:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:07.145+0000 7f2b3e4dc640 1 -- 172.21.15.7:0/995013030 wait complete. 2024-03-19T00:36:07.157 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-19T00:36:07.157 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:07.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-19T00:36:07.399 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:07 smithi007 ceph-mon[62854]: pgmap v294: 129 pgs: 107 active+clean, 17 creating+peering, 5 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-03-19T00:36:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:07 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3730099276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:07 smithi007 ceph-mon[62854]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T00:36:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:07 smithi007 ceph-mon[62854]: Cluster is now healthy 2024-03-19T00:36:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:07 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3137050879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:07 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/959000091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:07 smithi204 ceph-mon[66572]: pgmap v294: 129 pgs: 107 active+clean, 17 creating+peering, 5 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-03-19T00:36:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:07 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3730099276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:07 smithi204 ceph-mon[66572]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T00:36:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:07 smithi204 ceph-mon[66572]: Cluster is now healthy 2024-03-19T00:36:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:07 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3137050879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:07 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/959000091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:07 smithi044 ceph-mon[66366]: pgmap v294: 129 pgs: 107 active+clean, 17 creating+peering, 5 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-03-19T00:36:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:07 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3730099276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:07 smithi044 ceph-mon[66366]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T00:36:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:07 smithi044 ceph-mon[66366]: Cluster is now healthy 2024-03-19T00:36:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:07 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3137050879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:07.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:07 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/959000091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 -- 172.21.15.7:0/1703435286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 msgr2=0x7f8f74101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 --2- 172.21.15.7:0/1703435286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74101870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f640099c0 tx=0x7f8f64009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 -- 172.21.15.7:0/1703435286 shutdown_connections 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 --2- 172.21.15.7:0/1703435286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74106570 0x7f8f7410e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 --2- 172.21.15.7:0/1703435286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74101870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 --2- 172.21.15.7:0/1703435286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f74101e40 0x7f8f74105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 -- 172.21.15.7:0/1703435286 >> 172.21.15.7:0/1703435286 conn(0x7f8f74078270 msgr2=0x7f8f74078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 -- 172.21.15.7:0/1703435286 shutdown_connections 2024-03-19T00:36:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.059+0000 7f8f7947a700 1 -- 172.21.15.7:0/1703435286 wait complete. 2024-03-19T00:36:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 Processor -- start 2024-03-19T00:36:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 -- start start 2024-03-19T00:36:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74101e40 0x7f8f74199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f74106570 0x7f8f741a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74111480 con 0x7f8f74101490 2024-03-19T00:36:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74111300 con 0x7f8f74101e40 2024-03-19T00:36:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.060+0000 7f8f7947a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74111600 con 0x7f8f74106570 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43706/0 (socket says 172.21.15.7:43706) 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 -- 172.21.15.7:0/3877576480 learned_addr learned my addr 172.21.15.7:0/3877576480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f727fc700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74101e40 0x7f8f74199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f737fe700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f74106570 0x7f8f741a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 -- 172.21.15.7:0/3877576480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f74106570 msgr2=0x7f8f741a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f74106570 0x7f8f741a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 -- 172.21.15.7:0/3877576480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74101e40 msgr2=0x7f8f74199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74101e40 0x7f8f74199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 -- 172.21.15.7:0/3877576480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f741a4ee0 con 0x7f8f74101490 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f727fc700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74101e40 0x7f8f74199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f72ffd700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74199920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f640045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64026070 con 0x7f8f74101490 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6400f460 con 0x7f8f74101490 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f741a50b0 con 0x7f8f74101490 2024-03-19T00:36:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009e60 con 0x7f8f74101490 2024-03-19T00:36:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.061+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741a5500 con 0x7f8f74101490 2024-03-19T00:36:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.062+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f74068fd0 con 0x7f8f74101490 2024-03-19T00:36:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.063+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f8f64029070 con 0x7f8f74101490 2024-03-19T00:36:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.064+0000 7f8f5bfff700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8f5c0a2890 0x7f8f5c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.064+0000 7f8f727fc700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8f5c0a2890 0x7f8f5c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.064+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8f640cf4c0 con 0x7f8f74101490 2024-03-19T00:36:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.064+0000 7f8f727fc700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8f5c0a2890 0x7f8f5c0a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f74100e70 tx=0x7f8f68009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.068+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8f6405d740 con 0x7f8f74101490 2024-03-19T00:36:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.320+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8f7402dbb0 con 0x7f8f74101490 2024-03-19T00:36:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.320+0000 7f8f5bfff700 1 -- 172.21.15.7:0/3877576480 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f8f64013070 con 0x7f8f74101490 2024-03-19T00:36:09.321 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-19T00:36:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8f5c0a2890 msgr2=0x7f8f5c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8f5c0a2890 0x7f8f5c0a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f74100e70 tx=0x7f8f68009190 comp rx=0 tx=0).stop 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 msgr2=0x7f8f74199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74199920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f640045e0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 shutdown_connections 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8f5c0a2890 0x7f8f5c0a4d50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f74106570 0x7f8f741a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f74101490 0x7f8f74199920 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 --2- 172.21.15.7:0/3877576480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f74101e40 0x7f8f74199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.322+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 >> 172.21.15.7:0/3877576480 conn(0x7f8f74078270 msgr2=0x7f8f74105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.323+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 shutdown_connections 2024-03-19T00:36:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:09.323+0000 7f8f7947a700 1 -- 172.21.15.7:0/3877576480 wait complete. 2024-03-19T00:36:09.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:09 smithi204 ceph-mon[66572]: pgmap v295: 129 pgs: 112 active+clean, 17 creating+peering; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 4 op/s 2024-03-19T00:36:09.605 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:09 smithi044 ceph-mon[66366]: pgmap v295: 129 pgs: 112 active+clean, 17 creating+peering; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 4 op/s 2024-03-19T00:36:09.710 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:09 smithi007 ceph-mon[62854]: pgmap v295: 129 pgs: 112 active+clean, 17 creating+peering; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 4 op/s 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-19T00:36:10.061 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-19T00:36:10.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:10.062 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-19T00:36:10.079 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 40629sec preferred_lft 40629sec 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T00:36:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-03-19T00:36:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-03-19T00:36:10.081 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-19T00:36:10.081 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:36:10.081 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:36:10.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-03-19T00:36:10.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-03-19T00:36:10.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-03-19T00:36:10.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-19T00:36:10.082 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:36:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:36:10.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3877576480' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T00:36:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:10.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:10.311 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-19T00:36:10.339 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-03-19T00:36:10.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:10.341 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-03-19T00:36:10.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:10 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3877576480' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T00:36:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-03-19T00:36:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-19T00:36:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-19T00:36:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-19T00:36:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-19T00:36:10.519 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:36:10.519 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:36:10.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-19T00:36:10.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-19T00:36:10.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-19T00:36:10.519 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:36:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:36:10.605 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:10 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3877576480' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T00:36:10.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:10.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:10.815 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-19T00:36:10.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:10.830 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-03-19T00:36:10.884 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:36:10.884 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:36:10.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T00:36:10.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-19T00:36:10.884 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:36:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:36:11.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:11.006 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-19T00:36:11.007 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:36:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:36:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:11.143 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:36:11.143 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:36:11.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-03-19T00:36:11.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-03-19T00:36:11.143 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:36:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:36:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:11.224 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-19T00:36:11.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T00:36:11.225 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:11.246 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-19T00:36:11.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T00:36:11.246 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:11.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-03-19T00:36:11.368 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-03-19T00:36:11.424 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-03-19T00:36:11.425 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/eec6309f9aca95b52ece73994d8f5d8b19c016041c13a5d45577041ea9cf572b/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7e5bdd392ddb07b923686c020cd726eb8fd317947179f61b0ac1a40d7484cd0b/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d9a7c5ec22cb784f8af7efb9e6bb98977c51267bea00372c35bce5265d7204f2/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/b06edf6b7387b2971e764045f906446479b51a30e663b6dc282089af447ed081/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/32ff72bd23cb8453db9034251ada47f5b3f7f0ca02b79e5fc8329e3d2b7a7204/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/6a03db7480c6af03d4d2e805849736a351c69aab279a0c4a53db007a91fe6f02/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/346b7b69993fe11455a1b65856b99e55508617c5081222e7f146a136f8729de1/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/1ba5f749cb32564e79be73311e5836c1e1fb86dfcb258c79c6f2286ac02e6bb7/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0f9dd57b3e832766d06a403a3b37144dc77bf5b27b2c821d4e450fed440a939a/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/748dacd042f2c81e25c54a6c38828b7b5b65def2a6cb9624d4b39f23c50d803f/merged 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T00:36:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T00:36:11.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:11.427 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-03-19T00:36:11.478 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-19T00:36:11.479 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-19T00:36:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:11 smithi204 ceph-mon[66572]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-03-19T00:36:11.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-03-19T00:36:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:11 smithi007 ceph-mon[62854]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-03-19T00:36:11.547 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T00:36:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-03-19T00:36:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/eec6309f9aca95b52ece73994d8f5d8b19c016041c13a5d45577041ea9cf572b/merged 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7e5bdd392ddb07b923686c020cd726eb8fd317947179f61b0ac1a40d7484cd0b/merged 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-03-19T00:36:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d9a7c5ec22cb784f8af7efb9e6bb98977c51267bea00372c35bce5265d7204f2/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/b06edf6b7387b2971e764045f906446479b51a30e663b6dc282089af447ed081/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/32ff72bd23cb8453db9034251ada47f5b3f7f0ca02b79e5fc8329e3d2b7a7204/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/6a03db7480c6af03d4d2e805849736a351c69aab279a0c4a53db007a91fe6f02/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/346b7b69993fe11455a1b65856b99e55508617c5081222e7f146a136f8729de1/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/1ba5f749cb32564e79be73311e5836c1e1fb86dfcb258c79c6f2286ac02e6bb7/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0f9dd57b3e832766d06a403a3b37144dc77bf5b27b2c821d4e450fed440a939a/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/748dacd042f2c81e25c54a6c38828b7b5b65def2a6cb9624d4b39f23c50d803f/merged 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T00:36:11.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:11.605 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-03-19T00:36:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:11 smithi044 ceph-mon[66366]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-03-19T00:36:11.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2024-03-19T00:36:11.616+0000 7f99f140d480 -1 init, newargv = 0x56082705d160 newargc=15 2024-03-19T00:36:11.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[103537]: starting ceph client 2024-03-19T00:36:11.623 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[103537]: starting fuse 2024-03-19T00:36:11.632 INFO:teuthology.orchestra.run.smithi007.stdout:119 2024-03-19T00:36:11.632 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-19T00:36:11.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-03-19T00:36:11.632 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-19T00:36:11.633 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2024-03-19T00:36:11.634 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-03-19T00:36:11.753 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.103537.asok 2024-03-19T00:36:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:11.763 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.103537.asok 2024-03-19T00:36:11.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:11.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.103537.asok status 2024-03-19T00:36:11.901 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:36:11.901 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-03-19T00:36:11.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "b6ae24918b03b1b8a19c9263a50655bba80bb810", 2024-03-19T00:36:11.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:36:11.901 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2024-03-19T00:36:11.901 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "103537", 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 14790, 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 14790 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-19T00:36:11.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1920351798 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1920351798 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.14790 192.168.0.1:0/1920351798", 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1920351798", 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 17, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 89, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-03-19T00:36:11.903 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-03-19T00:36:11.904 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:36:11.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:11.909 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-03-19T00:36:11.910 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:11.924 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-03-19T00:36:11.924 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:11.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:11.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:11.992 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-19T00:36:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T00:36:12.102 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2024-03-19T00:36:12.102 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-19T00:36:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:12.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:36:12.108 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:36:12.108 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T00:36:12.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 -- 172.21.15.7:0/605435905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 msgr2=0x7f3710108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 --2- 172.21.15.7:0/605435905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f3710108b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3708009900 tx=0x7f370802bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 -- 172.21.15.7:0/605435905 shutdown_connections 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 --2- 172.21.15.7:0/605435905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f3710108b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 --2- 172.21.15.7:0/605435905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710101940 0x7f3710101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 --2- 172.21.15.7:0/605435905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710100750 0x7f3710100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.270+0000 7f3715c80640 1 -- 172.21.15.7:0/605435905 >> 172.21.15.7:0/605435905 conn(0x7f37100fbcc0 msgr2=0x7f37100fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 -- 172.21.15.7:0/605435905 shutdown_connections 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 -- 172.21.15.7:0/605435905 wait complete. 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 Processor -- start 2024-03-19T00:36:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 -- start start 2024-03-19T00:36:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710100750 0x7f3710196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710101940 0x7f3710196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f371019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f371010c2f0 con 0x7f3710102300 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f371010c170 con 0x7f3710100750 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.271+0000 7f3715c80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f371010c470 con 0x7f3710101940 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f371019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f371019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43718/0 (socket says 172.21.15.7:43718) 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 -- 172.21.15.7:0/1593607660 learned_addr learned my addr 172.21.15.7:0/1593607660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370f7fe640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710100750 0x7f3710196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370effd640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710101940 0x7f3710196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 -- 172.21.15.7:0/1593607660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710101940 msgr2=0x7f3710196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710101940 0x7f3710196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 -- 172.21.15.7:0/1593607660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710100750 msgr2=0x7f3710196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710100750 0x7f3710196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 -- 172.21.15.7:0/1593607660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371019e580 con 0x7f3710102300 2024-03-19T00:36:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370f7fe640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710100750 0x7f3710196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370effd640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710101940 0x7f3710196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370ffff640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f371019df10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3708009230 tx=0x7f37080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708043020 con 0x7f3710102300 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371019e810 con 0x7f3710102300 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37080044c0 con 0x7f3710102300 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708003bc0 con 0x7f3710102300 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.272+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371019ee30 con 0x7f3710102300 2024-03-19T00:36:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.273+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37100686a0 con 0x7f3710102300 2024-03-19T00:36:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.274+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f3708046020 con 0x7f3710102300 2024-03-19T00:36:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.275+0000 7f370cff9640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36f80a2800 0x7f36f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.275+0000 7f370f7fe640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36f80a2800 0x7f36f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.275+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f37080f0fe0 con 0x7f3710102300 2024-03-19T00:36:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.276+0000 7f370f7fe640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36f80a2800 0x7f36f80a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37101017a0 tx=0x7f370000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.280+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f370804d020 con 0x7f3710102300 2024-03-19T00:36:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.463+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3710062420 con 0x7f3710102300 2024-03-19T00:36:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.465+0000 7f370cff9640 1 -- 172.21.15.7:0/1593607660 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f37080bde20 con 0x7f3710102300 2024-03-19T00:36:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:12.466 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36f80a2800 msgr2=0x7f36f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36f80a2800 0x7f36f80a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37101017a0 tx=0x7f370000d040 comp rx=0 tx=0).stop 2024-03-19T00:36:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 msgr2=0x7f371019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f371019df10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3708009230 tx=0x7f37080040c0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 shutdown_connections 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36f80a2800 0x7f36f80a4cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710102300 0x7f371019df10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710101940 0x7f3710196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 --2- 172.21.15.7:0/1593607660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3710100750 0x7f3710196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 >> 172.21.15.7:0/1593607660 conn(0x7f37100fbcc0 msgr2=0x7f3710104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 shutdown_connections 2024-03-19T00:36:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.467+0000 7f3715c80640 1 -- 172.21.15.7:0/1593607660 wait complete. 2024-03-19T00:36:12.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:36:12.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 -- 172.21.15.7:0/2370191307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0075860 msgr2=0x7f16b0073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 --2- 172.21.15.7:0/2370191307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0075860 0x7f16b0073cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009900 tx=0x7f16a002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 -- 172.21.15.7:0/2370191307 shutdown_connections 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 --2- 172.21.15.7:0/2370191307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16b0109520 0x7f16b010b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 --2- 172.21.15.7:0/2370191307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16b0074280 0x7f16b0074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 --2- 172.21.15.7:0/2370191307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0075860 0x7f16b0073cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 -- 172.21.15.7:0/2370191307 >> 172.21.15.7:0/2370191307 conn(0x7f16b00fcbd0 msgr2=0x7f16b00ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 -- 172.21.15.7:0/2370191307 shutdown_connections 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 -- 172.21.15.7:0/2370191307 wait complete. 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 Processor -- start 2024-03-19T00:36:12.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.584+0000 7f16affff640 1 -- start start 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 0x7f16b0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16b0075860 0x7f16b0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16b0109520 0x7f16b01a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16b010f350 con 0x7f16b0074280 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16b010f1d0 con 0x7f16b0109520 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f16b010f4d0 con 0x7f16b0075860 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 0x7f16b0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 0x7f16b0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43732/0 (socket says 172.21.15.7:43732) 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 -- 172.21.15.7:0/3376113247 learned_addr learned my addr 172.21.15.7:0/3376113247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16ae7fc640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16b0075860 0x7f16b0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16af7fe640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16b0109520 0x7f16b01a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 -- 172.21.15.7:0/3376113247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16b0075860 msgr2=0x7f16b0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16b0075860 0x7f16b0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 -- 172.21.15.7:0/3376113247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16b0109520 msgr2=0x7f16b01a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16b0109520 0x7f16b01a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 -- 172.21.15.7:0/3376113247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b01a16c0 con 0x7f16b0074280 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16aeffd640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 0x7f16b0199aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f16a0006010 tx=0x7f16a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0043020 con 0x7f16b0074280 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a0004440 con 0x7f16b0074280 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b01a1950 con 0x7f16b0074280 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.585+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0003bf0 con 0x7f16b0074280 2024-03-19T00:36:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.586+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b01a1ea0 con 0x7f16b0074280 2024-03-19T00:36:12.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.587+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b004f580 con 0x7f16b0074280 2024-03-19T00:36:12.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.588+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f16a0046020 con 0x7f16b0074280 2024-03-19T00:36:12.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.588+0000 7f1697fff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16980a2800 0x7f16980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.588+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f16a00f15a0 con 0x7f16b0074280 2024-03-19T00:36:12.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.588+0000 7f16ae7fc640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16980a2800 0x7f16980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.589+0000 7f16ae7fc640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16980a2800 0x7f16980a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f16a40095f0 tx=0x7f16a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:12.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.592+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f16a004d020 con 0x7f16b0074280 2024-03-19T00:36:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.775+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16b002d1f0 con 0x7f16b0074280 2024-03-19T00:36:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.775+0000 7f1697fff640 1 -- 172.21.15.7:0/3376113247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f16a00be400 con 0x7f16b0074280 2024-03-19T00:36:12.776 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16980a2800 msgr2=0x7f16980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16980a2800 0x7f16980a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f16a40095f0 tx=0x7f16a4005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 msgr2=0x7f16b0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 0x7f16b0199aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f16a0006010 tx=0x7f16a0004040 comp rx=0 tx=0).stop 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 shutdown_connections 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16980a2800 0x7f16980a4cc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16b0109520 0x7f16b01a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16b0075860 0x7f16b0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 --2- 172.21.15.7:0/3376113247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b0074280 0x7f16b0199aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 >> 172.21.15.7:0/3376113247 conn(0x7f16b00fcbd0 msgr2=0x7f16b0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 shutdown_connections 2024-03-19T00:36:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.777+0000 7f16affff640 1 -- 172.21.15.7:0/3376113247 wait complete. 2024-03-19T00:36:12.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:36:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 -- 172.21.15.7:0/628052419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 msgr2=0x7fd874102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 --2- 172.21.15.7:0/628052419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874102ea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd868009900 tx=0x7fd86802bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 -- 172.21.15.7:0/628052419 shutdown_connections 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 --2- 172.21.15.7:0/628052419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8741033e0 0x7fd874109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 --2- 172.21.15.7:0/628052419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874102ea0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 --2- 172.21.15.7:0/628052419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd874101830 0x7fd874101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.893+0000 7fd8799bc640 1 -- 172.21.15.7:0/628052419 >> 172.21.15.7:0/628052419 conn(0x7fd8740fcda0 msgr2=0x7fd8740ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 -- 172.21.15.7:0/628052419 shutdown_connections 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 -- 172.21.15.7:0/628052419 wait complete. 2024-03-19T00:36:12.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 Processor -- start 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 -- start start 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd874101830 0x7fd874197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8741033e0 0x7fd87419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410d360 con 0x7fd874102a20 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410d1e0 con 0x7fd874101830 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.894+0000 7fd8799bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410d4e0 con 0x7fd8741033e0 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43740/0 (socket says 172.21.15.7:43740) 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 -- 172.21.15.7:0/3416637516 learned_addr learned my addr 172.21.15.7:0/3416637516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd872ffd640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd874101830 0x7fd874197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 -- 172.21.15.7:0/3416637516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8741033e0 msgr2=0x7fd87419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8741033e0 0x7fd87419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 -- 172.21.15.7:0/3416637516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd874101830 msgr2=0x7fd874197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd874101830 0x7fd874197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 -- 172.21.15.7:0/3416637516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87419f660 con 0x7fd874102a20 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8727fc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874197f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd868006010 tx=0x7fd868004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868043020 con 0x7fd874102a20 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87419f8f0 con 0x7fd874102a20 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd868004440 con 0x7fd874102a20 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868003bf0 con 0x7fd874102a20 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.895+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87419fe40 con 0x7fd874102a20 2024-03-19T00:36:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.896+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd874108bd0 con 0x7fd874102a20 2024-03-19T00:36:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.897+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fd868046020 con 0x7fd874102a20 2024-03-19T00:36:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.898+0000 7fd8789ba640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd85c0a2800 0x7fd85c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.898+0000 7fd872ffd640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd85c0a2800 0x7fd85c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.898+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd8680f16b0 con 0x7fd874102a20 2024-03-19T00:36:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.898+0000 7fd872ffd640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd85c0a2800 0x7fd85c0a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8640095f0 tx=0x7fd864005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:12.901+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd86804d020 con 0x7fd874102a20 2024-03-19T00:36:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.084+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd87402d380 con 0x7fd874102a20 2024-03-19T00:36:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.084+0000 7fd8789ba640 1 -- 172.21.15.7:0/3416637516 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fd8680be4b0 con 0x7fd874102a20 2024-03-19T00:36:13.085 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd85c0a2800 msgr2=0x7fd85c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd85c0a2800 0x7fd85c0a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8640095f0 tx=0x7fd864005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 msgr2=0x7fd874197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874197f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd868006010 tx=0x7fd868004040 comp rx=0 tx=0).stop 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 shutdown_connections 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd85c0a2800 0x7fd85c0a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8741033e0 0x7fd87419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd874102a20 0x7fd874197f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 --2- 172.21.15.7:0/3416637516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd874101830 0x7fd874197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 >> 172.21.15.7:0/3416637516 conn(0x7fd8740fcda0 msgr2=0x7fd874105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 shutdown_connections 2024-03-19T00:36:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.086+0000 7fd8799bc640 1 -- 172.21.15.7:0/3416637516 wait complete. 2024-03-19T00:36:13.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:36:13.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 -- 172.21.15.7:0/3275831817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc100740 msgr2=0x7f15cc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/3275831817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc100740 0x7f15cc100b60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009860 tx=0x7f15b402bb70 comp rx=0 tx=0).stop 2024-03-19T00:36:13.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 -- 172.21.15.7:0/3275831817 shutdown_connections 2024-03-19T00:36:13.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/3275831817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc1022f0 0x7f15cc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/3275831817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15cc101930 0x7f15cc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/3275831817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc100740 0x7f15cc100b60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 -- 172.21.15.7:0/3275831817 >> 172.21.15.7:0/3275831817 conn(0x7f15cc0fbcb0 msgr2=0x7f15cc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 -- 172.21.15.7:0/3275831817 shutdown_connections 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.202+0000 7f15d2c7e640 1 -- 172.21.15.7:0/3275831817 wait complete. 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 Processor -- start 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 -- start start 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 0x7f15cc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15cc101930 0x7f15cc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc1022f0 0x7f15cc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10c2c0 con 0x7f15cc100740 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10c140 con 0x7f15cc1022f0 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d2c7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10c440 con 0x7f15cc101930 2024-03-19T00:36:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d0a1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 0x7f15cc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d0a1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 0x7f15cc197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43754/0 (socket says 172.21.15.7:43754) 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d0a1a640 1 -- 172.21.15.7:0/2097597236 learned_addr learned my addr 172.21.15.7:0/2097597236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.203+0000 7f15d121b640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc1022f0 0x7f15cc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d0a1a640 1 -- 172.21.15.7:0/2097597236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15cc101930 msgr2=0x7f15cc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d0a1a640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15cc101930 0x7f15cc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d0a1a640 1 -- 172.21.15.7:0/2097597236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc1022f0 msgr2=0x7f15cc19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d0a1a640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc1022f0 0x7f15cc19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d0a1a640 1 -- 172.21.15.7:0/2097597236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc19f760 con 0x7f15cc100740 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d121b640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc1022f0 0x7f15cc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d0a1a640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 0x7f15cc197b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005940 tx=0x7f15b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4041020 con 0x7f15cc100740 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc19f9f0 con 0x7f15cc100740 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15b40035f0 con 0x7f15cc100740 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4039040 con 0x7f15cc100740 2024-03-19T00:36:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.204+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc19ffb0 con 0x7f15cc100740 2024-03-19T00:36:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.205+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15cc1a01e0 con 0x7f15cc100740 2024-03-19T00:36:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.206+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f15b4044020 con 0x7f15cc100740 2024-03-19T00:36:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.207+0000 7f15c9ffb640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15b80a2800 0x7f15b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.207+0000 7f15cbfff640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15b80a2800 0x7f15b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.207+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f15b40f0cc0 con 0x7f15cc100740 2024-03-19T00:36:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.207+0000 7f15cbfff640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15b80a2800 0x7f15b80a4cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f15cc199050 tx=0x7f15c4009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.211+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15b404b020 con 0x7f15cc100740 2024-03-19T00:36:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.394+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f15cc1a03b0 con 0x7f15cc100740 2024-03-19T00:36:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.395+0000 7f15c9ffb640 1 -- 172.21.15.7:0/2097597236 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f15b40bdac0 con 0x7f15cc100740 2024-03-19T00:36:13.396 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15b80a2800 msgr2=0x7f15b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15b80a2800 0x7f15b80a4cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f15cc199050 tx=0x7f15c4009250 comp rx=0 tx=0).stop 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 msgr2=0x7f15cc197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 0x7f15cc197b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005940 tx=0x7f15b40042f0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 shutdown_connections 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15b80a2800 0x7f15b80a4cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc1022f0 0x7f15cc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15cc101930 0x7f15cc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 --2- 172.21.15.7:0/2097597236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15cc100740 0x7f15cc197b40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 >> 172.21.15.7:0/2097597236 conn(0x7f15cc0fbcb0 msgr2=0x7f15cc069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 shutdown_connections 2024-03-19T00:36:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.397+0000 7f15d2c7e640 1 -- 172.21.15.7:0/2097597236 wait complete. 2024-03-19T00:36:13.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:13 smithi007 ceph-mon[62854]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-19T00:36:13.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:13 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1593607660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:13.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:13 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3376113247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:36:13.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:13 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3416637516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:36:13.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 -- 172.21.15.7:0/2575577693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818101830 msgr2=0x7ff818101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 --2- 172.21.15.7:0/2575577693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818101830 0x7ff818101c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff814009900 tx=0x7ff81402bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 -- 172.21.15.7:0/2575577693 shutdown_connections 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 --2- 172.21.15.7:0/2575577693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 0x7ff818109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 --2- 172.21.15.7:0/2575577693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff818102a20 0x7ff818102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 --2- 172.21.15.7:0/2575577693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818101830 0x7ff818101c50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 -- 172.21.15.7:0/2575577693 >> 172.21.15.7:0/2575577693 conn(0x7ff8180fcda0 msgr2=0x7ff8180ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 -- 172.21.15.7:0/2575577693 shutdown_connections 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.513+0000 7ff82027e640 1 -- 172.21.15.7:0/2575577693 wait complete. 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 Processor -- start 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 -- start start 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff818101830 0x7ff818197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 0x7ff818197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 0x7ff81819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff81810d350 con 0x7ff818102a20 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff81810d1d0 con 0x7ff8181033e0 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff82027e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff81810d4d0 con 0x7ff818101830 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 0x7ff818197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81e01a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff818101830 0x7ff818197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81e81b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 0x7ff81819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 0x7ff818197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43776/0 (socket says 172.21.15.7:43776) 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 -- 172.21.15.7:0/533670158 learned_addr learned my addr 172.21.15.7:0/533670158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 -- 172.21.15.7:0/533670158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff818101830 msgr2=0x7ff818197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff818101830 0x7ff818197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 -- 172.21.15.7:0/533670158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 msgr2=0x7ff81819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 0x7ff81819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81d819640 1 -- 172.21.15.7:0/533670158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81819f6a0 con 0x7ff818102a20 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.514+0000 7ff81e81b640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 0x7ff81819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.515+0000 7ff81d819640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 0x7ff818197fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c970 tx=0x7ff80800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.515+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808007bb0 con 0x7ff818102a20 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.515+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff808007d50 con 0x7ff818102a20 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.515+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81819f990 con 0x7ff818102a20 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.515+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80800f450 con 0x7ff818102a20 2024-03-19T00:36:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.515+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81819fe90 con 0x7ff818102a20 2024-03-19T00:36:13.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:13 smithi204 ceph-mon[66572]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-19T00:36:13.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:13 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1593607660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:13.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:13 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3376113247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:36:13.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:13 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3416637516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:36:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.516+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff818108bc0 con 0x7ff818102a20 2024-03-19T00:36:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.517+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7ff808004f20 con 0x7ff818102a20 2024-03-19T00:36:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.517+0000 7ff80f7fe640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8040a2800 0x7ff8040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.517+0000 7ff81e01a640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8040a2800 0x7ff8040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.517+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff8080ce950 con 0x7ff818102a20 2024-03-19T00:36:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.518+0000 7ff81e01a640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8040a2800 0x7ff8040a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff814005ab0 tx=0x7ff814005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:13.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.522+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff808014030 con 0x7ff818102a20 2024-03-19T00:36:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:13 smithi044 ceph-mon[66366]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-19T00:36:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:13 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1593607660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:13 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3376113247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:36:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:13 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3416637516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:36:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.705+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff818062420 con 0x7ff818102a20 2024-03-19T00:36:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.705+0000 7ff80f7fe640 1 -- 172.21.15.7:0/533670158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7ff80809b160 con 0x7ff818102a20 2024-03-19T00:36:13.706 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:36:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8040a2800 msgr2=0x7ff8040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8040a2800 0x7ff8040a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff814005ab0 tx=0x7ff814005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 msgr2=0x7ff818197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 0x7ff818197fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c970 tx=0x7ff80800ce30 comp rx=0 tx=0).stop 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 shutdown_connections 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8040a2800 0x7ff8040a4cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8181033e0 0x7ff81819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff818102a20 0x7ff818197fc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 --2- 172.21.15.7:0/533670158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff818101830 0x7ff818197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 >> 172.21.15.7:0/533670158 conn(0x7ff8180fcda0 msgr2=0x7ff818105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 shutdown_connections 2024-03-19T00:36:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.707+0000 7ff82027e640 1 -- 172.21.15.7:0/533670158 wait complete. 2024-03-19T00:36:13.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.821+0000 7f8264636640 1 -- 172.21.15.7:0/1825395712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c1033e0 msgr2=0x7f825c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.821+0000 7f8264636640 1 --2- 172.21.15.7:0/1825395712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c1033e0 0x7f825c109c70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8254009ca0 tx=0x7f825402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- 172.21.15.7:0/1825395712 shutdown_connections 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 --2- 172.21.15.7:0/1825395712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c1033e0 0x7f825c109c70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 --2- 172.21.15.7:0/1825395712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f825c102a20 0x7f825c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 --2- 172.21.15.7:0/1825395712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c101830 0x7f825c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- 172.21.15.7:0/1825395712 >> 172.21.15.7:0/1825395712 conn(0x7f825c0fcda0 msgr2=0x7f825c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- 172.21.15.7:0/1825395712 shutdown_connections 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- 172.21.15.7:0/1825395712 wait complete. 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 Processor -- start 2024-03-19T00:36:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- start start 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 0x7f825c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f825c102a20 0x7f825c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c1033e0 0x7f825c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10d360 con 0x7f825c101830 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10d1e0 con 0x7f825c1033e0 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f8264636640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10d4e0 con 0x7f825c102a20 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f82623d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 0x7f825c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f82623d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 0x7f825c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43794/0 (socket says 172.21.15.7:43794) 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.822+0000 7f82623d2640 1 -- 172.21.15.7:0/1691215397 learned_addr learned my addr 172.21.15.7:0/1691215397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f8262bd3640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c1033e0 0x7f825c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f8261bd1640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f825c102a20 0x7f825c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82623d2640 1 -- 172.21.15.7:0/1691215397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f825c102a20 msgr2=0x7f825c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82623d2640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f825c102a20 0x7f825c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82623d2640 1 -- 172.21.15.7:0/1691215397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c1033e0 msgr2=0x7f825c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82623d2640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c1033e0 0x7f825c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82623d2640 1 -- 172.21.15.7:0/1691215397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c19f650 con 0x7f825c101830 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f8262bd3640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c1033e0 0x7f825c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82623d2640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 0x7f825c197a30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f825800e9c0 tx=0x7f825800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825800cd80 con 0x7f825c101830 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82580035f0 con 0x7f825c101830 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c19f940 con 0x7f825c101830 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258010690 con 0x7f825c101830 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.823+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c19fee0 con 0x7f825c101830 2024-03-19T00:36:13.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.824+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c108b50 con 0x7f825c101830 2024-03-19T00:36:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.825+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f82580037a0 con 0x7f825c101830 2024-03-19T00:36:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.826+0000 7f82537fe640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f82480a2880 0x7f82480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.826+0000 7f8261bd1640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f82480a2880 0x7f82480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.826+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f82580cf1c0 con 0x7f825c101830 2024-03-19T00:36:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.827+0000 7f8261bd1640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f82480a2880 0x7f82480a4d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f824c005fd0 tx=0x7f824c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:13.830+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f825809c020 con 0x7f825c101830 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.026+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f825c02d1f0 con 0x7f825c101830 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.026+0000 7f82537fe640 1 -- 172.21.15.7:0/1691215397 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+577 (secure 0 0 0) 0x7f825809b920 con 0x7f825c101830 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1277095143 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4032873653 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1320706322 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:14.027 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/237878025 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:14.028 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:14.028 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:14.028 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3660465780 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:14.028 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/97837638 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f82480a2880 msgr2=0x7f82480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f82480a2880 0x7f82480a4d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f824c005fd0 tx=0x7f824c005850 comp rx=0 tx=0).stop 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 msgr2=0x7f825c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 0x7f825c197a30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f825800e9c0 tx=0x7f825800ee80 comp rx=0 tx=0).stop 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 shutdown_connections 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f82480a2880 0x7f82480a4d40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c1033e0 0x7f825c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f825c102a20 0x7f825c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 --2- 172.21.15.7:0/1691215397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825c101830 0x7f825c197a30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 >> 172.21.15.7:0/1691215397 conn(0x7f825c0fcda0 msgr2=0x7f825c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 shutdown_connections 2024-03-19T00:36:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.028+0000 7f8264636640 1 -- 172.21.15.7:0/1691215397 wait complete. 2024-03-19T00:36:14.039 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:36:14.040 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T00:36:14.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/2457804321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f88103460 msgr2=0x7f7f88109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/2457804321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f88103460 0x7f7f88109cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009ca0 tx=0x7f7f8002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/2457804321 shutdown_connections 2024-03-19T00:36:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/2457804321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f88103460 0x7f7f88109cf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/2457804321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f88102aa0 0x7f7f88102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/2457804321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f881018b0 0x7f7f88101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/2457804321 >> 172.21.15.7:0/2457804321 conn(0x7f7f880fce40 msgr2=0x7f7f880ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/2457804321 shutdown_connections 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.197+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/2457804321 wait complete. 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 Processor -- start 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 -- start start 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 0x7f7f880755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f88102aa0 0x7f7f88073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f88103460 0x7f7f88074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8810d370 con 0x7f7f881018b0 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8810d1f0 con 0x7f7f88103460 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8cdc4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8810d4f0 con 0x7f7f88102aa0 2024-03-19T00:36:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 0x7f7f880755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 0x7f7f880755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43812/0 (socket says 172.21.15.7:43812) 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 -- 172.21.15.7:0/607899713 learned_addr learned my addr 172.21.15.7:0/607899713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f86d9d640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f88103460 0x7f7f88074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 -- 172.21.15.7:0/607899713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f88102aa0 msgr2=0x7f7f88073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f88102aa0 0x7f7f88073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 -- 172.21.15.7:0/607899713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f88103460 msgr2=0x7f7f88074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f88103460 0x7f7f88074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.198+0000 7f7f8659c640 1 -- 172.21.15.7:0/607899713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f880749e0 con 0x7f7f881018b0 2024-03-19T00:36:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.199+0000 7f7f8659c640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 0x7f7f880755d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800d950 tx=0x7f7f7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.199+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78017070 con 0x7f7f881018b0 2024-03-19T00:36:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.199+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f881a1b00 con 0x7f7f881018b0 2024-03-19T00:36:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.199+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f78009c30 con 0x7f7f881018b0 2024-03-19T00:36:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.199+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7800f620 con 0x7f7f881018b0 2024-03-19T00:36:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.199+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f881a2050 con 0x7f7f881018b0 2024-03-19T00:36:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.200+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f88108c20 con 0x7f7f881018b0 2024-03-19T00:36:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.201+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f7f7801a070 con 0x7f7f881018b0 2024-03-19T00:36:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.202+0000 7f7f7f7fe640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f740a2800 0x7f7f740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.202+0000 7f7f7ffff640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f740a2800 0x7f7f740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.202+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7f780ce6d0 con 0x7f7f881018b0 2024-03-19T00:36:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.202+0000 7f7f7ffff640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f740a2800 0x7f7f740a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f700058c0 tx=0x7f7f70005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.206+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f78009de0 con 0x7f7f881018b0 2024-03-19T00:36:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.392+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f880624c0 con 0x7f7f881018b0 2024-03-19T00:36:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.394+0000 7f7f7f7fe640 1 -- 172.21.15.7:0/607899713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f7f7809b530 con 0x7f7f881018b0 2024-03-19T00:36:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:14.395 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f740a2800 msgr2=0x7f7f740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f740a2800 0x7f7f740a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f700058c0 tx=0x7f7f70005850 comp rx=0 tx=0).stop 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 msgr2=0x7f7f880755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 0x7f7f880755d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800d950 tx=0x7f7f7800de10 comp rx=0 tx=0).stop 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 shutdown_connections 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f740a2800 0x7f7f740a4cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f88103460 0x7f7f88074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f88102aa0 0x7f7f88073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 --2- 172.21.15.7:0/607899713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f881018b0 0x7f7f880755d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 >> 172.21.15.7:0/607899713 conn(0x7f7f880fce40 msgr2=0x7f7f88105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 shutdown_connections 2024-03-19T00:36:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.396+0000 7f7f8cdc4640 1 -- 172.21.15.7:0/607899713 wait complete. 2024-03-19T00:36:14.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2097597236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:36:14.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/533670158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:36:14.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1691215397' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:36:14.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:36:14.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 -- 172.21.15.7:0/413744840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 msgr2=0x7fc6ec102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 --2- 172.21.15.7:0/413744840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec102ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 -- 172.21.15.7:0/413744840 shutdown_connections 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 --2- 172.21.15.7:0/413744840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec1033e0 0x7fc6ec109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 --2- 172.21.15.7:0/413744840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec102ea0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 --2- 172.21.15.7:0/413744840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ec101830 0x7fc6ec101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 -- 172.21.15.7:0/413744840 >> 172.21.15.7:0/413744840 conn(0x7fc6ec0fcda0 msgr2=0x7fc6ec0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 -- 172.21.15.7:0/413744840 shutdown_connections 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.513+0000 7fc6f4068640 1 -- 172.21.15.7:0/413744840 wait complete. 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 Processor -- start 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 -- start start 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ec101830 0x7fc6ec197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec1033e0 0x7fc6ec19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec10d340 con 0x7fc6ec102a20 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec10d1c0 con 0x7fc6ec101830 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f4068640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec10d4c0 con 0x7fc6ec1033e0 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33734/0 (socket says 172.21.15.7:33734) 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 -- 172.21.15.7:0/1290853591 learned_addr learned my addr 172.21.15.7:0/1290853591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1e04640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ec101830 0x7fc6ec197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f2605640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec1033e0 0x7fc6ec19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 -- 172.21.15.7:0/1290853591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec1033e0 msgr2=0x7fc6ec19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec1033e0 0x7fc6ec19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 -- 172.21.15.7:0/1290853591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ec101830 msgr2=0x7fc6ec197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ec101830 0x7fc6ec197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.514+0000 7fc6f1603640 1 -- 172.21.15.7:0/1290853591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec19f690 con 0x7fc6ec102a20 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.515+0000 7fc6f1603640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec197fb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006010 tx=0x7fc6dc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.515+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc043020 con 0x7fc6ec102a20 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.515+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec19f920 con 0x7fc6ec102a20 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.515+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc0043d0 con 0x7fc6ec102a20 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.515+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc003b80 con 0x7fc6ec102a20 2024-03-19T00:36:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.515+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec19fe70 con 0x7fc6ec102a20 2024-03-19T00:36:14.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2097597236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:36:14.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/533670158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:36:14.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1691215397' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:36:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.516+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec068af0 con 0x7fc6ec102a20 2024-03-19T00:36:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.517+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fc6dc046020 con 0x7fc6ec102a20 2024-03-19T00:36:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.518+0000 7fc6e2ffd640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc6d80a2800 0x7fc6d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.518+0000 7fc6f1e04640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc6d80a2800 0x7fc6d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.518+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc6dc0f0eb0 con 0x7fc6ec102a20 2024-03-19T00:36:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.518+0000 7fc6f1e04640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc6d80a2800 0x7fc6d80a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80095f0 tx=0x7fc6e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.522+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6dc04d020 con 0x7fc6ec102a20 2024-03-19T00:36:14.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2097597236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:36:14.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/533670158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:36:14.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1691215397' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:36:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.706+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6ec062420 con 0x7fc6ec102a20 2024-03-19T00:36:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.706+0000 7fc6e2ffd640 1 -- 172.21.15.7:0/1290853591 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fc6dc0bdd10 con 0x7fc6ec102a20 2024-03-19T00:36:14.707 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:36:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.708+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc6d80a2800 msgr2=0x7fc6d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.708+0000 7fc6f4068640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc6d80a2800 0x7fc6d80a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80095f0 tx=0x7fc6e8005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.708+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 msgr2=0x7fc6ec197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.708+0000 7fc6f4068640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec197fb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006010 tx=0x7fc6dc005e40 comp rx=0 tx=0).stop 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 shutdown_connections 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc6d80a2800 0x7fc6d80a4cc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec1033e0 0x7fc6ec19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ec102a20 0x7fc6ec197fb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 --2- 172.21.15.7:0/1290853591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ec101830 0x7fc6ec197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 >> 172.21.15.7:0/1290853591 conn(0x7fc6ec0fcda0 msgr2=0x7fc6ec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 shutdown_connections 2024-03-19T00:36:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.709+0000 7fc6f4068640 1 -- 172.21.15.7:0/1290853591 wait complete. 2024-03-19T00:36:14.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:36:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 -- 172.21.15.7:0/3375207143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4102360 msgr2=0x7f0de4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 --2- 172.21.15.7:0/3375207143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4102360 0x7f0de4104750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009860 tx=0x7f0ddc02bb70 comp rx=0 tx=0).stop 2024-03-19T00:36:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 -- 172.21.15.7:0/3375207143 shutdown_connections 2024-03-19T00:36:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 --2- 172.21.15.7:0/3375207143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de41075c0 0x7f0de41099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 --2- 172.21.15.7:0/3375207143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de4104c90 0x7f0de4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 --2- 172.21.15.7:0/3375207143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4102360 0x7f0de4104750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 -- 172.21.15.7:0/3375207143 >> 172.21.15.7:0/3375207143 conn(0x7f0de40fbcb0 msgr2=0x7f0de40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 -- 172.21.15.7:0/3375207143 shutdown_connections 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 -- 172.21.15.7:0/3375207143 wait complete. 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 Processor -- start 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.825+0000 7f0deb777640 1 -- start start 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de4102360 0x7f0de4101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4104c90 0x7f0de4100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 0x7f0de4100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0de410d380 con 0x7f0de41075c0 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0de410d200 con 0x7f0de4104c90 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0de410d500 con 0x7f0de4102360 2024-03-19T00:36:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 0x7f0de4100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 0x7f0de4100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33756/0 (socket says 172.21.15.7:33756) 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 -- 172.21.15.7:0/1601281040 learned_addr learned my addr 172.21.15.7:0/1601281040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de8d12640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4104c90 0x7f0de4100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9513640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de4102360 0x7f0de4101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 -- 172.21.15.7:0/1601281040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de4102360 msgr2=0x7f0de4101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de4102360 0x7f0de4101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 -- 172.21.15.7:0/1601281040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4104c90 msgr2=0x7f0de4100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4104c90 0x7f0de4100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 -- 172.21.15.7:0/1601281040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de4101110 con 0x7f0de41075c0 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de8d12640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4104c90 0x7f0de4100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9513640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de4102360 0x7f0de4101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de9d14640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 0x7f0de4100850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400eaa0 tx=0x7f0dd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400cdb0 con 0x7f0de41075c0 2024-03-19T00:36:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de41a1c40 con 0x7f0de41075c0 2024-03-19T00:36:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd4005020 con 0x7f0de41075c0 2024-03-19T00:36:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40116d0 con 0x7f0de41075c0 2024-03-19T00:36:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.826+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de41a2160 con 0x7f0de41075c0 2024-03-19T00:36:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.827+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de40fd7a0 con 0x7f0de41075c0 2024-03-19T00:36:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.828+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f0dd40051d0 con 0x7f0de41075c0 2024-03-19T00:36:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.829+0000 7f0de27fc640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dcc0a2800 0x7f0dcc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.829+0000 7f0de9513640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dcc0a2800 0x7f0dcc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.829+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0dd40cee20 con 0x7f0de41075c0 2024-03-19T00:36:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.829+0000 7f0de9513640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dcc0a2800 0x7f0dcc0a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc000c00 tx=0x7f0ddc034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:14.833+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0dd4016030 con 0x7f0de41075c0 2024-03-19T00:36:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.018+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0de40624c0 con 0x7f0de41075c0 2024-03-19T00:36:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.018+0000 7f0de27fc640 1 -- 172.21.15.7:0/1601281040 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0dd409bc80 con 0x7f0de41075c0 2024-03-19T00:36:15.019 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.020+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dcc0a2800 msgr2=0x7f0dcc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.020+0000 7f0deb777640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dcc0a2800 0x7f0dcc0a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc000c00 tx=0x7f0ddc034980 comp rx=0 tx=0).stop 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.020+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 msgr2=0x7f0de4100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.020+0000 7f0deb777640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 0x7f0de4100850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400eaa0 tx=0x7f0dd400ef60 comp rx=0 tx=0).stop 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 shutdown_connections 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dcc0a2800 0x7f0dcc0a4cc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de41075c0 0x7f0de4100850 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de4104c90 0x7f0de4100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 --2- 172.21.15.7:0/1601281040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de4102360 0x7f0de4101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 >> 172.21.15.7:0/1601281040 conn(0x7f0de40fbcb0 msgr2=0x7f0de4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 shutdown_connections 2024-03-19T00:36:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.021+0000 7f0deb777640 1 -- 172.21.15.7:0/1601281040 wait complete. 2024-03-19T00:36:15.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 -- 172.21.15.7:0/1595894877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8101830 msgr2=0x7f01c8101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 --2- 172.21.15.7:0/1595894877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8101830 0x7f01c8101c50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009900 tx=0x7f01c002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 -- 172.21.15.7:0/1595894877 shutdown_connections 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 --2- 172.21.15.7:0/1595894877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c81033e0 0x7f01c8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 --2- 172.21.15.7:0/1595894877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c8102a20 0x7f01c8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 --2- 172.21.15.7:0/1595894877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8101830 0x7f01c8101c50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 -- 172.21.15.7:0/1595894877 >> 172.21.15.7:0/1595894877 conn(0x7f01c80fcda0 msgr2=0x7f01c80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 -- 172.21.15.7:0/1595894877 shutdown_connections 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.137+0000 7f01d024b640 1 -- 172.21.15.7:0/1595894877 wait complete. 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 Processor -- start 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 -- start start 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 0x7f01c81958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8102a20 0x7f01c8195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c81033e0 0x7f01c819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01c810b1d0 con 0x7f01c8101830 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01c810b050 con 0x7f01c8102a20 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01d024b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01c810b350 con 0x7f01c81033e0 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cdfe7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 0x7f01c81958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8102a20 0x7f01c8195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cdfe7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 0x7f01c81958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33780/0 (socket says 172.21.15.7:33780) 2024-03-19T00:36:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cdfe7640 1 -- 172.21.15.7:0/2558417519 learned_addr learned my addr 172.21.15.7:0/2558417519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01ce7e8640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c81033e0 0x7f01c819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 -- 172.21.15.7:0/2558417519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c81033e0 msgr2=0x7f01c819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c81033e0 0x7f01c819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 -- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 msgr2=0x7f01c81958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 0x7f01c81958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 -- 172.21.15.7:0/2558417519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c819d4d0 con 0x7f01c8102a20 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cdfe7640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 0x7f01c81958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01ce7e8640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c81033e0 0x7f01c819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.138+0000 7f01cd7e6640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8102a20 0x7f01c8195df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9f0 tx=0x7f01bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.139+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00ccc0 con 0x7f01c8102a20 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.139+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c819d7c0 con 0x7f01c8102a20 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.139+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01bc005020 con 0x7f01c8102a20 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.139+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc004a00 con 0x7f01c8102a20 2024-03-19T00:36:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.139+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c819dcc0 con 0x7f01c8102a20 2024-03-19T00:36:15.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.140+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8108c20 con 0x7f01c8102a20 2024-03-19T00:36:15.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.141+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f01bc004ba0 con 0x7f01c8102a20 2024-03-19T00:36:15.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.141+0000 7f01c6ffd640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01b00a2800 0x7f01b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.141+0000 7f01cdfe7640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01b00a2800 0x7f01b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.141+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f01bc0cec70 con 0x7f01c8102a20 2024-03-19T00:36:15.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.142+0000 7f01cdfe7640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01b00a2800 0x7f01b00a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01c00096c0 tx=0x7f01c002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:15.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.145+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f01bc09bad0 con 0x7f01c8102a20 2024-03-19T00:36:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.330+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01c802d440 con 0x7f01c8102a20 2024-03-19T00:36:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.331+0000 7f01c6ffd640 1 -- 172.21.15.7:0/2558417519 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f01bc09b3d0 con 0x7f01c8102a20 2024-03-19T00:36:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:36:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01b00a2800 msgr2=0x7f01b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01b00a2800 0x7f01b00a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01c00096c0 tx=0x7f01c002c7e0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8102a20 msgr2=0x7f01c8195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8102a20 0x7f01c8195df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9f0 tx=0x7f01bc00eeb0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 shutdown_connections 2024-03-19T00:36:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01b00a2800 0x7f01b00a4cc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01c81033e0 0x7f01c819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c8102a20 0x7f01c8195df0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 --2- 172.21.15.7:0/2558417519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01c8101830 0x7f01c81958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 >> 172.21.15.7:0/2558417519 conn(0x7f01c80fcda0 msgr2=0x7f01c8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 shutdown_connections 2024-03-19T00:36:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.333+0000 7f01d024b640 1 -- 172.21.15.7:0/2558417519 wait complete. 2024-03-19T00:36:15.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:15 smithi007 ceph-mon[62854]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 314 B/s wr, 0 op/s 2024-03-19T00:36:15.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/607899713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:15.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1290853591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:36:15.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1601281040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:36:15.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:36:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 -- 172.21.15.7:0/1779891915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37581033e0 msgr2=0x7f3758109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 --2- 172.21.15.7:0/1779891915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37581033e0 0x7f3758109c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3748009ca0 tx=0x7f374802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 -- 172.21.15.7:0/1779891915 shutdown_connections 2024-03-19T00:36:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 --2- 172.21.15.7:0/1779891915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37581033e0 0x7f3758109c70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 --2- 172.21.15.7:0/1779891915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 0x7f3758102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 --2- 172.21.15.7:0/1779891915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3758101830 0x7f3758101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 -- 172.21.15.7:0/1779891915 >> 172.21.15.7:0/1779891915 conn(0x7f37580fcda0 msgr2=0x7f37580ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 -- 172.21.15.7:0/1779891915 shutdown_connections 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.449+0000 7f375d26c640 1 -- 172.21.15.7:0/1779891915 wait complete. 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 Processor -- start 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 -- start start 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 0x7f3758197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 0x7f3758197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37581033e0 0x7f375819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f375810d360 con 0x7f3758101830 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f375810d1e0 con 0x7f3758102a20 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f375d26c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f375810d4e0 con 0x7f37581033e0 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 0x7f3758197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 0x7f3758197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33796/0 (socket says 172.21.15.7:33796) 2024-03-19T00:36:15.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f37567fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 0x7f3758197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 -- 172.21.15.7:0/751311067 learned_addr learned my addr 172.21.15.7:0/751311067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f37577fe640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37581033e0 0x7f375819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 -- 172.21.15.7:0/751311067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37581033e0 msgr2=0x7f375819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37581033e0 0x7f375819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 -- 172.21.15.7:0/751311067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 msgr2=0x7f3758197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 0x7f3758197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f3756ffd640 1 -- 172.21.15.7:0/751311067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375819f650 con 0x7f3758101830 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.450+0000 7f37567fc640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 0x7f3758197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.451+0000 7f3756ffd640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 0x7f3758197a30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f375000e9c0 tx=0x7f375000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.451+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375000cd80 con 0x7f3758101830 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.451+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375819f940 con 0x7f3758101830 2024-03-19T00:36:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.451+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37500035f0 con 0x7f3758101830 2024-03-19T00:36:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.451+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750010690 con 0x7f3758101830 2024-03-19T00:36:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.451+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375819fe90 con 0x7f3758101830 2024-03-19T00:36:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.452+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3758108ba0 con 0x7f3758101830 2024-03-19T00:36:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.453+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f37500037a0 con 0x7f3758101830 2024-03-19T00:36:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.453+0000 7f374ffff640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f37440a2800 0x7f37440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.454+0000 7f37567fc640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f37440a2800 0x7f37440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.454+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f37500cf3c0 con 0x7f3758101830 2024-03-19T00:36:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.454+0000 7f37567fc640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f37440a2800 0x7f37440a4cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3758198f40 tx=0x7f3740005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:15.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.457+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f375009c220 con 0x7f3758101830 2024-03-19T00:36:15.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:15 smithi204 ceph-mon[66572]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 314 B/s wr, 0 op/s 2024-03-19T00:36:15.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/607899713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:15.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1290853591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:36:15.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1601281040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:36:15.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:15 smithi044 ceph-mon[66366]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 314 B/s wr, 0 op/s 2024-03-19T00:36:15.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/607899713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:15.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1290853591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:36:15.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1601281040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:36:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.642+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f375802d1f0 con 0x7f3758101830 2024-03-19T00:36:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.642+0000 7f374ffff640 1 -- 172.21.15.7:0/751311067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f375009bb20 con 0x7f3758101830 2024-03-19T00:36:15.643 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:36:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f37440a2800 msgr2=0x7f37440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f37440a2800 0x7f37440a4cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3758198f40 tx=0x7f3740005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 msgr2=0x7f3758197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 0x7f3758197a30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f375000e9c0 tx=0x7f375000ee80 comp rx=0 tx=0).stop 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 shutdown_connections 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f37440a2800 0x7f37440a4cc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37581033e0 0x7f375819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3758102a20 0x7f3758197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 --2- 172.21.15.7:0/751311067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3758101830 0x7f3758197a30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 >> 172.21.15.7:0/751311067 conn(0x7f37580fcda0 msgr2=0x7f3758105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 shutdown_connections 2024-03-19T00:36:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.644+0000 7f375d26c640 1 -- 172.21.15.7:0/751311067 wait complete. 2024-03-19T00:36:15.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T00:36:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 -- 172.21.15.7:0/1025912571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c101930 msgr2=0x7fc18c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 --2- 172.21.15.7:0/1025912571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c101930 0x7fc18c101db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc174009860 tx=0x7fc17402bb70 comp rx=0 tx=0).stop 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 -- 172.21.15.7:0/1025912571 shutdown_connections 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 --2- 172.21.15.7:0/1025912571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c1022f0 0x7fc18c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 --2- 172.21.15.7:0/1025912571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c101930 0x7fc18c101db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 --2- 172.21.15.7:0/1025912571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc18c100740 0x7fc18c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 -- 172.21.15.7:0/1025912571 >> 172.21.15.7:0/1025912571 conn(0x7fc18c0fbcb0 msgr2=0x7fc18c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 -- 172.21.15.7:0/1025912571 shutdown_connections 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.759+0000 7fc1909d4640 1 -- 172.21.15.7:0/1025912571 wait complete. 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 Processor -- start 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 -- start start 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc18c100740 0x7fc18c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 0x7fc18c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c1022f0 0x7fc18c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c10d390 con 0x7fc18c101930 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c10d210 con 0x7fc18c1022f0 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc1909d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c10d510 con 0x7fc18c100740 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc189d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 0x7fc18c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc189d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 0x7fc18c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33802/0 (socket says 172.21.15.7:33802) 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.760+0000 7fc189d9b640 1 -- 172.21.15.7:0/3050092657 learned_addr learned my addr 172.21.15.7:0/3050092657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc18ad9d640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c1022f0 0x7fc18c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc18a59c640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc18c100740 0x7fc18c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc189d9b640 1 -- 172.21.15.7:0/3050092657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc18c100740 msgr2=0x7fc18c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc189d9b640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc18c100740 0x7fc18c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc189d9b640 1 -- 172.21.15.7:0/3050092657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c1022f0 msgr2=0x7fc18c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc189d9b640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c1022f0 0x7fc18c19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc189d9b640 1 -- 172.21.15.7:0/3050092657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c19f6b0 con 0x7fc18c101930 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc189d9b640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 0x7fc18c198020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1740057c0 tx=0x7fc1740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174041030 con 0x7fc18c101930 2024-03-19T00:36:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c19f940 con 0x7fc18c101930 2024-03-19T00:36:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc17402c9c0 con 0x7fc18c101930 2024-03-19T00:36:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174039040 con 0x7fc18c101930 2024-03-19T00:36:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.761+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c19fe30 con 0x7fc18c101930 2024-03-19T00:36:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.762+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18c107b70 con 0x7fc18c101930 2024-03-19T00:36:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.763+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fc174044070 con 0x7fc18c101930 2024-03-19T00:36:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.764+0000 7fc1837fe640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc1780a2800 0x7fc1780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.764+0000 7fc18a59c640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc1780a2800 0x7fc1780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.764+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1740f03b0 con 0x7fc18c101930 2024-03-19T00:36:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.764+0000 7fc18a59c640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc1780a2800 0x7fc1780a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc184005c40 tx=0x7fc184005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.768+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc1740bd210 con 0x7fc18c101930 2024-03-19T00:36:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.965+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc18c062420 con 0x7fc18c101930 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.966+0000 7fc1837fe640 1 -- 172.21.15.7:0/3050092657 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+577 (secure 0 0 0) 0x7fc1740bcb10 con 0x7fc18c101930 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1277095143 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4032873653 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1320706322 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/237878025 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:15.967 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:15.968 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3660465780 2024-03-20T00:28:30.106926+0000 2024-03-19T00:36:15.968 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/97837638 2024-03-20T00:28:55.916728+0000 2024-03-19T00:36:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc1780a2800 msgr2=0x7fc1780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc1780a2800 0x7fc1780a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc184005c40 tx=0x7fc184005bd0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 msgr2=0x7fc18c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 0x7fc18c198020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1740057c0 tx=0x7fc1740042f0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 shutdown_connections 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc1780a2800 0x7fc1780a4cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc18c1022f0 0x7fc18c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18c101930 0x7fc18c198020 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 --2- 172.21.15.7:0/3050092657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc18c100740 0x7fc18c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 >> 172.21.15.7:0/3050092657 conn(0x7fc18c0fbcb0 msgr2=0x7fc18c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 shutdown_connections 2024-03-19T00:36:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:15.968+0000 7fc1909d4640 1 -- 172.21.15.7:0/3050092657 wait complete. 2024-03-19T00:36:15.979 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-03-19T00:36:15.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:15.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-19T00:36:16.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[103537]: fuse finished with error 0 and tester_r 0 2024-03-19T00:36:16.007 INFO:teuthology.orchestra.run:waiting for 300 2024-03-19T00:36:16.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2558417519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:36:16.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/751311067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:36:16.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3050092657' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:36:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2558417519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:36:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/751311067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:36:16.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3050092657' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:36:16.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2558417519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:36:16.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/751311067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:36:16.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3050092657' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:36:17.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:17 smithi204 ceph-mon[66572]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 285 B/s wr, 0 op/s 2024-03-19T00:36:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:17 smithi044 ceph-mon[66366]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 285 B/s wr, 0 op/s 2024-03-19T00:36:17.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:17 smithi007 ceph-mon[62854]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 285 B/s wr, 0 op/s 2024-03-19T00:36:19.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:19 smithi204 ceph-mon[66572]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-19T00:36:19.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:19 smithi007 ceph-mon[62854]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-19T00:36:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:19 smithi044 ceph-mon[66366]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-19T00:36:21.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:21 smithi204 ceph-mon[66572]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:21 smithi044 ceph-mon[66366]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:21.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:21 smithi007 ceph-mon[62854]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:22.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-03-19T00:36:22.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:36:22.109 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-19T00:36:22.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:22.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.291+0000 7f86d82b2640 1 -- 172.21.15.7:0/2564045683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100a80 msgr2=0x7f86d0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.291+0000 7f86d82b2640 1 --2- 172.21.15.7:0/2564045683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100a80 0x7f86d0100ee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009900 tx=0x7f86c002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 -- 172.21.15.7:0/2564045683 shutdown_connections 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 --2- 172.21.15.7:0/2564045683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0101420 0x7f86d01018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 --2- 172.21.15.7:0/2564045683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100a80 0x7f86d0100ee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 --2- 172.21.15.7:0/2564045683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0100120 0x7f86d0100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 -- 172.21.15.7:0/2564045683 >> 172.21.15.7:0/2564045683 conn(0x7f86d00fbcc0 msgr2=0x7f86d00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 -- 172.21.15.7:0/2564045683 shutdown_connections 2024-03-19T00:36:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 -- 172.21.15.7:0/2564045683 wait complete. 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 Processor -- start 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.292+0000 7f86d82b2640 1 -- start start 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d82b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 0x7f86d0199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d82b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0100a80 0x7f86d019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d82b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101420 0x7f86d01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d82b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010f560 con 0x7f86d0100a80 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d82b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010f3e0 con 0x7f86d0100120 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d82b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010f6e0 con 0x7f86d0101420 2024-03-19T00:36:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d584d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0100a80 0x7f86d019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 0x7f86d0199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 0x7f86d0199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:51966/0 (socket says 172.21.15.7:51966) 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 -- 172.21.15.7:0/1784295327 learned_addr learned my addr 172.21.15.7:0/1784295327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d684f640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101420 0x7f86d01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 -- 172.21.15.7:0/1784295327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101420 msgr2=0x7f86d01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101420 0x7f86d01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 -- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0100a80 msgr2=0x7f86d019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0100a80 0x7f86d019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 -- 172.21.15.7:0/1784295327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d01a1840 con 0x7f86d0100120 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d584d640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0100a80 0x7f86d019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86d604e640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 0x7f86d0199c30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9f0 tx=0x7f86c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.293+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800cd80 con 0x7f86d0100120 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.294+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86c80054d0 con 0x7f86d0100120 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.294+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d01a1b30 con 0x7f86d0100120 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.294+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8010490 con 0x7f86d0100120 2024-03-19T00:36:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.294+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d01a2080 con 0x7f86d0100120 2024-03-19T00:36:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.295+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d010b050 con 0x7f86d0100120 2024-03-19T00:36:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.295+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f86c8005020 con 0x7f86d0100120 2024-03-19T00:36:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.296+0000 7f86c77fe640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86bc0a2880 0x7f86bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.296+0000 7f86d584d640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86bc0a2880 0x7f86bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.296+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f86c80cf000 con 0x7f86d0100120 2024-03-19T00:36:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.296+0000 7f86d584d640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86bc0a2880 0x7f86bc0a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009230 tx=0x7f86c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.301+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86c809be60 con 0x7f86d0100120 2024-03-19T00:36:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.486+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86d0062420 con 0x7f86d0100120 2024-03-19T00:36:22.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.487+0000 7f86c77fe640 1 -- 172.21.15.7:0/1784295327 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f86c809b760 con 0x7f86d0100120 2024-03-19T00:36:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86bc0a2880 msgr2=0x7f86bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86bc0a2880 0x7f86bc0a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009230 tx=0x7f86c0005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 msgr2=0x7f86d0199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 0x7f86d0199c30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9f0 tx=0x7f86c800edf0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 shutdown_connections 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86bc0a2880 0x7f86bc0a4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86d0101420 0x7f86d01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0100a80 0x7f86d019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 --2- 172.21.15.7:0/1784295327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86d0100120 0x7f86d0199c30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 >> 172.21.15.7:0/1784295327 conn(0x7f86d00fbcc0 msgr2=0x7f86d00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 shutdown_connections 2024-03-19T00:36:22.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.490+0000 7f86d82b2640 1 -- 172.21.15.7:0/1784295327 wait complete. 2024-03-19T00:36:22.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 -- 172.21.15.7:0/3053218341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff838102350 msgr2=0x7ff838104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 --2- 172.21.15.7:0/3053218341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff838102350 0x7ff838104740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff834009900 tx=0x7ff83402bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 -- 172.21.15.7:0/3053218341 shutdown_connections 2024-03-19T00:36:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 --2- 172.21.15.7:0/3053218341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8381075b0 0x7ff8381099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 --2- 172.21.15.7:0/3053218341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 0x7ff838107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 --2- 172.21.15.7:0/3053218341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff838102350 0x7ff838104740 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 -- 172.21.15.7:0/3053218341 >> 172.21.15.7:0/3053218341 conn(0x7ff8380fbce0 msgr2=0x7ff8380fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 -- 172.21.15.7:0/3053218341 shutdown_connections 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.604+0000 7ff84040c640 1 -- 172.21.15.7:0/3053218341 wait complete. 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 Processor -- start 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 -- start start 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff838102350 0x7ff838101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 0x7ff838100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 0x7ff838100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff83810d3a0 con 0x7ff8381075b0 2024-03-19T00:36:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff83810d220 con 0x7ff838102350 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff84040c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff83810d520 con 0x7ff838104c80 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e9a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 0x7ff838100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e9a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 0x7ff838100840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33844/0 (socket says 172.21.15.7:33844) 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e9a9640 1 -- 172.21.15.7:0/3708782877 learned_addr learned my addr 172.21.15.7:0/3708782877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e1a8640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff838102350 0x7ff838101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e9a9640 1 -- 172.21.15.7:0/3708782877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 msgr2=0x7ff838100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83d9a7640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 0x7ff838100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e9a9640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 0x7ff838100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.605+0000 7ff83e9a9640 1 -- 172.21.15.7:0/3708782877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff838102350 msgr2=0x7ff838101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff83e9a9640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff838102350 0x7ff838101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff83e9a9640 1 -- 172.21.15.7:0/3708782877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8381010d0 con 0x7ff8381075b0 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff83e1a8640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff838102350 0x7ff838101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff83d9a7640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 0x7ff838100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:36:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff83e9a9640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 0x7ff838100840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff83000efd0 tx=0x7ff83000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830003b70 con 0x7ff8381075b0 2024-03-19T00:36:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8381a1c00 con 0x7ff8381075b0 2024-03-19T00:36:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff830003d10 con 0x7ff8381075b0 2024-03-19T00:36:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830010980 con 0x7ff8381075b0 2024-03-19T00:36:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.606+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381a2150 con 0x7ff8381075b0 2024-03-19T00:36:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.607+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8380fd7d0 con 0x7ff8381075b0 2024-03-19T00:36:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.608+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7ff830004020 con 0x7ff8381075b0 2024-03-19T00:36:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.608+0000 7ff82f7fe640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8240a2800 0x7ff8240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.609+0000 7ff83e1a8640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8240a2800 0x7ff8240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.609+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff8300cf210 con 0x7ff8381075b0 2024-03-19T00:36:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.609+0000 7ff83e1a8640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8240a2800 0x7ff8240a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff834009230 tx=0x7ff834005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.613+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff83009c010 con 0x7ff8381075b0 2024-03-19T00:36:22.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.796+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff838062420 con 0x7ff8381075b0 2024-03-19T00:36:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.798+0000 7ff82f7fe640 1 -- 172.21.15.7:0/3708782877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7ff83009b910 con 0x7ff8381075b0 2024-03-19T00:36:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:22.799 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8240a2800 msgr2=0x7ff8240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8240a2800 0x7ff8240a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff834009230 tx=0x7ff834005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 msgr2=0x7ff838100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 0x7ff838100840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff83000efd0 tx=0x7ff83000c6a0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 shutdown_connections 2024-03-19T00:36:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff8240a2800 0x7ff8240a4cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8381075b0 0x7ff838100840 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838104c80 0x7ff838100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 --2- 172.21.15.7:0/3708782877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff838102350 0x7ff838101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 >> 172.21.15.7:0/3708782877 conn(0x7ff8380fbce0 msgr2=0x7ff838104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 shutdown_connections 2024-03-19T00:36:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.800+0000 7ff84040c640 1 -- 172.21.15.7:0/3708782877 wait complete. 2024-03-19T00:36:22.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-19T00:36:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2181962476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 msgr2=0x7f9e94101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2181962476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e94101db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009900 tx=0x7f9e8402bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2181962476 shutdown_connections 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2181962476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 0x7f9e94108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2181962476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e94101db0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2181962476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e94100740 0x7f9e94100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2181962476 >> 172.21.15.7:0/2181962476 conn(0x7f9e940fbcb0 msgr2=0x7f9e940fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2181962476 shutdown_connections 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2181962476 wait complete. 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 Processor -- start 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 -- start start 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e94100740 0x7f9e94069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.916+0000 7f9e9bb0c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e9419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9bb0c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 0x7f9e9419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9bb0c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410c250 con 0x7f9e94101930 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9bb0c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410c0d0 con 0x7f9e941022f0 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9bb0c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410c3d0 con 0x7f9e94100740 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e9419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e9419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33872/0 (socket says 172.21.15.7:33872) 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 -- 172.21.15.7:0/2971674801 learned_addr learned my addr 172.21.15.7:0/2971674801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9a0a9640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 0x7f9e9419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e998a8640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e94100740 0x7f9e94069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 -- 172.21.15.7:0/2971674801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e94100740 msgr2=0x7f9e94069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e94100740 0x7f9e94069610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 -- 172.21.15.7:0/2971674801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 msgr2=0x7f9e9419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 0x7f9e9419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 -- 172.21.15.7:0/2971674801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9419f6c0 con 0x7f9e94101930 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9a0a9640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 0x7f9e9419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e990a7640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e9419c830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e84006010 tx=0x7f9e84004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84043020 con 0x7f9e94101930 2024-03-19T00:36:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e84004440 con 0x7f9e94101930 2024-03-19T00:36:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84003a80 con 0x7f9e94101930 2024-03-19T00:36:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9419f950 con 0x7f9e94101930 2024-03-19T00:36:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.917+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9419fea0 con 0x7f9e94101930 2024-03-19T00:36:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.918+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e94107bd0 con 0x7f9e94101930 2024-03-19T00:36:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.919+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f9e84046020 con 0x7f9e94101930 2024-03-19T00:36:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.920+0000 7f9e8affd640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9e800a2800 0x7f9e800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.920+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9e840f1530 con 0x7f9e94101930 2024-03-19T00:36:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.920+0000 7f9e998a8640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9e800a2800 0x7f9e800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.920+0000 7f9e998a8640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9e800a2800 0x7f9e800a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005fd0 tx=0x7f9e9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:22.923+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e8404d020 con 0x7f9e94101930 2024-03-19T00:36:23.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.146+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9e9402d1f0 con 0x7f9e94101930 2024-03-19T00:36:23.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.267+0000 7f9e8affd640 1 -- 172.21.15.7:0/2971674801 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f9e840be390 con 0x7f9e94101930 2024-03-19T00:36:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9e800a2800 msgr2=0x7f9e800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9e800a2800 0x7f9e800a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005fd0 tx=0x7f9e9000b410 comp rx=0 tx=0).stop 2024-03-19T00:36:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 msgr2=0x7f9e9419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e9419c830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e84006010 tx=0x7f9e84004040 comp rx=0 tx=0).stop 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 shutdown_connections 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9e800a2800 0x7f9e800a4cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e941022f0 0x7f9e9419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e94101930 0x7f9e9419c830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 --2- 172.21.15.7:0/2971674801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e94100740 0x7f9e94069610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 >> 172.21.15.7:0/2971674801 conn(0x7f9e940fbcb0 msgr2=0x7f9e94104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 shutdown_connections 2024-03-19T00:36:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.269+0000 7f9e9bb0c640 1 -- 172.21.15.7:0/2971674801 wait complete. 2024-03-19T00:36:23.280 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-19T00:36:23.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-19T00:36:23.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 -- 172.21.15.7:0/1679547949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc102a60 msgr2=0x7f6cfc102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:23.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/1679547949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc102a60 0x7f6cfc102ee0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009900 tx=0x7f6ce402bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 -- 172.21.15.7:0/1679547949 shutdown_connections 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/1679547949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cfc103420 0x7f6cfc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/1679547949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc102a60 0x7f6cfc102ee0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/1679547949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 0x7f6cfc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 -- 172.21.15.7:0/1679547949 >> 172.21.15.7:0/1679547949 conn(0x7f6cfc0fcde0 msgr2=0x7f6cfc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 -- 172.21.15.7:0/1679547949 shutdown_connections 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.385+0000 7f6d01b1b640 1 -- 172.21.15.7:0/1679547949 wait complete. 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 Processor -- start 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 -- start start 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 0x7f6cfc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cfc102a60 0x7f6cfc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 0x7f6cfc19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10d3a0 con 0x7f6cfc103420 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10d220 con 0x7f6cfc101870 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6d01b1b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10d520 con 0x7f6cfc102a60 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6cfbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 0x7f6cfc19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6cfbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 0x7f6cfc19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33894/0 (socket says 172.21.15.7:33894) 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6cfbfff640 1 -- 172.21.15.7:0/3462860989 learned_addr learned my addr 172.21.15.7:0/3462860989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.386+0000 7f6cfb7fe640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 0x7f6cfc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfaffd640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cfc102a60 0x7f6cfc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfb7fe640 1 -- 172.21.15.7:0/3462860989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cfc102a60 msgr2=0x7f6cfc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfb7fe640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cfc102a60 0x7f6cfc198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfb7fe640 1 -- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 msgr2=0x7f6cfc19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfb7fe640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 0x7f6cfc19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfb7fe640 1 -- 172.21.15.7:0/3462860989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc19f6d0 con 0x7f6cfc101870 2024-03-19T00:36:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfbfff640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 0x7f6cfc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cfb7fe640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 0x7f6cfc197b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cec005fd0 tx=0x7f6cec00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec003970 con 0x7f6cfc101870 2024-03-19T00:36:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cec004030 con 0x7f6cfc101870 2024-03-19T00:36:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec010720 con 0x7f6cfc101870 2024-03-19T00:36:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc19f9c0 con 0x7f6cfc101870 2024-03-19T00:36:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.387+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc19fec0 con 0x7f6cfc101870 2024-03-19T00:36:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.388+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc108d00 con 0x7f6cfc101870 2024-03-19T00:36:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.389+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f6cec0041e0 con 0x7f6cfc101870 2024-03-19T00:36:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.389+0000 7f6cf8ff9640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6ce80a2800 0x7f6ce80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.390+0000 7f6cfaffd640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6ce80a2800 0x7f6ce80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.390+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6cec0cf1d0 con 0x7f6cfc101870 2024-03-19T00:36:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.390+0000 7f6cfaffd640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6ce80a2800 0x7f6ce80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc198fc0 tx=0x7f6ce4005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:23.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.394+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6cec09c030 con 0x7f6cfc101870 2024-03-19T00:36:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:23 smithi204 ceph-mon[66572]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1784295327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3708782877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2971674801' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T00:36:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:23 smithi044 ceph-mon[66366]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1784295327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3708782877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2971674801' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T00:36:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:23.619+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f6cfc062450 con 0x7f6cfc101870 2024-03-19T00:36:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:23 smithi007 ceph-mon[62854]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1784295327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3708782877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2971674801' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T00:36:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.274+0000 7f6cf8ff9640 1 -- 172.21.15.7:0/3462860989 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f6cec09b930 con 0x7f6cfc101870 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6ce80a2800 msgr2=0x7f6ce80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6ce80a2800 0x7f6ce80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc198fc0 tx=0x7f6ce4005be0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 msgr2=0x7f6cfc197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 0x7f6cfc197b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cec005fd0 tx=0x7f6cec00ef90 comp rx=0 tx=0).stop 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 shutdown_connections 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6ce80a2800 0x7f6ce80a4cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cfc103420 0x7f6cfc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cfc102a60 0x7f6cfc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 --2- 172.21.15.7:0/3462860989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cfc101870 0x7f6cfc197b00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.277+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 >> 172.21.15.7:0/3462860989 conn(0x7f6cfc0fcde0 msgr2=0x7f6cfc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.278+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 shutdown_connections 2024-03-19T00:36:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.278+0000 7f6d01b1b640 1 -- 172.21.15.7:0/3462860989 wait complete. 2024-03-19T00:36:24.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-19T00:36:24.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 -- 172.21.15.7:0/965529918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 msgr2=0x7f69dc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:24.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 --2- 172.21.15.7:0/965529918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc100b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009900 tx=0x7f69cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 -- 172.21.15.7:0/965529918 shutdown_connections 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 --2- 172.21.15.7:0/965529918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc1022f0 0x7f69dc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 --2- 172.21.15.7:0/965529918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69dc101930 0x7f69dc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 --2- 172.21.15.7:0/965529918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc100b60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 -- 172.21.15.7:0/965529918 >> 172.21.15.7:0/965529918 conn(0x7f69dc0fbcb0 msgr2=0x7f69dc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 -- 172.21.15.7:0/965529918 shutdown_connections 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.405+0000 7f69e08cc640 1 -- 172.21.15.7:0/965529918 wait complete. 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 Processor -- start 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 -- start start 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:24.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc101930 0x7f69dc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69dc1022f0 0x7f69dc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc10d390 con 0x7f69dc100740 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc10d210 con 0x7f69dc101930 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69e08cc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc10d510 con 0x7f69dc1022f0 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69da59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69da59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33922/0 (socket says 172.21.15.7:33922) 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69da59c640 1 -- 172.21.15.7:0/527641197 learned_addr learned my addr 172.21.15.7:0/527641197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69d1d9b640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc101930 0x7f69dc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.406+0000 7f69dad9d640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69dc1022f0 0x7f69dc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69da59c640 1 -- 172.21.15.7:0/527641197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69dc1022f0 msgr2=0x7f69dc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69da59c640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69dc1022f0 0x7f69dc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69da59c640 1 -- 172.21.15.7:0/527641197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc101930 msgr2=0x7f69dc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69da59c640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc101930 0x7f69dc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69da59c640 1 -- 172.21.15.7:0/527641197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc19f6c0 con 0x7f69dc100740 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69da59c640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc197aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c00 tx=0x7f69cc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc043020 con 0x7f69dc100740 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc19f950 con 0x7f69dc100740 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69cc004440 con 0x7f69dc100740 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc003c90 con 0x7f69dc100740 2024-03-19T00:36:24.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.407+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc19fef0 con 0x7f69dc100740 2024-03-19T00:36:24.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.408+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc107b30 con 0x7f69dc100740 2024-03-19T00:36:24.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.409+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f69cc046020 con 0x7f69dc100740 2024-03-19T00:36:24.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.410+0000 7f69d3fff640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2800 0x7f69c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:24.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.410+0000 7f69d1d9b640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2800 0x7f69c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:24.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.410+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f69cc0f0f60 con 0x7f69dc100740 2024-03-19T00:36:24.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.410+0000 7f69d1d9b640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2800 0x7f69c80a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69dc198fb0 tx=0x7f69c400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:24.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.414+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f69cc04d020 con 0x7f69dc100740 2024-03-19T00:36:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2971674801' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T00:36:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:24 smithi044 ceph-mon[66366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3462860989' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-19T00:36:24.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:24 smithi044 ceph-mon[66366]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-19T00:36:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:24.637+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f69dc02d1f0 con 0x7f69dc100740 2024-03-19T00:36:24.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2971674801' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T00:36:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:24 smithi007 ceph-mon[62854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3462860989' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-19T00:36:24.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:24 smithi007 ceph-mon[62854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-19T00:36:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2971674801' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T00:36:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:24 smithi204 ceph-mon[66572]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3462860989' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-19T00:36:24.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:24 smithi204 ceph-mon[66572]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-19T00:36:25.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.279+0000 7f69d3fff640 1 -- 172.21.15.7:0/527641197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f69cc0bddc0 con 0x7f69dc100740 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2800 msgr2=0x7f69c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2800 0x7f69c80a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69dc198fb0 tx=0x7f69c400a250 comp rx=0 tx=0).stop 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 msgr2=0x7f69dc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc197aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c00 tx=0x7f69cc004040 comp rx=0 tx=0).stop 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 shutdown_connections 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2800 0x7f69c80a4cc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69dc1022f0 0x7f69dc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69dc101930 0x7f69dc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 --2- 172.21.15.7:0/527641197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69dc100740 0x7f69dc197aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 >> 172.21.15.7:0/527641197 conn(0x7f69dc0fbcb0 msgr2=0x7f69dc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.282+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 shutdown_connections 2024-03-19T00:36:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.283+0000 7f69e08cc640 1 -- 172.21.15.7:0/527641197 wait complete. 2024-03-19T00:36:25.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-19T00:36:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2775614006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94104c30 msgr2=0x7f7d94107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2775614006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94104c30 0x7f7d94107020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009860 tx=0x7f7d8802bb70 comp rx=0 tx=0).stop 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2775614006 shutdown_connections 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2775614006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94107560 0x7f7d94109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2775614006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94104c30 0x7f7d94107020 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2775614006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d94102300 0x7f7d941046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2775614006 >> 172.21.15.7:0/2775614006 conn(0x7f7d940fbcb0 msgr2=0x7f7d940fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2775614006 shutdown_connections 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.411+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2775614006 wait complete. 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 Processor -- start 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 -- start start 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 0x7f7d94197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94104c30 0x7f7d94198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d94107560 0x7f7d9419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410d350 con 0x7f7d94102300 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410d1d0 con 0x7f7d94104c30 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d9a1c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410d4d0 con 0x7f7d94107560 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 0x7f7d94197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.412+0000 7f7d937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 0x7f7d94197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52446/0 (socket says 172.21.15.7:52446) 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 -- 172.21.15.7:0/2106720867 learned_addr learned my addr 172.21.15.7:0/2106720867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d92ffd640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94104c30 0x7f7d94198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d93fff640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d94107560 0x7f7d9419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 -- 172.21.15.7:0/2106720867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d94107560 msgr2=0x7f7d9419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d94107560 0x7f7d9419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 -- 172.21.15.7:0/2106720867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94104c30 msgr2=0x7f7d94198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94104c30 0x7f7d94198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 -- 172.21.15.7:0/2106720867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9419f6e0 con 0x7f7d94102300 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d92ffd640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94104c30 0x7f7d94198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d937fe640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 0x7f7d94197ac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400d950 tx=0x7f7d8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84017070 con 0x7f7d94102300 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9419f9d0 con 0x7f7d94102300 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d84009c30 con 0x7f7d94102300 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400f620 con 0x7f7d94102300 2024-03-19T00:36:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.413+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9419ff40 con 0x7f7d94102300 2024-03-19T00:36:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.414+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d940fd7a0 con 0x7f7d94102300 2024-03-19T00:36:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.415+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f7d840109a0 con 0x7f7d94102300 2024-03-19T00:36:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.416+0000 7f7d90ff9640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.416+0000 7f7d92ffd640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.416+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7d840ce920 con 0x7f7d94102300 2024-03-19T00:36:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.417+0000 7f7d92ffd640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009830 tx=0x7f7d88034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.420+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d84009de0 con 0x7f7d94102300 2024-03-19T00:36:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T00:36:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-19T00:36:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T00:36:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-19T00:36:25.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-19T00:36:25.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:25 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/527641197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T00:36:25.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:25.642+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7d9402d1f0 con 0x7f7d94102300 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-19T00:36:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/527641197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-19T00:36:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:25 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/527641197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T00:36:26.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.291+0000 7f7d90ff9640 1 -- 172.21.15.7:0/2106720867 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f7d8409b720 con 0x7f7d94102300 2024-03-19T00:36:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d7c0a2800 msgr2=0x7f7d7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009830 tx=0x7f7d88034980 comp rx=0 tx=0).stop 2024-03-19T00:36:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 msgr2=0x7f7d94197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 0x7f7d94197ac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400d950 tx=0x7f7d8400de10 comp rx=0 tx=0).stop 2024-03-19T00:36:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 shutdown_connections 2024-03-19T00:36:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d94107560 0x7f7d9419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d94104c30 0x7f7d94198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 --2- 172.21.15.7:0/2106720867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d94102300 0x7f7d94197ac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 >> 172.21.15.7:0/2106720867 conn(0x7f7d940fbcb0 msgr2=0x7f7d941041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 shutdown_connections 2024-03-19T00:36:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.294+0000 7f7d9a1c4640 1 -- 172.21.15.7:0/2106720867 wait complete. 2024-03-19T00:36:26.309 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-19T00:36:26.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-19T00:36:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 -- 172.21.15.7:0/771474456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80100a10 msgr2=0x7fcc80100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 --2- 172.21.15.7:0/771474456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80100a10 0x7fcc80100e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009860 tx=0x7fcc7002bb70 comp rx=0 tx=0).stop 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 -- 172.21.15.7:0/771474456 shutdown_connections 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 --2- 172.21.15.7:0/771474456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc80101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 --2- 172.21.15.7:0/771474456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80100a10 0x7fcc80100e70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 --2- 172.21.15.7:0/771474456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc801000b0 0x7fcc801004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 -- 172.21.15.7:0/771474456 >> 172.21.15.7:0/771474456 conn(0x7fcc800fbcd0 msgr2=0x7fcc800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 -- 172.21.15.7:0/771474456 shutdown_connections 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.420+0000 7fcc87a19640 1 -- 172.21.15.7:0/771474456 wait complete. 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 Processor -- start 2024-03-19T00:36:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 -- start start 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc801000b0 0x7fcc80199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc80100a10 0x7fcc8019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc801a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010f530 con 0x7fcc801013b0 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010f3b0 con 0x7fcc80100a10 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc87a19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010f6b0 con 0x7fcc801000b0 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc801a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc801a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52450/0 (socket says 172.21.15.7:52450) 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 -- 172.21.15.7:0/2438494885 learned_addr learned my addr 172.21.15.7:0/2438494885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc84fb4640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc80100a10 0x7fcc8019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc857b5640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc801000b0 0x7fcc80199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 -- 172.21.15.7:0/2438494885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc801000b0 msgr2=0x7fcc80199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc801000b0 0x7fcc80199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 -- 172.21.15.7:0/2438494885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc80100a10 msgr2=0x7fcc8019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.421+0000 7fcc85fb6640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc80100a10 0x7fcc8019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc85fb6640 1 -- 172.21.15.7:0/2438494885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc801a18b0 con 0x7fcc801013b0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc84fb4640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc80100a10 0x7fcc8019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc85fb6640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc801a1210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800efc0 tx=0x7fcc7800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78003970 con 0x7fcc801013b0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc78004020 con 0x7fcc801013b0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc801a1ba0 con 0x7fcc801013b0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800b940 con 0x7fcc801013b0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.422+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc801a2070 con 0x7fcc801013b0 2024-03-19T00:36:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.423+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8010ae00 con 0x7fcc801013b0 2024-03-19T00:36:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.424+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fcc7801d020 con 0x7fcc801013b0 2024-03-19T00:36:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.425+0000 7fcc767fc640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc6c0a2800 0x7fcc6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.425+0000 7fcc857b5640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc6c0a2800 0x7fcc6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.425+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fcc780cf1b0 con 0x7fcc801013b0 2024-03-19T00:36:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.425+0000 7fcc857b5640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc6c0a2800 0x7fcc6c0a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc8019ab40 tx=0x7fcc7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.428+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcc7809bfb0 con 0x7fcc801013b0 2024-03-19T00:36:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/527641197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T00:36:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] up:active 2024-03-19T00:36:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T00:36:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T00:36:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: Cluster is now healthy 2024-03-19T00:36:26.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-19T00:36:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-19T00:36:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-19T00:36:26.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2106720867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T00:36:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.637+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcc80061c60 con 0x7fcc6c0a2800 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/527641197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] up:active 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: Cluster is now healthy 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-19T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-19T00:36:26.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-19T00:36:26.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2106720867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T00:36:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.662+0000 7fcc767fc640 1 -- 172.21.15.7:0/2438494885 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc80061c60 con 0x7fcc6c0a2800 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc6c0a2800 msgr2=0x7fcc6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc6c0a2800 0x7fcc6c0a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc8019ab40 tx=0x7fcc7c005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 msgr2=0x7fcc801a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc801a1210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800efc0 tx=0x7fcc7800c800 comp rx=0 tx=0).stop 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 shutdown_connections 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc6c0a2800 0x7fcc6c0a4cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801013b0 0x7fcc801a1210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc80100a10 0x7fcc8019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 --2- 172.21.15.7:0/2438494885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc801000b0 0x7fcc80199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 >> 172.21.15.7:0/2438494885 conn(0x7fcc800fbcd0 msgr2=0x7fcc800fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:26.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 shutdown_connections 2024-03-19T00:36:26.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.664+0000 7fcc87a19640 1 -- 172.21.15.7:0/2438494885 wait complete. 2024-03-19T00:36:26.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/527641197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] up:active 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: Cluster is now healthy 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-19T00:36:26.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2106720867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T00:36:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 -- 172.21.15.7:0/4234447253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa201033e0 msgr2=0x7faa20109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 --2- 172.21.15.7:0/4234447253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa201033e0 0x7faa20109c70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faa10009ca0 tx=0x7faa1002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 -- 172.21.15.7:0/4234447253 shutdown_connections 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 --2- 172.21.15.7:0/4234447253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa201033e0 0x7faa20109c70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 --2- 172.21.15.7:0/4234447253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa20102a20 0x7faa20102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 --2- 172.21.15.7:0/4234447253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa20101830 0x7faa20101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 -- 172.21.15.7:0/4234447253 >> 172.21.15.7:0/4234447253 conn(0x7faa200fcda0 msgr2=0x7faa200ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 -- 172.21.15.7:0/4234447253 shutdown_connections 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 -- 172.21.15.7:0/4234447253 wait complete. 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 Processor -- start 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.782+0000 7faa28377640 1 -- start start 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa20101830 0x7faa20197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 0x7faa20197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa201033e0 0x7faa2019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa2010d360 con 0x7faa20102a20 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faa2010d1e0 con 0x7faa20101830 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa2010d4e0 con 0x7faa201033e0 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 0x7faa20197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 0x7faa20197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52498/0 (socket says 172.21.15.7:52498) 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 -- 172.21.15.7:0/4202303859 learned_addr learned my addr 172.21.15.7:0/4202303859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa26113640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa20101830 0x7faa20197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa26914640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa201033e0 0x7faa2019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 -- 172.21.15.7:0/4202303859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa201033e0 msgr2=0x7faa2019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa201033e0 0x7faa2019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 -- 172.21.15.7:0/4202303859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa20101830 msgr2=0x7faa20197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa20101830 0x7faa20197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 -- 172.21.15.7:0/4202303859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2019f650 con 0x7faa20102a20 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa25912640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 0x7faa20197f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa1400ea60 tx=0x7faa1400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1400cde0 con 0x7faa20102a20 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa140035f0 con 0x7faa20102a20 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2019f940 con 0x7faa20102a20 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14010690 con 0x7faa20102a20 2024-03-19T00:36:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.783+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2019fe90 con 0x7faa20102a20 2024-03-19T00:36:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.784+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa20108b50 con 0x7faa20102a20 2024-03-19T00:36:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.785+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7faa140037a0 con 0x7faa20102a20 2024-03-19T00:36:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.786+0000 7faa1f7fe640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faa080a2800 0x7faa080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.786+0000 7faa26113640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faa080a2800 0x7faa080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.786+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7faa140cf1b0 con 0x7faa20102a20 2024-03-19T00:36:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.786+0000 7faa26113640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faa080a2800 0x7faa080a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa20102880 tx=0x7faa18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:26.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:26.790+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faa14053740 con 0x7faa20102a20 2024-03-19T00:36:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:36:26.642+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:36:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:36:26.642+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:36:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:36:26.642+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:36:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:36:26.642+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:36:26.965 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:36:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:36:26.642+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:36:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.001+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faa20061c60 con 0x7faa080a2800 2024-03-19T00:36:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.023+0000 7faa1f7fe640 1 -- 172.21.15.7:0/4202303859 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa20061c60 con 0x7faa080a2800 2024-03-19T00:36:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faa080a2800 msgr2=0x7faa080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faa080a2800 0x7faa080a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa20102880 tx=0x7faa18005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 msgr2=0x7faa20197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 0x7faa20197f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa1400ea60 tx=0x7faa1400ef20 comp rx=0 tx=0).stop 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 shutdown_connections 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faa080a2800 0x7faa080a4cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa201033e0 0x7faa2019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa20102a20 0x7faa20197f70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 --2- 172.21.15.7:0/4202303859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa20101830 0x7faa20197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 >> 172.21.15.7:0/4202303859 conn(0x7faa200fcda0 msgr2=0x7faa20105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.025+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 shutdown_connections 2024-03-19T00:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.026+0000 7faa28377640 1 -- 172.21.15.7:0/4202303859 wait complete. 2024-03-19T00:36:27.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-19T00:36:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 -- 172.21.15.7:0/2648372354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 msgr2=0x7f5f1c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 --2- 172.21.15.7:0/2648372354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 0x7f5f1c108b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009ca0 tx=0x7f5f1402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 -- 172.21.15.7:0/2648372354 shutdown_connections 2024-03-19T00:36:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 --2- 172.21.15.7:0/2648372354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 0x7f5f1c108b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 --2- 172.21.15.7:0/2648372354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 --2- 172.21.15.7:0/2648372354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f1c100740 0x7f5f1c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 -- 172.21.15.7:0/2648372354 >> 172.21.15.7:0/2648372354 conn(0x7f5f1c0fbcb0 msgr2=0x7f5f1c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 -- 172.21.15.7:0/2648372354 shutdown_connections 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.144+0000 7f5f24668640 1 -- 172.21.15.7:0/2648372354 wait complete. 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 Processor -- start 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 -- start start 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f1c100740 0x7f5f1c1046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 0x7f5f1c103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c10c270 con 0x7f5f1c101930 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c10c0f0 con 0x7f5f1c1022f0 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f24668640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c10c3f0 con 0x7f5f1c100740 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52524/0 (socket says 172.21.15.7:52524) 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 -- 172.21.15.7:0/3274762350 learned_addr learned my addr 172.21.15.7:0/3274762350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f22c05640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 0x7f5f1c103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f22404640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f1c100740 0x7f5f1c1046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 -- 172.21.15.7:0/3274762350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f1c100740 msgr2=0x7f5f1c1046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f1c100740 0x7f5f1c1046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 -- 172.21.15.7:0/3274762350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 msgr2=0x7f5f1c103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 0x7f5f1c103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 -- 172.21.15.7:0/3274762350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c103ae0 con 0x7f5f1c101930 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.145+0000 7f5f21c03640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c102d10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00ea60 tx=0x7f5f0c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.146+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00cde0 con 0x7f5f1c101930 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.146+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f0c0035f0 con 0x7f5f1c101930 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.146+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c1a1b60 con 0x7f5f1c101930 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.146+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c010690 con 0x7f5f1c101930 2024-03-19T00:36:27.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.146+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c1a20b0 con 0x7f5f1c101930 2024-03-19T00:36:27.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.147+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1c1a22e0 con 0x7f5f1c101930 2024-03-19T00:36:27.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.148+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f5f0c0037a0 con 0x7f5f1c101930 2024-03-19T00:36:27.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.149+0000 7f5f137fe640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5f080a2800 0x7f5f080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.149+0000 7f5f22404640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5f080a2800 0x7f5f080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.149+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5f0c0cefa0 con 0x7f5f1c101930 2024-03-19T00:36:27.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.149+0000 7f5f22404640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5f080a2800 0x7f5f080a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c101790 tx=0x7f5f18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:27.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.153+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5f0c09bda0 con 0x7f5f1c101930 2024-03-19T00:36:27.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.367+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5f1c0fdeb0 con 0x7f5f080a2800 2024-03-19T00:36:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.371+0000 7f5f137fe640 1 -- 172.21.15.7:0/3274762350 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5f1c0fdeb0 con 0x7f5f080a2800 2024-03-19T00:36:27.372 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/1e7a6851-7e82-4590-b004-54f2635f2344 2024-03-19T00:36:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5f080a2800 msgr2=0x7f5f080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5f080a2800 0x7f5f080a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c101790 tx=0x7f5f18005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 msgr2=0x7f5f1c102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c102d10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00ea60 tx=0x7f5f0c00ef20 comp rx=0 tx=0).stop 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 shutdown_connections 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5f080a2800 0x7f5f080a4cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c1022f0 0x7f5f1c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c101930 0x7f5f1c102d10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 --2- 172.21.15.7:0/3274762350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f1c100740 0x7f5f1c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 >> 172.21.15.7:0/3274762350 conn(0x7f5f1c0fbcb0 msgr2=0x7f5f1c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 shutdown_connections 2024-03-19T00:36:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.373+0000 7f5f24668640 1 -- 172.21.15.7:0/3274762350 wait complete. 2024-03-19T00:36:27.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-19T00:36:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 -- 172.21.15.7:0/4000732461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 msgr2=0x7f2520101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 --2- 172.21.15.7:0/4000732461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520101da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f250c009900 tx=0x7f250c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 -- 172.21.15.7:0/4000732461 shutdown_connections 2024-03-19T00:36:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 --2- 172.21.15.7:0/4000732461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25201022e0 0x7f2520108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 --2- 172.21.15.7:0/4000732461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520101da0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 --2- 172.21.15.7:0/4000732461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2520100730 0x7f2520100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 -- 172.21.15.7:0/4000732461 >> 172.21.15.7:0/4000732461 conn(0x7f25200fbca0 msgr2=0x7f25200fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 -- 172.21.15.7:0/4000732461 shutdown_connections 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.491+0000 7f252713b640 1 -- 172.21.15.7:0/4000732461 wait complete. 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 Processor -- start 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 -- start start 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 0x7f2520197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25201022e0 0x7f252019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f252010c260 con 0x7f2520101920 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f252010c0e0 con 0x7f2520100730 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f252713b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f252010c3e0 con 0x7f25201022e0 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f2524ed7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 0x7f2520197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f2524ed7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 0x7f2520197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44312/0 (socket says 172.21.15.7:44312) 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f2524ed7640 1 -- 172.21.15.7:0/3176025655 learned_addr learned my addr 172.21.15.7:0/3176025655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52530/0 (socket says 172.21.15.7:52530) 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f25256d8640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25201022e0 0x7f252019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 -- 172.21.15.7:0/3176025655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25201022e0 msgr2=0x7f252019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25201022e0 0x7f252019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 -- 172.21.15.7:0/3176025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 msgr2=0x7f2520197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 0x7f2520197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 -- 172.21.15.7:0/3176025655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252019f700 con 0x7f2520101920 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f2524ed7640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 0x7f2520197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251ffff640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520198020 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f250c000c00 tx=0x7f250c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.492+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c043020 con 0x7f2520101920 2024-03-19T00:36:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.493+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f250c004350 con 0x7f2520101920 2024-03-19T00:36:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.493+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252019f990 con 0x7f2520101920 2024-03-19T00:36:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.493+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c003b50 con 0x7f2520101920 2024-03-19T00:36:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.493+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252019fee0 con 0x7f2520101920 2024-03-19T00:36:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.494+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2520068af0 con 0x7f2520101920 2024-03-19T00:36:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.495+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f250c046020 con 0x7f2520101920 2024-03-19T00:36:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.495+0000 7f251dffb640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25100a2800 0x7f25100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.495+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f250c0f0d00 con 0x7f2520101920 2024-03-19T00:36:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.495+0000 7f2524ed7640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25100a2800 0x7f25100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.496+0000 7f2524ed7640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25100a2800 0x7f25100a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f25140058c0 tx=0x7f2514005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:27.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.500+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f250c04d020 con 0x7f2520101920 2024-03-19T00:36:27.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: 2024-03-19T00:36:27.408+0000 7f85dccc4700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-19T00:36:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-mon[66366]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-19T00:36:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2106720867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T00:36:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] up:active 2024-03-19T00:36:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-03-19T00:36:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:36:27.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:27 smithi044 ceph-mon[66366]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.711+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f252006a720 con 0x7f25100a2800 2024-03-19T00:36:27.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:27 smithi007 ceph-mon[62854]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-19T00:36:27.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:27 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2106720867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T00:36:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:27 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] up:active 2024-03-19T00:36:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:27 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-03-19T00:36:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:27 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:36:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:27 smithi007 ceph-mon[62854]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:27.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.733+0000 7f251dffb640 1 -- 172.21.15.7:0/3176025655 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f252006a720 con 0x7f25100a2800 2024-03-19T00:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25100a2800 msgr2=0x7f25100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25100a2800 0x7f25100a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f25140058c0 tx=0x7f2514005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 msgr2=0x7f2520198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520198020 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f250c000c00 tx=0x7f250c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 shutdown_connections 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25100a2800 0x7f25100a4cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25201022e0 0x7f252019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520101920 0x7f2520198020 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 --2- 172.21.15.7:0/3176025655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2520100730 0x7f2520197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.735+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 >> 172.21.15.7:0/3176025655 conn(0x7f25200fbca0 msgr2=0x7f252006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.736+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 shutdown_connections 2024-03-19T00:36:27.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.736+0000 7f252713b640 1 -- 172.21.15.7:0/3176025655 wait complete. 2024-03-19T00:36:27.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-19T00:36:27.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:27 smithi204 ceph-mon[66572]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-19T00:36:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:27 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2106720867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T00:36:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:27 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] up:active 2024-03-19T00:36:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:27 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-03-19T00:36:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:27 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:36:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:27 smithi204 ceph-mon[66572]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 -- 172.21.15.7:0/4232053961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0102a20 msgr2=0x7fdbe0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 --2- 172.21.15.7:0/4232053961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0102a20 0x7fdbe0102ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009900 tx=0x7fdbd002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 -- 172.21.15.7:0/4232053961 shutdown_connections 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 --2- 172.21.15.7:0/4232053961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbe01033e0 0x7fdbe0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 --2- 172.21.15.7:0/4232053961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0102a20 0x7fdbe0102ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 --2- 172.21.15.7:0/4232053961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0101830 0x7fdbe0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 -- 172.21.15.7:0/4232053961 >> 172.21.15.7:0/4232053961 conn(0x7fdbe00fcda0 msgr2=0x7fdbe00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 -- 172.21.15.7:0/4232053961 shutdown_connections 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.854+0000 7fdbe743f640 1 -- 172.21.15.7:0/4232053961 wait complete. 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 Processor -- start 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 -- start start 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 0x7fdbe0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbe0102a20 0x7fdbe0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe01033e0 0x7fdbe019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010d340 con 0x7fdbe0101830 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010d1c0 con 0x7fdbe0102a20 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe743f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010d4c0 con 0x7fdbe01033e0 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe51db640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 0x7fdbe0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe51db640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 0x7fdbe0197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52554/0 (socket says 172.21.15.7:52554) 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe51db640 1 -- 172.21.15.7:0/3181186970 learned_addr learned my addr 172.21.15.7:0/3181186970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe59dc640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe01033e0 0x7fdbe019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.855+0000 7fdbe49da640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbe0102a20 0x7fdbe0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe51db640 1 -- 172.21.15.7:0/3181186970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe01033e0 msgr2=0x7fdbe019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe51db640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe01033e0 0x7fdbe019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe51db640 1 -- 172.21.15.7:0/3181186970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbe0102a20 msgr2=0x7fdbe0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe51db640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbe0102a20 0x7fdbe0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe51db640 1 -- 172.21.15.7:0/3181186970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe019f6a0 con 0x7fdbe0101830 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe51db640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 0x7fdbe0197a80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00e9c0 tx=0x7fdbdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00cd80 con 0x7fdbe0101830 2024-03-19T00:36:27.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbdc0035f0 con 0x7fdbe0101830 2024-03-19T00:36:27.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe019f990 con 0x7fdbe0101830 2024-03-19T00:36:27.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc010690 con 0x7fdbe0101830 2024-03-19T00:36:27.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.856+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe019feb0 con 0x7fdbe0101830 2024-03-19T00:36:27.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.857+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe0108c70 con 0x7fdbe0101830 2024-03-19T00:36:27.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.858+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fdbdc0037a0 con 0x7fdbe0101830 2024-03-19T00:36:27.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.859+0000 7fdbd67fc640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbcc0a2800 0x7fdbcc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:27.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.859+0000 7fdbe49da640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbcc0a2800 0x7fdbcc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:27.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.859+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdbdc0cf090 con 0x7fdbe0101830 2024-03-19T00:36:27.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.859+0000 7fdbe49da640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbcc0a2800 0x7fdbcc0a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009230 tx=0x7fdbd0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:27.863+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdbdc09be90 con 0x7fdbe0101830 2024-03-19T00:36:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.076+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fdbe0061c60 con 0x7fdbcc0a2800 2024-03-19T00:36:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.081+0000 7fdbd67fc640 1 -- 172.21.15.7:0/3181186970 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdbe0061c60 con 0x7fdbcc0a2800 2024-03-19T00:36:28.082 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/22c988fa-2a23-4a09-9b03-5202ec10dda6 2024-03-19T00:36:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.082+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbcc0a2800 msgr2=0x7fdbcc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.082+0000 7fdbe743f640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbcc0a2800 0x7fdbcc0a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009230 tx=0x7fdbd0005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.082+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 msgr2=0x7fdbe0197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.082+0000 7fdbe743f640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 0x7fdbe0197a80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00e9c0 tx=0x7fdbdc00ee80 comp rx=0 tx=0).stop 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 shutdown_connections 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbcc0a2800 0x7fdbcc0a4cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe01033e0 0x7fdbe019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbe0102a20 0x7fdbe0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 --2- 172.21.15.7:0/3181186970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe0101830 0x7fdbe0197a80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 >> 172.21.15.7:0/3181186970 conn(0x7fdbe00fcda0 msgr2=0x7fdbe0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 shutdown_connections 2024-03-19T00:36:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.083+0000 7fdbe743f640 1 -- 172.21.15.7:0/3181186970 wait complete. 2024-03-19T00:36:28.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:36:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 -- 172.21.15.7:0/195921675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c101940 msgr2=0x7f091c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 --2- 172.21.15.7:0/195921675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c101940 0x7f091c101dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0904009900 tx=0x7f090402bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:28.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 -- 172.21.15.7:0/195921675 shutdown_connections 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 --2- 172.21.15.7:0/195921675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c102300 0x7f091c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 --2- 172.21.15.7:0/195921675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c101940 0x7f091c101dc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 --2- 172.21.15.7:0/195921675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c100750 0x7f091c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 -- 172.21.15.7:0/195921675 >> 172.21.15.7:0/195921675 conn(0x7f091c0fbce0 msgr2=0x7f091c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 -- 172.21.15.7:0/195921675 shutdown_connections 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.196+0000 7f0920d62640 1 -- 172.21.15.7:0/195921675 wait complete. 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 Processor -- start 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 -- start start 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 0x7f091c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c101940 0x7f091c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 0x7f091c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f091c10d3a0 con 0x7f091c102300 2024-03-19T00:36:28.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f091c10d220 con 0x7f091c100750 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0920d62640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f091c10d520 con 0x7f091c101940 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 0x7f091c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 0x7f091c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 0x7f091c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52564/0 (socket says 172.21.15.7:52564) 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 0x7f091c197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44350/0 (socket says 172.21.15.7:44350) 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 -- 172.21.15.7:0/1733690104 learned_addr learned my addr 172.21.15.7:0/1733690104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f0913fff640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c101940 0x7f091c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 -- 172.21.15.7:0/1733690104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c101940 msgr2=0x7f091c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c101940 0x7f091c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 -- 172.21.15.7:0/1733690104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 msgr2=0x7f091c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 0x7f091c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091ad9d640 1 -- 172.21.15.7:0/1733690104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c19f680 con 0x7f091c102300 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.197+0000 7f091a59c640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 0x7f091c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.198+0000 7f091ad9d640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 0x7f091c19f010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f091400eb10 tx=0x7f091400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.198+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914004b20 con 0x7f091c102300 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.198+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0914004cc0 con 0x7f091c102300 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.198+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c19f970 con 0x7f091c102300 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.198+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914010690 con 0x7f091c102300 2024-03-19T00:36:28.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.198+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c19fe90 con 0x7f091c102300 2024-03-19T00:36:28.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.199+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091c107af0 con 0x7f091c102300 2024-03-19T00:36:28.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.200+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7f0914005020 con 0x7f091c102300 2024-03-19T00:36:28.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.200+0000 7f09137fe640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09080a2800 0x7f09080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.200+0000 7f091a59c640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09080a2800 0x7f09080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.200+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f09140cf360 con 0x7f091c102300 2024-03-19T00:36:28.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.201+0000 7f091a59c640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09080a2800 0x7f09080a4cc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f091c1017a0 tx=0x7f090c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.205+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f091409c1c0 con 0x7f091c102300 2024-03-19T00:36:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.430+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f091c062420 con 0x7f091c102300 2024-03-19T00:36:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.430+0000 7f09137fe640 1 -- 172.21.15.7:0/1733690104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f091409bac0 con 0x7f091c102300 2024-03-19T00:36:28.432 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:28.432 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:26.295874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:36:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09080a2800 msgr2=0x7f09080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09080a2800 0x7f09080a4cc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f091c1017a0 tx=0x7f090c005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 msgr2=0x7f091c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 0x7f091c19f010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f091400eb10 tx=0x7f091400c2d0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 shutdown_connections 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09080a2800 0x7f09080a4cc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091c102300 0x7f091c19f010 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c101940 0x7f091c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 --2- 172.21.15.7:0/1733690104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c100750 0x7f091c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 >> 172.21.15.7:0/1733690104 conn(0x7f091c0fbce0 msgr2=0x7f091c104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.432+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 shutdown_connections 2024-03-19T00:36:28.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.433+0000 7f0920d62640 1 -- 172.21.15.7:0/1733690104 wait complete. 2024-03-19T00:36:28.443 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:28 smithi007 ceph-mon[62854]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.443 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:28 smithi007 ceph-mon[62854]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.443 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:28 smithi007 ceph-mon[62854]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:36:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/247920897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8100750 msgr2=0x7fbea8100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/247920897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8100750 0x7fbea8100b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009900 tx=0x7fbea002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/247920897 shutdown_connections 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/247920897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea8102300 0x7fbea8108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/247920897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea8101940 0x7fbea8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/247920897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8100750 0x7fbea8100b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.546+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/247920897 >> 172.21.15.7:0/247920897 conn(0x7fbea80fbcc0 msgr2=0x7fbea80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/247920897 shutdown_connections 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/247920897 wait complete. 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 Processor -- start 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 -- start start 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea8100750 0x7fbea8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea8101940 0x7fbea8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 0x7fbea819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810c250 con 0x7fbea8102300 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810c0d0 con 0x7fbea8100750 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeaf5f3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810c3d0 con 0x7fbea8101940 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeadb90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 0x7fbea819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeadb90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 0x7fbea819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52586/0 (socket says 172.21.15.7:52586) 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeadb90640 1 -- 172.21.15.7:0/1075806257 learned_addr learned my addr 172.21.15.7:0/1075806257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbead38f640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea8100750 0x7fbea8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.547+0000 7fbeacb8e640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea8101940 0x7fbea8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeadb90640 1 -- 172.21.15.7:0/1075806257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea8101940 msgr2=0x7fbea8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeadb90640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea8101940 0x7fbea8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeadb90640 1 -- 172.21.15.7:0/1075806257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea8100750 msgr2=0x7fbea8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeadb90640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea8100750 0x7fbea8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeadb90640 1 -- 172.21.15.7:0/1075806257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea819f690 con 0x7fbea8102300 2024-03-19T00:36:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeadb90640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 0x7fbea819f020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800eb10 tx=0x7fbe9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98004b20 con 0x7fbea8102300 2024-03-19T00:36:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe98004cc0 con 0x7fbea8102300 2024-03-19T00:36:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98010690 con 0x7fbea8102300 2024-03-19T00:36:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea819f980 con 0x7fbea8102300 2024-03-19T00:36:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.548+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea819fe80 con 0x7fbea8102300 2024-03-19T00:36:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.549+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea81a00b0 con 0x7fbea8102300 2024-03-19T00:36:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.550+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fbe98005020 con 0x7fbea8102300 2024-03-19T00:36:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.551+0000 7fbea67fc640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe900a2800 0x7fbe900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.551+0000 7fbead38f640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe900a2800 0x7fbe900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.551+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbe980cf290 con 0x7fbea8102300 2024-03-19T00:36:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.551+0000 7fbead38f640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe900a2800 0x7fbe900a4cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005ab0 tx=0x7fbea0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.555+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbe9809c090 con 0x7fbea8102300 2024-03-19T00:36:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:28 smithi044 ceph-mon[66366]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:28 smithi044 ceph-mon[66366]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:28 smithi044 ceph-mon[66366]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.738+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbea81a0320 con 0x7fbea8102300 2024-03-19T00:36:28.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.740+0000 7fbea67fc640 1 -- 172.21.15.7:0/1075806257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7fbe9809b990 con 0x7fbea8102300 2024-03-19T00:36:28.741 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:28.741 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe900a2800 msgr2=0x7fbe900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe900a2800 0x7fbe900a4cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005ab0 tx=0x7fbea0005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 msgr2=0x7fbea819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 0x7fbea819f020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800eb10 tx=0x7fbe9800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 shutdown_connections 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe900a2800 0x7fbe900a4cc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8102300 0x7fbea819f020 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea8101940 0x7fbea8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 --2- 172.21.15.7:0/1075806257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea8100750 0x7fbea8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 >> 172.21.15.7:0/1075806257 conn(0x7fbea80fbcc0 msgr2=0x7fbea80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:28.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 shutdown_connections 2024-03-19T00:36:28.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.742+0000 7fbeaf5f3640 1 -- 172.21.15.7:0/1075806257 wait complete. 2024-03-19T00:36:28.754 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-19T00:36:28.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T00:36:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:28 smithi204 ceph-mon[66572]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:28 smithi204 ceph-mon[66572]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:28 smithi204 ceph-mon[66572]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 -- 172.21.15.7:0/1477144893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8102aa0 msgr2=0x7fe9b8102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 --2- 172.21.15.7:0/1477144893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8102aa0 0x7fe9b8102f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009900 tx=0x7fe9a002bc30 comp rx=0 tx=0).stop 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 -- 172.21.15.7:0/1477144893 shutdown_connections 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 --2- 172.21.15.7:0/1477144893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b8103460 0x7fe9b8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 --2- 172.21.15.7:0/1477144893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8102aa0 0x7fe9b8102f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 --2- 172.21.15.7:0/1477144893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b81018b0 0x7fe9b8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 -- 172.21.15.7:0/1477144893 >> 172.21.15.7:0/1477144893 conn(0x7fe9b80fce40 msgr2=0x7fe9b80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 -- 172.21.15.7:0/1477144893 shutdown_connections 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.855+0000 7fe9bd019640 1 -- 172.21.15.7:0/1477144893 wait complete. 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 Processor -- start 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 -- start start 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 0x7fe9b8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b8102aa0 0x7fe9b8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 0x7fe9b819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b810d370 con 0x7fe9b8103460 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b810d1f0 con 0x7fe9b81018b0 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9bd019640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b810d4f0 con 0x7fe9b8102aa0 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9b759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 0x7fe9b819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 0x7fe9b8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9affff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b8102aa0 0x7fe9b8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 0x7fe9b8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44390/0 (socket says 172.21.15.7:44390) 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9b6d9d640 1 -- 172.21.15.7:0/2679972977 learned_addr learned my addr 172.21.15.7:0/2679972977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.856+0000 7fe9b759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 0x7fe9b819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52606/0 (socket says 172.21.15.7:52606) 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b6d9d640 1 -- 172.21.15.7:0/2679972977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b8102aa0 msgr2=0x7fe9b8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b6d9d640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b8102aa0 0x7fe9b8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b6d9d640 1 -- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 msgr2=0x7fe9b819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b6d9d640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 0x7fe9b819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b6d9d640 1 -- 172.21.15.7:0/2679972977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b819f670 con 0x7fe9b81018b0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b759e640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 0x7fe9b819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b6d9d640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 0x7fe9b8197a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e9c0 tx=0x7fe9a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800cd80 con 0x7fe9b81018b0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b819f960 con 0x7fe9b81018b0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9a80035f0 con 0x7fe9b81018b0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8010690 con 0x7fe9b81018b0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.857+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b819fe30 con 0x7fe9b81018b0 2024-03-19T00:36:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.858+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b8108bf0 con 0x7fe9b81018b0 2024-03-19T00:36:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.859+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fe9a80037a0 con 0x7fe9b81018b0 2024-03-19T00:36:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.859+0000 7fe9b4d99640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9a40a2800 0x7fe9a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.860+0000 7fe9affff640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9a40a2800 0x7fe9a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.860+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe9a80cf190 con 0x7fe9b81018b0 2024-03-19T00:36:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.860+0000 7fe9affff640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9a40a2800 0x7fe9a40a4cc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009230 tx=0x7fe9a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:28.863+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe9a809bff0 con 0x7fe9b81018b0 2024-03-19T00:36:29.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.042+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9b8061c60 con 0x7fe9a40a2800 2024-03-19T00:36:29.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.053+0000 7fe9b4d99640 1 -- 172.21.15.7:0/2679972977 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340268 (secure 0 0 0) 0x7fe9b8061c60 con 0x7fe9a40a2800 2024-03-19T00:36:29.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:29.062 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:36:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9a40a2800 msgr2=0x7fe9a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9a40a2800 0x7fe9a40a4cc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009230 tx=0x7fe9a0005fd0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 msgr2=0x7fe9b8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 0x7fe9b8197a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e9c0 tx=0x7fe9a800ee80 comp rx=0 tx=0).stop 2024-03-19T00:36:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 shutdown_connections 2024-03-19T00:36:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9a40a2800 0x7fe9a40a4cc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9b8103460 0x7fe9b819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b8102aa0 0x7fe9b8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 --2- 172.21.15.7:0/2679972977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9b81018b0 0x7fe9b8197a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 >> 172.21.15.7:0/2679972977 conn(0x7fe9b80fce40 msgr2=0x7fe9b8105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:29.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 shutdown_connections 2024-03-19T00:36:29.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.056+0000 7fe9bd019640 1 -- 172.21.15.7:0/2679972977 wait complete. 2024-03-19T00:36:29.068 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-03-19T00:36:28.003311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595143,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":132,"num_read_kb":115,"num_write":261,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400024,"kb_used_data":13384,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541800,"statfs":{"total":1151940427776,"available":1151530803200,"internally_reserved":0,"allocated":13705216,"data_stored":7511977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":275,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":33,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003633"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307135+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:36:05.307135+0000","last_peered":"2024-03-19T00:36:05.307135+0000","last_clean":"2024-03-19T00:36:05.307135+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:36:05.307135+0000","last_undegraded":"2024-03-19T00:36:05.307135+0000","last_fullsized":"2024-03-19T00:36:05.307135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758129+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:36:05.758129+0000","last_peered":"2024-03-19T00:36:05.758129+0000","last_clean":"2024-03-19T00:36:05.758129+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:36:05.758129+0000","last_undegraded":"2024-03-19T00:36:05.758129+0000","last_fullsized":"2024-03-19T00:36:05.758129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757899+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:36:05.757899+0000","last_peered":"2024-03-19T00:36:05.757899+0000","last_clean":"2024-03-19T00:36:05.757899+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:36:05.757899+0000","last_undegraded":"2024-03-19T00:36:05.757899+0000","last_fullsized":"2024-03-19T00:36:05.757899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307052+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:36:05.307052+0000","last_peered":"2024-03-19T00:36:05.307052+0000","last_clean":"2024-03-19T00:36:05.307052+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:36:05.307052+0000","last_undegraded":"2024-03-19T00:36:05.307052+0000","last_fullsized":"2024-03-19T00:36:05.307052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306652+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:36:05.306652+0000","last_peered":"2024-03-19T00:36:05.306652+0000","last_clean":"2024-03-19T00:36:05.306652+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:36:05.306652+0000","last_undegraded":"2024-03-19T00:36:05.306652+0000","last_fullsized":"2024-03-19T00:36:05.306652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306173+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:36:05.306173+0000","last_peered":"2024-03-19T00:36:05.306173+0000","last_clean":"2024-03-19T00:36:05.306173+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:36:05.306173+0000","last_undegraded":"2024-03-19T00:36:05.306173+0000","last_fullsized":"2024-03-19T00:36:05.306173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.235902+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:36:05.235902+0000","last_peered":"2024-03-19T00:36:05.235902+0000","last_clean":"2024-03-19T00:36:05.235902+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:36:05.235902+0000","last_undegraded":"2024-03-19T00:36:05.235902+0000","last_fullsized":"2024-03-19T00:36:05.235902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306470+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:36:05.306470+0000","last_peered":"2024-03-19T00:36:05.306470+0000","last_clean":"2024-03-19T00:36:05.306470+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:36:05.306470+0000","last_undegraded":"2024-03-19T00:36:05.306470+0000","last_fullsized":"2024-03-19T00:36:05.306470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306269+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:36:05.306269+0000","last_peered":"2024-03-19T00:36:05.306269+0000","last_clean":"2024-03-19T00:36:05.306269+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:36:05.306269+0000","last_undegraded":"2024-03-19T00:36:05.306269+0000","last_fullsized":"2024-03-19T00:36:05.306269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306967+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:36:05.306967+0000","last_peered":"2024-03-19T00:36:05.306967+0000","last_clean":"2024-03-19T00:36:05.306967+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:36:05.306967+0000","last_undegraded":"2024-03-19T00:36:05.306967+0000","last_fullsized":"2024-03-19T00:36:05.306967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236390+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:36:05.236390+0000","last_peered":"2024-03-19T00:36:05.236390+0000","last_clean":"2024-03-19T00:36:05.236390+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:36:05.236390+0000","last_undegraded":"2024-03-19T00:36:05.236390+0000","last_fullsized":"2024-03-19T00:36:05.236390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306962+0000","last_change":"2024-03-19T00:36:00.219901+0000","last_active":"2024-03-19T00:36:05.306962+0000","last_peered":"2024-03-19T00:36:05.306962+0000","last_clean":"2024-03-19T00:36:05.306962+0000","last_became_active":"2024-03-19T00:36:00.219554+0000","last_became_peered":"2024-03-19T00:36:00.219554+0000","last_unstale":"2024-03-19T00:36:05.306962+0000","last_undegraded":"2024-03-19T00:36:05.306962+0000","last_fullsized":"2024-03-19T00:36:05.306962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:05:48.030960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306658+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:36:05.306658+0000","last_peered":"2024-03-19T00:36:05.306658+0000","last_clean":"2024-03-19T00:36:05.306658+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:36:05.306658+0000","last_undegraded":"2024-03-19T00:36:05.306658+0000","last_fullsized":"2024-03-19T00:36:05.306658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757657+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:36:05.757657+0000","last_peered":"2024-03-19T00:36:05.757657+0000","last_clean":"2024-03-19T00:36:05.757657+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:36:05.757657+0000","last_undegraded":"2024-03-19T00:36:05.757657+0000","last_fullsized":"2024-03-19T00:36:05.757657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757702+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:36:05.757702+0000","last_peered":"2024-03-19T00:36:05.757702+0000","last_clean":"2024-03-19T00:36:05.757702+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:36:05.757702+0000","last_undegraded":"2024-03-19T00:36:05.757702+0000","last_fullsized":"2024-03-19T00:36:05.757702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.305781+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:36:05.305781+0000","last_peered":"2024-03-19T00:36:05.305781+0000","last_clean":"2024-03-19T00:36:05.305781+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:36:05.305781+0000","last_undegraded":"2024-03-19T00:36:05.305781+0000","last_fullsized":"2024-03-19T00:36:05.305781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307029+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:36:05.307029+0000","last_peered":"2024-03-19T00:36:05.307029+0000","last_clean":"2024-03-19T00:36:05.307029+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:36:05.307029+0000","last_undegraded":"2024-03-19T00:36:05.307029+0000","last_fullsized":"2024-03-19T00:36:05.307029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234358+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:36:05.234358+0000","last_peered":"2024-03-19T00:36:05.234358+0000","last_clean":"2024-03-19T00:36:05.234358+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:36:05.234358+0000","last_undegraded":"2024-03-19T00:36:05.234358+0000","last_fullsized":"2024-03-19T00:36:05.234358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234769+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:36:05.234769+0000","last_peered":"2024-03-19T00:36:05.234769+0000","last_clean":"2024-03-19T00:36:05.234769+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:36:05.234769+0000","last_undegraded":"2024-03-19T00:36:05.234769+0000","last_fullsized":"2024-03-19T00:36:05.234769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234352+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:05.234352+0000","last_peered":"2024-03-19T00:36:05.234352+0000","last_clean":"2024-03-19T00:36:05.234352+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:05.234352+0000","last_undegraded":"2024-03-19T00:36:05.234352+0000","last_fullsized":"2024-03-19T00:36:05.234352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.296390+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:25.296390+0000","last_peered":"2024-03-19T00:36:25.296390+0000","last_clean":"2024-03-19T00:36:25.296390+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:25.296390+0000","last_undegraded":"2024-03-19T00:36:25.296390+0000","last_fullsized":"2024-03-19T00:36:25.296390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306863+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:36:05.306863+0000","last_peered":"2024-03-19T00:36:05.306863+0000","last_clean":"2024-03-19T00:36:05.306863+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:36:05.306863+0000","last_undegraded":"2024-03-19T00:36:05.306863+0000","last_fullsized":"2024-03-19T00:36:05.306863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306516+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:05.306516+0000","last_peered":"2024-03-19T00:36:05.306516+0000","last_clean":"2024-03-19T00:36:05.306516+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:05.306516+0000","last_undegraded":"2024-03-19T00:36:05.306516+0000","last_fullsized":"2024-03-19T00:36:05.306516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306991+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:05.306991+0000","last_peered":"2024-03-19T00:36:05.306991+0000","last_clean":"2024-03-19T00:36:05.306991+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:05.306991+0000","last_undegraded":"2024-03-19T00:36:05.306991+0000","last_fullsized":"2024-03-19T00:36:05.306991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758141+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:36:05.758141+0000","last_peered":"2024-03-19T00:36:05.758141+0000","last_clean":"2024-03-19T00:36:05.758141+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:36:05.758141+0000","last_undegraded":"2024-03-19T00:36:05.758141+0000","last_fullsized":"2024-03-19T00:36:05.758141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306070+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:05.306070+0000","last_peered":"2024-03-19T00:36:05.306070+0000","last_clean":"2024-03-19T00:36:05.306070+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:05.306070+0000","last_undegraded":"2024-03-19T00:36:05.306070+0000","last_fullsized":"2024-03-19T00:36:05.306070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.295188+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:25.295188+0000","last_peered":"2024-03-19T00:36:25.295188+0000","last_clean":"2024-03-19T00:36:25.295188+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:25.295188+0000","last_undegraded":"2024-03-19T00:36:25.295188+0000","last_fullsized":"2024-03-19T00:36:25.295188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306894+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:36:05.306894+0000","last_peered":"2024-03-19T00:36:05.306894+0000","last_clean":"2024-03-19T00:36:05.306894+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:36:05.306894+0000","last_undegraded":"2024-03-19T00:36:05.306894+0000","last_fullsized":"2024-03-19T00:36:05.306894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306129+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:05.306129+0000","last_peered":"2024-03-19T00:36:05.306129+0000","last_clean":"2024-03-19T00:36:05.306129+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:05.306129+0000","last_undegraded":"2024-03-19T00:36:05.306129+0000","last_fullsized":"2024-03-19T00:36:05.306129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307235+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:05.307235+0000","last_peered":"2024-03-19T00:36:05.307235+0000","last_clean":"2024-03-19T00:36:05.307235+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:05.307235+0000","last_undegraded":"2024-03-19T00:36:05.307235+0000","last_fullsized":"2024-03-19T00:36:05.307235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307565+0000","last_change":"2024-03-19T00:36:00.212990+0000","last_active":"2024-03-19T00:36:05.307565+0000","last_peered":"2024-03-19T00:36:05.307565+0000","last_clean":"2024-03-19T00:36:05.307565+0000","last_became_active":"2024-03-19T00:36:00.212662+0000","last_became_peered":"2024-03-19T00:36:00.212662+0000","last_unstale":"2024-03-19T00:36:05.307565+0000","last_undegraded":"2024-03-19T00:36:05.307565+0000","last_fullsized":"2024-03-19T00:36:05.307565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:53:24.125610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234533+0000","last_change":"2024-03-19T00:36:04.240480+0000","last_active":"2024-03-19T00:36:05.234533+0000","last_peered":"2024-03-19T00:36:05.234533+0000","last_clean":"2024-03-19T00:36:05.234533+0000","last_became_active":"2024-03-19T00:36:04.240016+0000","last_became_peered":"2024-03-19T00:36:04.240016+0000","last_unstale":"2024-03-19T00:36:05.234533+0000","last_undegraded":"2024-03-19T00:36:05.234533+0000","last_fullsized":"2024-03-19T00:36:05.234533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:27.272829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307125+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:36:05.307125+0000","last_peered":"2024-03-19T00:36:05.307125+0000","last_clean":"2024-03-19T00:36:05.307125+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:36:05.307125+0000","last_undegraded":"2024-03-19T00:36:05.307125+0000","last_fullsized":"2024-03-19T00:36:05.307125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306827+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:36:05.306827+0000","last_peered":"2024-03-19T00:36:05.306827+0000","last_clean":"2024-03-19T00:36:05.306827+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:36:05.306827+0000","last_undegraded":"2024-03-19T00:36:05.306827+0000","last_fullsized":"2024-03-19T00:36:05.306827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234608+0000","last_change":"2024-03-19T00:36:04.245145+0000","last_active":"2024-03-19T00:36:05.234608+0000","last_peered":"2024-03-19T00:36:05.234608+0000","last_clean":"2024-03-19T00:36:05.234608+0000","last_became_active":"2024-03-19T00:36:04.244563+0000","last_became_peered":"2024-03-19T00:36:04.244563+0000","last_unstale":"2024-03-19T00:36:05.234608+0000","last_undegraded":"2024-03-19T00:36:05.234608+0000","last_fullsized":"2024-03-19T00:36:05.234608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:56:32.245777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306066+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:05.306066+0000","last_peered":"2024-03-19T00:36:05.306066+0000","last_clean":"2024-03-19T00:36:05.306066+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:05.306066+0000","last_undegraded":"2024-03-19T00:36:05.306066+0000","last_fullsized":"2024-03-19T00:36:05.306066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307685+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:36:05.307685+0000","last_peered":"2024-03-19T00:36:05.307685+0000","last_clean":"2024-03-19T00:36:05.307685+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:36:05.307685+0000","last_undegraded":"2024-03-19T00:36:05.307685+0000","last_fullsized":"2024-03-19T00:36:05.307685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306899+0000","last_change":"2024-03-19T00:36:04.237487+0000","last_active":"2024-03-19T00:36:05.306899+0000","last_peered":"2024-03-19T00:36:05.306899+0000","last_clean":"2024-03-19T00:36:05.306899+0000","last_became_active":"2024-03-19T00:36:04.237048+0000","last_became_peered":"2024-03-19T00:36:04.237048+0000","last_unstale":"2024-03-19T00:36:05.306899+0000","last_undegraded":"2024-03-19T00:36:05.306899+0000","last_fullsized":"2024-03-19T00:36:05.306899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:38.844336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306396+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:05.306396+0000","last_peered":"2024-03-19T00:36:05.306396+0000","last_clean":"2024-03-19T00:36:05.306396+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:05.306396+0000","last_undegraded":"2024-03-19T00:36:05.306396+0000","last_fullsized":"2024-03-19T00:36:05.306396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306344+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:36:05.306344+0000","last_peered":"2024-03-19T00:36:05.306344+0000","last_clean":"2024-03-19T00:36:05.306344+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:36:05.306344+0000","last_undegraded":"2024-03-19T00:36:05.306344+0000","last_fullsized":"2024-03-19T00:36:05.306344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758024+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:05.758024+0000","last_peered":"2024-03-19T00:36:05.758024+0000","last_clean":"2024-03-19T00:36:05.758024+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:05.758024+0000","last_undegraded":"2024-03-19T00:36:05.758024+0000","last_fullsized":"2024-03-19T00:36:05.758024+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.308010+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:05.308010+0000","last_peered":"2024-03-19T00:36:05.308010+0000","last_clean":"2024-03-19T00:36:05.308010+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:05.308010+0000","last_undegraded":"2024-03-19T00:36:05.308010+0000","last_fullsized":"2024-03-19T00:36:05.308010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.233822+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:36:05.233822+0000","last_peered":"2024-03-19T00:36:05.233822+0000","last_clean":"2024-03-19T00:36:05.233822+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:36:05.233822+0000","last_undegraded":"2024-03-19T00:36:05.233822+0000","last_fullsized":"2024-03-19T00:36:05.233822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306947+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:05.306947+0000","last_peered":"2024-03-19T00:36:05.306947+0000","last_clean":"2024-03-19T00:36:05.306947+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:05.306947+0000","last_undegraded":"2024-03-19T00:36:05.306947+0000","last_fullsized":"2024-03-19T00:36:05.306947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306534+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:36:05.306534+0000","last_peered":"2024-03-19T00:36:05.306534+0000","last_clean":"2024-03-19T00:36:05.306534+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:36:05.306534+0000","last_undegraded":"2024-03-19T00:36:05.306534+0000","last_fullsized":"2024-03-19T00:36:05.306534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306649+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:36:05.306649+0000","last_peered":"2024-03-19T00:36:05.306649+0000","last_clean":"2024-03-19T00:36:05.306649+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:36:05.306649+0000","last_undegraded":"2024-03-19T00:36:05.306649+0000","last_fullsized":"2024-03-19T00:36:05.306649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.233866+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:05.233866+0000","last_peered":"2024-03-19T00:36:05.233866+0000","last_clean":"2024-03-19T00:36:05.233866+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:05.233866+0000","last_undegraded":"2024-03-19T00:36:05.233866+0000","last_fullsized":"2024-03-19T00:36:05.233866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.287406+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:24.287406+0000","last_peered":"2024-03-19T00:36:24.287406+0000","last_clean":"2024-03-19T00:36:24.287406+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:24.287406+0000","last_undegraded":"2024-03-19T00:36:24.287406+0000","last_fullsized":"2024-03-19T00:36:24.287406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758128+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:36:05.758128+0000","last_peered":"2024-03-19T00:36:05.758128+0000","last_clean":"2024-03-19T00:36:05.758128+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:36:05.758128+0000","last_undegraded":"2024-03-19T00:36:05.758128+0000","last_fullsized":"2024-03-19T00:36:05.758128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307326+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:05.307326+0000","last_peered":"2024-03-19T00:36:05.307326+0000","last_clean":"2024-03-19T00:36:05.307326+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:05.307326+0000","last_undegraded":"2024-03-19T00:36:05.307326+0000","last_fullsized":"2024-03-19T00:36:05.307326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307431+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:05.307431+0000","last_peered":"2024-03-19T00:36:05.307431+0000","last_clean":"2024-03-19T00:36:05.307431+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:05.307431+0000","last_undegraded":"2024-03-19T00:36:05.307431+0000","last_fullsized":"2024-03-19T00:36:05.307431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307050+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:36:05.307050+0000","last_peered":"2024-03-19T00:36:05.307050+0000","last_clean":"2024-03-19T00:36:05.307050+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:36:05.307050+0000","last_undegraded":"2024-03-19T00:36:05.307050+0000","last_fullsized":"2024-03-19T00:36:05.307050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.233950+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:05.233950+0000","last_peered":"2024-03-19T00:36:05.233950+0000","last_clean":"2024-03-19T00:36:05.233950+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:05.233950+0000","last_undegraded":"2024-03-19T00:36:05.233950+0000","last_fullsized":"2024-03-19T00:36:05.233950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'10","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:16.007959+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:36:16.007959+0000","last_peered":"2024-03-19T00:36:16.007959+0000","last_clean":"2024-03-19T00:36:16.007959+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:36:16.007959+0000","last_undegraded":"2024-03-19T00:36:16.007959+0000","last_fullsized":"2024-03-19T00:36:16.007959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3843,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307372+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:36:05.307372+0000","last_peered":"2024-03-19T00:36:05.307372+0000","last_clean":"2024-03-19T00:36:05.307372+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:36:05.307372+0000","last_undegraded":"2024-03-19T00:36:05.307372+0000","last_fullsized":"2024-03-19T00:36:05.307372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.308061+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:05.308061+0000","last_peered":"2024-03-19T00:36:05.308061+0000","last_clean":"2024-03-19T00:36:05.308061+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:05.308061+0000","last_undegraded":"2024-03-19T00:36:05.308061+0000","last_fullsized":"2024-03-19T00:36:05.308061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234609+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:05.234609+0000","last_peered":"2024-03-19T00:36:05.234609+0000","last_clean":"2024-03-19T00:36:05.234609+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:05.234609+0000","last_undegraded":"2024-03-19T00:36:05.234609+0000","last_fullsized":"2024-03-19T00:36:05.234609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306435+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:36:05.306435+0000","last_peered":"2024-03-19T00:36:05.306435+0000","last_clean":"2024-03-19T00:36:05.306435+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:36:05.306435+0000","last_undegraded":"2024-03-19T00:36:05.306435+0000","last_fullsized":"2024-03-19T00:36:05.306435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306701+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:05.306701+0000","last_peered":"2024-03-19T00:36:05.306701+0000","last_clean":"2024-03-19T00:36:05.306701+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:05.306701+0000","last_undegraded":"2024-03-19T00:36:05.306701+0000","last_fullsized":"2024-03-19T00:36:05.306701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307704+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:05.307704+0000","last_peered":"2024-03-19T00:36:05.307704+0000","last_clean":"2024-03-19T00:36:05.307704+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:05.307704+0000","last_undegraded":"2024-03-19T00:36:05.307704+0000","last_fullsized":"2024-03-19T00:36:05.307704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236299+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:36:05.236299+0000","last_peered":"2024-03-19T00:36:05.236299+0000","last_clean":"2024-03-19T00:36:05.236299+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:36:05.236299+0000","last_undegraded":"2024-03-19T00:36:05.236299+0000","last_fullsized":"2024-03-19T00:36:05.236299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234377+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:05.234377+0000","last_peered":"2024-03-19T00:36:05.234377+0000","last_clean":"2024-03-19T00:36:05.234377+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:05.234377+0000","last_undegraded":"2024-03-19T00:36:05.234377+0000","last_fullsized":"2024-03-19T00:36:05.234377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306188+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:36:05.306188+0000","last_peered":"2024-03-19T00:36:05.306188+0000","last_clean":"2024-03-19T00:36:05.306188+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:36:05.306188+0000","last_undegraded":"2024-03-19T00:36:05.306188+0000","last_fullsized":"2024-03-19T00:36:05.306188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236311+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:36:05.236311+0000","last_peered":"2024-03-19T00:36:05.236311+0000","last_clean":"2024-03-19T00:36:05.236311+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:36:05.236311+0000","last_undegraded":"2024-03-19T00:36:05.236311+0000","last_fullsized":"2024-03-19T00:36:05.236311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307526+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:05.307526+0000","last_peered":"2024-03-19T00:36:05.307526+0000","last_clean":"2024-03-19T00:36:05.307526+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:05.307526+0000","last_undegraded":"2024-03-19T00:36:05.307526+0000","last_fullsized":"2024-03-19T00:36:05.307526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.297155+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:25.297155+0000","last_peered":"2024-03-19T00:36:25.297155+0000","last_clean":"2024-03-19T00:36:25.297155+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:25.297155+0000","last_undegraded":"2024-03-19T00:36:25.297155+0000","last_fullsized":"2024-03-19T00:36:25.297155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306829+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:36:05.306829+0000","last_peered":"2024-03-19T00:36:05.306829+0000","last_clean":"2024-03-19T00:36:05.306829+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:36:05.306829+0000","last_undegraded":"2024-03-19T00:36:05.306829+0000","last_fullsized":"2024-03-19T00:36:05.306829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306715+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:05.306715+0000","last_peered":"2024-03-19T00:36:05.306715+0000","last_clean":"2024-03-19T00:36:05.306715+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:05.306715+0000","last_undegraded":"2024-03-19T00:36:05.306715+0000","last_fullsized":"2024-03-19T00:36:05.306715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307051+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:05.307051+0000","last_peered":"2024-03-19T00:36:05.307051+0000","last_clean":"2024-03-19T00:36:05.307051+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:05.307051+0000","last_undegraded":"2024-03-19T00:36:05.307051+0000","last_fullsized":"2024-03-19T00:36:05.307051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307117+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:36:05.307117+0000","last_peered":"2024-03-19T00:36:05.307117+0000","last_clean":"2024-03-19T00:36:05.307117+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:36:05.307117+0000","last_undegraded":"2024-03-19T00:36:05.307117+0000","last_fullsized":"2024-03-19T00:36:05.307117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306764+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:05.306764+0000","last_peered":"2024-03-19T00:36:05.306764+0000","last_clean":"2024-03-19T00:36:05.306764+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:05.306764+0000","last_undegraded":"2024-03-19T00:36:05.306764+0000","last_fullsized":"2024-03-19T00:36:05.306764+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234150+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:05.234150+0000","last_peered":"2024-03-19T00:36:05.234150+0000","last_clean":"2024-03-19T00:36:05.234150+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:05.234150+0000","last_undegraded":"2024-03-19T00:36:05.234150+0000","last_fullsized":"2024-03-19T00:36:05.234150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307360+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:36:05.307360+0000","last_peered":"2024-03-19T00:36:05.307360+0000","last_clean":"2024-03-19T00:36:05.307360+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:36:05.307360+0000","last_undegraded":"2024-03-19T00:36:05.307360+0000","last_fullsized":"2024-03-19T00:36:05.307360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306841+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:05.306841+0000","last_peered":"2024-03-19T00:36:05.306841+0000","last_clean":"2024-03-19T00:36:05.306841+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:05.306841+0000","last_undegraded":"2024-03-19T00:36:05.306841+0000","last_fullsized":"2024-03-19T00:36:05.306841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.235950+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:05.235950+0000","last_peered":"2024-03-19T00:36:05.235950+0000","last_clean":"2024-03-19T00:36:05.235950+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:05.235950+0000","last_undegraded":"2024-03-19T00:36:05.235950+0000","last_fullsized":"2024-03-19T00:36:05.235950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306506+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:36:05.306506+0000","last_peered":"2024-03-19T00:36:05.306506+0000","last_clean":"2024-03-19T00:36:05.306506+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:36:05.306506+0000","last_undegraded":"2024-03-19T00:36:05.306506+0000","last_fullsized":"2024-03-19T00:36:05.306506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307401+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:05.307401+0000","last_peered":"2024-03-19T00:36:05.307401+0000","last_clean":"2024-03-19T00:36:05.307401+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:05.307401+0000","last_undegraded":"2024-03-19T00:36:05.307401+0000","last_fullsized":"2024-03-19T00:36:05.307401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.308093+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:36:05.308093+0000","last_peered":"2024-03-19T00:36:05.308093+0000","last_clean":"2024-03-19T00:36:05.308093+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:36:05.308093+0000","last_undegraded":"2024-03-19T00:36:05.308093+0000","last_fullsized":"2024-03-19T00:36:05.308093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306332+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:36:05.306332+0000","last_peered":"2024-03-19T00:36:05.306332+0000","last_clean":"2024-03-19T00:36:05.306332+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:36:05.306332+0000","last_undegraded":"2024-03-19T00:36:05.306332+0000","last_fullsized":"2024-03-19T00:36:05.306332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236138+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:05.236138+0000","last_peered":"2024-03-19T00:36:05.236138+0000","last_clean":"2024-03-19T00:36:05.236138+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:05.236138+0000","last_undegraded":"2024-03-19T00:36:05.236138+0000","last_fullsized":"2024-03-19T00:36:05.236138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306809+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:36:05.306809+0000","last_peered":"2024-03-19T00:36:05.306809+0000","last_clean":"2024-03-19T00:36:05.306809+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:36:05.306809+0000","last_undegraded":"2024-03-19T00:36:05.306809+0000","last_fullsized":"2024-03-19T00:36:05.306809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307252+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:05.307252+0000","last_peered":"2024-03-19T00:36:05.307252+0000","last_clean":"2024-03-19T00:36:05.307252+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:05.307252+0000","last_undegraded":"2024-03-19T00:36:05.307252+0000","last_fullsized":"2024-03-19T00:36:05.307252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.235913+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:05.235913+0000","last_peered":"2024-03-19T00:36:05.235913+0000","last_clean":"2024-03-19T00:36:05.235913+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:05.235913+0000","last_undegraded":"2024-03-19T00:36:05.235913+0000","last_fullsized":"2024-03-19T00:36:05.235913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306923+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:05.306923+0000","last_peered":"2024-03-19T00:36:05.306923+0000","last_clean":"2024-03-19T00:36:05.306923+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:05.306923+0000","last_undegraded":"2024-03-19T00:36:05.306923+0000","last_fullsized":"2024-03-19T00:36:05.306923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306245+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:36:05.306245+0000","last_peered":"2024-03-19T00:36:05.306245+0000","last_clean":"2024-03-19T00:36:05.306245+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:36:05.306245+0000","last_undegraded":"2024-03-19T00:36:05.306245+0000","last_fullsized":"2024-03-19T00:36:05.306245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236293+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:05.236293+0000","last_peered":"2024-03-19T00:36:05.236293+0000","last_clean":"2024-03-19T00:36:05.236293+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:05.236293+0000","last_undegraded":"2024-03-19T00:36:05.236293+0000","last_fullsized":"2024-03-19T00:36:05.236293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306448+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:05.306448+0000","last_peered":"2024-03-19T00:36:05.306448+0000","last_clean":"2024-03-19T00:36:05.306448+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:05.306448+0000","last_undegraded":"2024-03-19T00:36:05.306448+0000","last_fullsized":"2024-03-19T00:36:05.306448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.288538+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:24.288538+0000","last_peered":"2024-03-19T00:36:24.288538+0000","last_clean":"2024-03-19T00:36:24.288538+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:24.288538+0000","last_undegraded":"2024-03-19T00:36:24.288538+0000","last_fullsized":"2024-03-19T00:36:24.288538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306964+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:36:05.306964+0000","last_peered":"2024-03-19T00:36:05.306964+0000","last_clean":"2024-03-19T00:36:05.306964+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:36:05.306964+0000","last_undegraded":"2024-03-19T00:36:05.306964+0000","last_fullsized":"2024-03-19T00:36:05.306964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306406+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:05.306406+0000","last_peered":"2024-03-19T00:36:05.306406+0000","last_clean":"2024-03-19T00:36:05.306406+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:05.306406+0000","last_undegraded":"2024-03-19T00:36:05.306406+0000","last_fullsized":"2024-03-19T00:36:05.306406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.295363+0000","last_change":"2024-03-19T00:35:59.209510+0000","last_active":"2024-03-19T00:36:25.295363+0000","last_peered":"2024-03-19T00:36:25.295363+0000","last_clean":"2024-03-19T00:36:25.295363+0000","last_became_active":"2024-03-19T00:35:59.209175+0000","last_became_peered":"2024-03-19T00:35:59.209175+0000","last_unstale":"2024-03-19T00:36:25.295363+0000","last_undegraded":"2024-03-19T00:36:25.295363+0000","last_fullsized":"2024-03-19T00:36:25.295363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:33:14.523970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307478+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:36:05.307478+0000","last_peered":"2024-03-19T00:36:05.307478+0000","last_clean":"2024-03-19T00:36:05.307478+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:36:05.307478+0000","last_undegraded":"2024-03-19T00:36:05.307478+0000","last_fullsized":"2024-03-19T00:36:05.307478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234249+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:05.234249+0000","last_peered":"2024-03-19T00:36:05.234249+0000","last_clean":"2024-03-19T00:36:05.234249+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:05.234249+0000","last_undegraded":"2024-03-19T00:36:05.234249+0000","last_fullsized":"2024-03-19T00:36:05.234249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307029+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:05.307029+0000","last_peered":"2024-03-19T00:36:05.307029+0000","last_clean":"2024-03-19T00:36:05.307029+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:05.307029+0000","last_undegraded":"2024-03-19T00:36:05.307029+0000","last_fullsized":"2024-03-19T00:36:05.307029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758071+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:36:05.758071+0000","last_peered":"2024-03-19T00:36:05.758071+0000","last_clean":"2024-03-19T00:36:05.758071+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:36:05.758071+0000","last_undegraded":"2024-03-19T00:36:05.758071+0000","last_fullsized":"2024-03-19T00:36:05.758071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306540+0000","last_change":"2024-03-19T00:36:04.247824+0000","last_active":"2024-03-19T00:36:05.306540+0000","last_peered":"2024-03-19T00:36:05.306540+0000","last_clean":"2024-03-19T00:36:05.306540+0000","last_became_active":"2024-03-19T00:36:04.247287+0000","last_became_peered":"2024-03-19T00:36:04.247287+0000","last_unstale":"2024-03-19T00:36:05.306540+0000","last_undegraded":"2024-03-19T00:36:05.306540+0000","last_fullsized":"2024-03-19T00:36:05.306540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:08:39.283999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.288572+0000","last_change":"2024-03-19T00:35:59.204325+0000","last_active":"2024-03-19T00:36:24.288572+0000","last_peered":"2024-03-19T00:36:24.288572+0000","last_clean":"2024-03-19T00:36:24.288572+0000","last_became_active":"2024-03-19T00:35:59.203962+0000","last_became_peered":"2024-03-19T00:35:59.203962+0000","last_unstale":"2024-03-19T00:36:24.288572+0000","last_undegraded":"2024-03-19T00:36:24.288572+0000","last_fullsized":"2024-03-19T00:36:24.288572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:31.104229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306333+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:36:05.306333+0000","last_peered":"2024-03-19T00:36:05.306333+0000","last_clean":"2024-03-19T00:36:05.306333+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:36:05.306333+0000","last_undegraded":"2024-03-19T00:36:05.306333+0000","last_fullsized":"2024-03-19T00:36:05.306333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234535+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:05.234535+0000","last_peered":"2024-03-19T00:36:05.234535+0000","last_clean":"2024-03-19T00:36:05.234535+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:05.234535+0000","last_undegraded":"2024-03-19T00:36:05.234535+0000","last_fullsized":"2024-03-19T00:36:05.234535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306758+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:05.306758+0000","last_peered":"2024-03-19T00:36:05.306758+0000","last_clean":"2024-03-19T00:36:05.306758+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:05.306758+0000","last_undegraded":"2024-03-19T00:36:05.306758+0000","last_fullsized":"2024-03-19T00:36:05.306758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306559+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:36:05.306559+0000","last_peered":"2024-03-19T00:36:05.306559+0000","last_clean":"2024-03-19T00:36:05.306559+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:36:05.306559+0000","last_undegraded":"2024-03-19T00:36:05.306559+0000","last_fullsized":"2024-03-19T00:36:05.306559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306810+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:05.306810+0000","last_peered":"2024-03-19T00:36:05.306810+0000","last_clean":"2024-03-19T00:36:05.306810+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:05.306810+0000","last_undegraded":"2024-03-19T00:36:05.306810+0000","last_fullsized":"2024-03-19T00:36:05.306810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234405+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:36:05.234405+0000","last_peered":"2024-03-19T00:36:05.234405+0000","last_clean":"2024-03-19T00:36:05.234405+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:36:05.234405+0000","last_undegraded":"2024-03-19T00:36:05.234405+0000","last_fullsized":"2024-03-19T00:36:05.234405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307264+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:36:05.307264+0000","last_peered":"2024-03-19T00:36:05.307264+0000","last_clean":"2024-03-19T00:36:05.307264+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:36:05.307264+0000","last_undegraded":"2024-03-19T00:36:05.307264+0000","last_fullsized":"2024-03-19T00:36:05.307264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306494+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:05.306494+0000","last_peered":"2024-03-19T00:36:05.306494+0000","last_clean":"2024-03-19T00:36:05.306494+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:05.306494+0000","last_undegraded":"2024-03-19T00:36:05.306494+0000","last_fullsized":"2024-03-19T00:36:05.306494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.288830+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:24.288830+0000","last_peered":"2024-03-19T00:36:24.288830+0000","last_clean":"2024-03-19T00:36:24.288830+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:24.288830+0000","last_undegraded":"2024-03-19T00:36:24.288830+0000","last_fullsized":"2024-03-19T00:36:24.288830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236050+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:36:05.236050+0000","last_peered":"2024-03-19T00:36:05.236050+0000","last_clean":"2024-03-19T00:36:05.236050+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:36:05.236050+0000","last_undegraded":"2024-03-19T00:36:05.236050+0000","last_fullsized":"2024-03-19T00:36:05.236050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306043+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:05.306043+0000","last_peered":"2024-03-19T00:36:05.306043+0000","last_clean":"2024-03-19T00:36:05.306043+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:05.306043+0000","last_undegraded":"2024-03-19T00:36:05.306043+0000","last_fullsized":"2024-03-19T00:36:05.306043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234841+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:05.234841+0000","last_peered":"2024-03-19T00:36:05.234841+0000","last_clean":"2024-03-19T00:36:05.234841+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:05.234841+0000","last_undegraded":"2024-03-19T00:36:05.234841+0000","last_fullsized":"2024-03-19T00:36:05.234841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236433+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:36:05.236433+0000","last_peered":"2024-03-19T00:36:05.236433+0000","last_clean":"2024-03-19T00:36:05.236433+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:36:05.236433+0000","last_undegraded":"2024-03-19T00:36:05.236433+0000","last_fullsized":"2024-03-19T00:36:05.236433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757785+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:05.757785+0000","last_peered":"2024-03-19T00:36:05.757785+0000","last_clean":"2024-03-19T00:36:05.757785+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:05.757785+0000","last_undegraded":"2024-03-19T00:36:05.757785+0000","last_fullsized":"2024-03-19T00:36:05.757785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307680+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:05.307680+0000","last_peered":"2024-03-19T00:36:05.307680+0000","last_clean":"2024-03-19T00:36:05.307680+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:05.307680+0000","last_undegraded":"2024-03-19T00:36:05.307680+0000","last_fullsized":"2024-03-19T00:36:05.307680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236100+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:36:05.236100+0000","last_peered":"2024-03-19T00:36:05.236100+0000","last_clean":"2024-03-19T00:36:05.236100+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:36:05.236100+0000","last_undegraded":"2024-03-19T00:36:05.236100+0000","last_fullsized":"2024-03-19T00:36:05.236100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307845+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:05.307845+0000","last_peered":"2024-03-19T00:36:05.307845+0000","last_clean":"2024-03-19T00:36:05.307845+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:05.307845+0000","last_undegraded":"2024-03-19T00:36:05.307845+0000","last_fullsized":"2024-03-19T00:36:05.307845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307326+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:05.307326+0000","last_peered":"2024-03-19T00:36:05.307326+0000","last_clean":"2024-03-19T00:36:05.307326+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:05.307326+0000","last_undegraded":"2024-03-19T00:36:05.307326+0000","last_fullsized":"2024-03-19T00:36:05.307326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236574+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:36:05.236574+0000","last_peered":"2024-03-19T00:36:05.236574+0000","last_clean":"2024-03-19T00:36:05.236574+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:36:05.236574+0000","last_undegraded":"2024-03-19T00:36:05.236574+0000","last_fullsized":"2024-03-19T00:36:05.236574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306679+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:36:05.306679+0000","last_peered":"2024-03-19T00:36:05.306679+0000","last_clean":"2024-03-19T00:36:05.306679+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:36:05.306679+0000","last_undegraded":"2024-03-19T00:36:05.306679+0000","last_fullsized":"2024-03-19T00:36:05.306679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234780+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:36:05.234780+0000","last_peered":"2024-03-19T00:36:05.234780+0000","last_clean":"2024-03-19T00:36:05.234780+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:36:05.234780+0000","last_undegraded":"2024-03-19T00:36:05.234780+0000","last_fullsized":"2024-03-19T00:36:05.234780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234220+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:36:05.234220+0000","last_peered":"2024-03-19T00:36:05.234220+0000","last_clean":"2024-03-19T00:36:05.234220+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:36:05.234220+0000","last_undegraded":"2024-03-19T00:36:05.234220+0000","last_fullsized":"2024-03-19T00:36:05.234220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757948+0000","last_change":"2024-03-19T00:36:00.224831+0000","last_active":"2024-03-19T00:36:05.757948+0000","last_peered":"2024-03-19T00:36:05.757948+0000","last_clean":"2024-03-19T00:36:05.757948+0000","last_became_active":"2024-03-19T00:36:00.224454+0000","last_became_peered":"2024-03-19T00:36:00.224454+0000","last_unstale":"2024-03-19T00:36:05.757948+0000","last_undegraded":"2024-03-19T00:36:05.757948+0000","last_fullsized":"2024-03-19T00:36:05.757948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:44:13.635476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306853+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:36:05.306853+0000","last_peered":"2024-03-19T00:36:05.306853+0000","last_clean":"2024-03-19T00:36:05.306853+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:36:05.306853+0000","last_undegraded":"2024-03-19T00:36:05.306853+0000","last_fullsized":"2024-03-19T00:36:05.306853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306972+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:36:05.306972+0000","last_peered":"2024-03-19T00:36:05.306972+0000","last_clean":"2024-03-19T00:36:05.306972+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:36:05.306972+0000","last_undegraded":"2024-03-19T00:36:05.306972+0000","last_fullsized":"2024-03-19T00:36:05.306972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234661+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:36:05.234661+0000","last_peered":"2024-03-19T00:36:05.234661+0000","last_clean":"2024-03-19T00:36:05.234661+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:36:05.234661+0000","last_undegraded":"2024-03-19T00:36:05.234661+0000","last_fullsized":"2024-03-19T00:36:05.234661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307497+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:36:05.307497+0000","last_peered":"2024-03-19T00:36:05.307497+0000","last_clean":"2024-03-19T00:36:05.307497+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:36:05.307497+0000","last_undegraded":"2024-03-19T00:36:05.307497+0000","last_fullsized":"2024-03-19T00:36:05.307497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307974+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:36:05.307974+0000","last_peered":"2024-03-19T00:36:05.307974+0000","last_clean":"2024-03-19T00:36:05.307974+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:36:05.307974+0000","last_undegraded":"2024-03-19T00:36:05.307974+0000","last_fullsized":"2024-03-19T00:36:05.307974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306666+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:36:05.306666+0000","last_peered":"2024-03-19T00:36:05.306666+0000","last_clean":"2024-03-19T00:36:05.306666+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:36:05.306666+0000","last_undegraded":"2024-03-19T00:36:05.306666+0000","last_fullsized":"2024-03-19T00:36:05.306666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306269+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:36:05.306269+0000","last_peered":"2024-03-19T00:36:05.306269+0000","last_clean":"2024-03-19T00:36:05.306269+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:36:05.306269+0000","last_undegraded":"2024-03-19T00:36:05.306269+0000","last_fullsized":"2024-03-19T00:36:05.306269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307020+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:36:05.307020+0000","last_peered":"2024-03-19T00:36:05.307020+0000","last_clean":"2024-03-19T00:36:05.307020+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:36:05.307020+0000","last_undegraded":"2024-03-19T00:36:05.307020+0000","last_fullsized":"2024-03-19T00:36:05.307020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306826+0000","last_change":"2024-03-19T00:36:00.222669+0000","last_active":"2024-03-19T00:36:05.306826+0000","last_peered":"2024-03-19T00:36:05.306826+0000","last_clean":"2024-03-19T00:36:05.306826+0000","last_became_active":"2024-03-19T00:36:00.222305+0000","last_became_peered":"2024-03-19T00:36:00.222305+0000","last_unstale":"2024-03-19T00:36:05.306826+0000","last_undegraded":"2024-03-19T00:36:05.306826+0000","last_fullsized":"2024-03-19T00:36:05.306826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:52.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4775,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":80,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":15519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1071620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":477953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":481092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":477319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1589248,"data_stored":1067765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:36:29.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T00:36:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 -- 172.21.15.7:0/3349108622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4601075a0 msgr2=0x7fe4601099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 --2- 172.21.15.7:0/3349108622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4601075a0 0x7fe4601099f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe458009ca0 tx=0x7fe45802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 -- 172.21.15.7:0/3349108622 shutdown_connections 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 --2- 172.21.15.7:0/3349108622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4601075a0 0x7fe4601099f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 --2- 172.21.15.7:0/3349108622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe460104c70 0x7fe460107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 --2- 172.21.15.7:0/3349108622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe460102340 0x7fe460104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 -- 172.21.15.7:0/3349108622 >> 172.21.15.7:0/3349108622 conn(0x7fe4600fbd30 msgr2=0x7fe4600fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 -- 172.21.15.7:0/3349108622 shutdown_connections 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 -- 172.21.15.7:0/3349108622 wait complete. 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 Processor -- start 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.211+0000 7fe466590640 1 -- start start 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 0x7fe460197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe460104c70 0x7fe460197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601075a0 0x7fe46019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d380 con 0x7fe460102340 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d200 con 0x7fe4601075a0 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d500 con 0x7fe460104c70 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 0x7fe460197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 0x7fe460197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52624/0 (socket says 172.21.15.7:52624) 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 -- 172.21.15.7:0/1381089143 learned_addr learned my addr 172.21.15.7:0/1381089143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45f7fe640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe460104c70 0x7fe460197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe464b2d640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601075a0 0x7fe46019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 -- 172.21.15.7:0/1381089143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe460104c70 msgr2=0x7fe460197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe460104c70 0x7fe460197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 -- 172.21.15.7:0/1381089143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601075a0 msgr2=0x7fe46019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601075a0 0x7fe46019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 -- 172.21.15.7:0/1381089143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46019f610 con 0x7fe460102340 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe464b2d640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601075a0 0x7fe46019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45ffff640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 0x7fe460197a90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe44800e9f0 tx=0x7fe44800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800cd80 con 0x7fe460102340 2024-03-19T00:36:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4480108e0 con 0x7fe460102340 2024-03-19T00:36:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46019f900 con 0x7fe460102340 2024-03-19T00:36:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448018920 con 0x7fe460102340 2024-03-19T00:36:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.212+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46019fe50 con 0x7fe460102340 2024-03-19T00:36:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.214+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe460068af0 con 0x7fe460102340 2024-03-19T00:36:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.215+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137289+0+0 (secure 0 0 0) 0x7fe448010a90 con 0x7fe460102340 2024-03-19T00:36:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.215+0000 7fe45d7fa640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe44c0a2800 0x7fe44c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.215+0000 7fe45f7fe640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe44c0a2800 0x7fe44c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.215+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4480cef90 con 0x7fe460102340 2024-03-19T00:36:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.216+0000 7fe45f7fe640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe44c0a2800 0x7fe44c0a4cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4500095f0 tx=0x7fe450005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.220+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe44809b7a0 con 0x7fe460102340 2024-03-19T00:36:29.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.318+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fe448097e10 con 0x7fe460102340 2024-03-19T00:36:29.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.404+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe460061d20 con 0x7fe44c0a2800 2024-03-19T00:36:29.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.414+0000 7fe45d7fa640 1 -- 172.21.15.7:0/1381089143 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340268 (secure 0 0 0) 0x7fe460061d20 con 0x7fe44c0a2800 2024-03-19T00:36:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:29.420 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:36:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe44c0a2800 msgr2=0x7fe44c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe44c0a2800 0x7fe44c0a4cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4500095f0 tx=0x7fe450005f90 comp rx=0 tx=0).stop 2024-03-19T00:36:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 msgr2=0x7fe460197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 0x7fe460197a90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe44800e9f0 tx=0x7fe44800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 shutdown_connections 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe44c0a2800 0x7fe44c0a4cc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4601075a0 0x7fe46019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe460104c70 0x7fe460197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 --2- 172.21.15.7:0/1381089143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe460102340 0x7fe460197a90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 >> 172.21.15.7:0/1381089143 conn(0x7fe4600fbd30 msgr2=0x7fe460102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 shutdown_connections 2024-03-19T00:36:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:29.417+0000 7fe466590640 1 -- 172.21.15.7:0/1381089143 wait complete. 2024-03-19T00:36:29.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:29 smithi007 ceph-mon[62854]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1023 B/s wr, 3 op/s 2024-03-19T00:36:29.427 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:29 smithi007 ceph-mon[62854]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:29.428 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:29 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1733690104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:29.428 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:29 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1075806257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:29.428 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-03-19T00:36:28.003311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595143,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":132,"num_read_kb":115,"num_write":261,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400024,"kb_used_data":13384,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541800,"statfs":{"total":1151940427776,"available":1151530803200,"internally_reserved":0,"allocated":13705216,"data_stored":7511977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":275,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":33,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003633"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307135+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:36:05.307135+0000","last_peered":"2024-03-19T00:36:05.307135+0000","last_clean":"2024-03-19T00:36:05.307135+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:36:05.307135+0000","last_undegraded":"2024-03-19T00:36:05.307135+0000","last_fullsized":"2024-03-19T00:36:05.307135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758129+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:36:05.758129+0000","last_peered":"2024-03-19T00:36:05.758129+0000","last_clean":"2024-03-19T00:36:05.758129+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:36:05.758129+0000","last_undegraded":"2024-03-19T00:36:05.758129+0000","last_fullsized":"2024-03-19T00:36:05.758129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757899+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:36:05.757899+0000","last_peered":"2024-03-19T00:36:05.757899+0000","last_clean":"2024-03-19T00:36:05.757899+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:36:05.757899+0000","last_undegraded":"2024-03-19T00:36:05.757899+0000","last_fullsized":"2024-03-19T00:36:05.757899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307052+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:36:05.307052+0000","last_peered":"2024-03-19T00:36:05.307052+0000","last_clean":"2024-03-19T00:36:05.307052+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:36:05.307052+0000","last_undegraded":"2024-03-19T00:36:05.307052+0000","last_fullsized":"2024-03-19T00:36:05.307052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306652+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:36:05.306652+0000","last_peered":"2024-03-19T00:36:05.306652+0000","last_clean":"2024-03-19T00:36:05.306652+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:36:05.306652+0000","last_undegraded":"2024-03-19T00:36:05.306652+0000","last_fullsized":"2024-03-19T00:36:05.306652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306173+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:36:05.306173+0000","last_peered":"2024-03-19T00:36:05.306173+0000","last_clean":"2024-03-19T00:36:05.306173+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:36:05.306173+0000","last_undegraded":"2024-03-19T00:36:05.306173+0000","last_fullsized":"2024-03-19T00:36:05.306173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.235902+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:36:05.235902+0000","last_peered":"2024-03-19T00:36:05.235902+0000","last_clean":"2024-03-19T00:36:05.235902+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:36:05.235902+0000","last_undegraded":"2024-03-19T00:36:05.235902+0000","last_fullsized":"2024-03-19T00:36:05.235902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306470+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:36:05.306470+0000","last_peered":"2024-03-19T00:36:05.306470+0000","last_clean":"2024-03-19T00:36:05.306470+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:36:05.306470+0000","last_undegraded":"2024-03-19T00:36:05.306470+0000","last_fullsized":"2024-03-19T00:36:05.306470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306269+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:36:05.306269+0000","last_peered":"2024-03-19T00:36:05.306269+0000","last_clean":"2024-03-19T00:36:05.306269+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:36:05.306269+0000","last_undegraded":"2024-03-19T00:36:05.306269+0000","last_fullsized":"2024-03-19T00:36:05.306269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306967+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:36:05.306967+0000","last_peered":"2024-03-19T00:36:05.306967+0000","last_clean":"2024-03-19T00:36:05.306967+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:36:05.306967+0000","last_undegraded":"2024-03-19T00:36:05.306967+0000","last_fullsized":"2024-03-19T00:36:05.306967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236390+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:36:05.236390+0000","last_peered":"2024-03-19T00:36:05.236390+0000","last_clean":"2024-03-19T00:36:05.236390+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:36:05.236390+0000","last_undegraded":"2024-03-19T00:36:05.236390+0000","last_fullsized":"2024-03-19T00:36:05.236390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306962+0000","last_change":"2024-03-19T00:36:00.219901+0000","last_active":"2024-03-19T00:36:05.306962+0000","last_peered":"2024-03-19T00:36:05.306962+0000","last_clean":"2024-03-19T00:36:05.306962+0000","last_became_active":"2024-03-19T00:36:00.219554+0000","last_became_peered":"2024-03-19T00:36:00.219554+0000","last_unstale":"2024-03-19T00:36:05.306962+0000","last_undegraded":"2024-03-19T00:36:05.306962+0000","last_fullsized":"2024-03-19T00:36:05.306962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:05:48.030960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306658+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:36:05.306658+0000","last_peered":"2024-03-19T00:36:05.306658+0000","last_clean":"2024-03-19T00:36:05.306658+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:36:05.306658+0000","last_undegraded":"2024-03-19T00:36:05.306658+0000","last_fullsized":"2024-03-19T00:36:05.306658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757657+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:36:05.757657+0000","last_peered":"2024-03-19T00:36:05.757657+0000","last_clean":"2024-03-19T00:36:05.757657+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:36:05.757657+0000","last_undegraded":"2024-03-19T00:36:05.757657+0000","last_fullsized":"2024-03-19T00:36:05.757657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757702+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:36:05.757702+0000","last_peered":"2024-03-19T00:36:05.757702+0000","last_clean":"2024-03-19T00:36:05.757702+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:36:05.757702+0000","last_undegraded":"2024-03-19T00:36:05.757702+0000","last_fullsized":"2024-03-19T00:36:05.757702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.305781+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:36:05.305781+0000","last_peered":"2024-03-19T00:36:05.305781+0000","last_clean":"2024-03-19T00:36:05.305781+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:36:05.305781+0000","last_undegraded":"2024-03-19T00:36:05.305781+0000","last_fullsized":"2024-03-19T00:36:05.305781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307029+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:36:05.307029+0000","last_peered":"2024-03-19T00:36:05.307029+0000","last_clean":"2024-03-19T00:36:05.307029+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:36:05.307029+0000","last_undegraded":"2024-03-19T00:36:05.307029+0000","last_fullsized":"2024-03-19T00:36:05.307029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234358+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:36:05.234358+0000","last_peered":"2024-03-19T00:36:05.234358+0000","last_clean":"2024-03-19T00:36:05.234358+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:36:05.234358+0000","last_undegraded":"2024-03-19T00:36:05.234358+0000","last_fullsized":"2024-03-19T00:36:05.234358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234769+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:36:05.234769+0000","last_peered":"2024-03-19T00:36:05.234769+0000","last_clean":"2024-03-19T00:36:05.234769+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:36:05.234769+0000","last_undegraded":"2024-03-19T00:36:05.234769+0000","last_fullsized":"2024-03-19T00:36:05.234769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234352+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:05.234352+0000","last_peered":"2024-03-19T00:36:05.234352+0000","last_clean":"2024-03-19T00:36:05.234352+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:05.234352+0000","last_undegraded":"2024-03-19T00:36:05.234352+0000","last_fullsized":"2024-03-19T00:36:05.234352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.296390+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:25.296390+0000","last_peered":"2024-03-19T00:36:25.296390+0000","last_clean":"2024-03-19T00:36:25.296390+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:25.296390+0000","last_undegraded":"2024-03-19T00:36:25.296390+0000","last_fullsized":"2024-03-19T00:36:25.296390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306863+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:36:05.306863+0000","last_peered":"2024-03-19T00:36:05.306863+0000","last_clean":"2024-03-19T00:36:05.306863+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:36:05.306863+0000","last_undegraded":"2024-03-19T00:36:05.306863+0000","last_fullsized":"2024-03-19T00:36:05.306863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306516+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:05.306516+0000","last_peered":"2024-03-19T00:36:05.306516+0000","last_clean":"2024-03-19T00:36:05.306516+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:05.306516+0000","last_undegraded":"2024-03-19T00:36:05.306516+0000","last_fullsized":"2024-03-19T00:36:05.306516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306991+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:05.306991+0000","last_peered":"2024-03-19T00:36:05.306991+0000","last_clean":"2024-03-19T00:36:05.306991+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:05.306991+0000","last_undegraded":"2024-03-19T00:36:05.306991+0000","last_fullsized":"2024-03-19T00:36:05.306991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758141+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:36:05.758141+0000","last_peered":"2024-03-19T00:36:05.758141+0000","last_clean":"2024-03-19T00:36:05.758141+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:36:05.758141+0000","last_undegraded":"2024-03-19T00:36:05.758141+0000","last_fullsized":"2024-03-19T00:36:05.758141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306070+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:05.306070+0000","last_peered":"2024-03-19T00:36:05.306070+0000","last_clean":"2024-03-19T00:36:05.306070+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:05.306070+0000","last_undegraded":"2024-03-19T00:36:05.306070+0000","last_fullsized":"2024-03-19T00:36:05.306070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.295188+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:25.295188+0000","last_peered":"2024-03-19T00:36:25.295188+0000","last_clean":"2024-03-19T00:36:25.295188+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:25.295188+0000","last_undegraded":"2024-03-19T00:36:25.295188+0000","last_fullsized":"2024-03-19T00:36:25.295188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306894+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:36:05.306894+0000","last_peered":"2024-03-19T00:36:05.306894+0000","last_clean":"2024-03-19T00:36:05.306894+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:36:05.306894+0000","last_undegraded":"2024-03-19T00:36:05.306894+0000","last_fullsized":"2024-03-19T00:36:05.306894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306129+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:05.306129+0000","last_peered":"2024-03-19T00:36:05.306129+0000","last_clean":"2024-03-19T00:36:05.306129+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:05.306129+0000","last_undegraded":"2024-03-19T00:36:05.306129+0000","last_fullsized":"2024-03-19T00:36:05.306129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307235+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:05.307235+0000","last_peered":"2024-03-19T00:36:05.307235+0000","last_clean":"2024-03-19T00:36:05.307235+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:05.307235+0000","last_undegraded":"2024-03-19T00:36:05.307235+0000","last_fullsized":"2024-03-19T00:36:05.307235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307565+0000","last_change":"2024-03-19T00:36:00.212990+0000","last_active":"2024-03-19T00:36:05.307565+0000","last_peered":"2024-03-19T00:36:05.307565+0000","last_clean":"2024-03-19T00:36:05.307565+0000","last_became_active":"2024-03-19T00:36:00.212662+0000","last_became_peered":"2024-03-19T00:36:00.212662+0000","last_unstale":"2024-03-19T00:36:05.307565+0000","last_undegraded":"2024-03-19T00:36:05.307565+0000","last_fullsized":"2024-03-19T00:36:05.307565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:53:24.125610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234533+0000","last_change":"2024-03-19T00:36:04.240480+0000","last_active":"2024-03-19T00:36:05.234533+0000","last_peered":"2024-03-19T00:36:05.234533+0000","last_clean":"2024-03-19T00:36:05.234533+0000","last_became_active":"2024-03-19T00:36:04.240016+0000","last_became_peered":"2024-03-19T00:36:04.240016+0000","last_unstale":"2024-03-19T00:36:05.234533+0000","last_undegraded":"2024-03-19T00:36:05.234533+0000","last_fullsized":"2024-03-19T00:36:05.234533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:27.272829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307125+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:36:05.307125+0000","last_peered":"2024-03-19T00:36:05.307125+0000","last_clean":"2024-03-19T00:36:05.307125+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:36:05.307125+0000","last_undegraded":"2024-03-19T00:36:05.307125+0000","last_fullsized":"2024-03-19T00:36:05.307125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306827+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:36:05.306827+0000","last_peered":"2024-03-19T00:36:05.306827+0000","last_clean":"2024-03-19T00:36:05.306827+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:36:05.306827+0000","last_undegraded":"2024-03-19T00:36:05.306827+0000","last_fullsized":"2024-03-19T00:36:05.306827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234608+0000","last_change":"2024-03-19T00:36:04.245145+0000","last_active":"2024-03-19T00:36:05.234608+0000","last_peered":"2024-03-19T00:36:05.234608+0000","last_clean":"2024-03-19T00:36:05.234608+0000","last_became_active":"2024-03-19T00:36:04.244563+0000","last_became_peered":"2024-03-19T00:36:04.244563+0000","last_unstale":"2024-03-19T00:36:05.234608+0000","last_undegraded":"2024-03-19T00:36:05.234608+0000","last_fullsized":"2024-03-19T00:36:05.234608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:56:32.245777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306066+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:05.306066+0000","last_peered":"2024-03-19T00:36:05.306066+0000","last_clean":"2024-03-19T00:36:05.306066+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:05.306066+0000","last_undegraded":"2024-03-19T00:36:05.306066+0000","last_fullsized":"2024-03-19T00:36:05.306066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307685+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:36:05.307685+0000","last_peered":"2024-03-19T00:36:05.307685+0000","last_clean":"2024-03-19T00:36:05.307685+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:36:05.307685+0000","last_undegraded":"2024-03-19T00:36:05.307685+0000","last_fullsized":"2024-03-19T00:36:05.307685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306899+0000","last_change":"2024-03-19T00:36:04.237487+0000","last_active":"2024-03-19T00:36:05.306899+0000","last_peered":"2024-03-19T00:36:05.306899+0000","last_clean":"2024-03-19T00:36:05.306899+0000","last_became_active":"2024-03-19T00:36:04.237048+0000","last_became_peered":"2024-03-19T00:36:04.237048+0000","last_unstale":"2024-03-19T00:36:05.306899+0000","last_undegraded":"2024-03-19T00:36:05.306899+0000","last_fullsized":"2024-03-19T00:36:05.306899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:38.844336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306396+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:05.306396+0000","last_peered":"2024-03-19T00:36:05.306396+0000","last_clean":"2024-03-19T00:36:05.306396+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:05.306396+0000","last_undegraded":"2024-03-19T00:36:05.306396+0000","last_fullsized":"2024-03-19T00:36:05.306396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306344+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:36:05.306344+0000","last_peered":"2024-03-19T00:36:05.306344+0000","last_clean":"2024-03-19T00:36:05.306344+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:36:05.306344+0000","last_undegraded":"2024-03-19T00:36:05.306344+0000","last_fullsized":"2024-03-19T00:36:05.306344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758024+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:05.758024+0000","last_peered":"2024-03-19T00:36:05.758024+0000","last_clean":"2024-03-19T00:36:05.758024+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:05.758024+0000","last_undegraded":"2024-03-19T00:36:05.758024+0000","last_fullsized":"2024-03-19T00:36:05.758024+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.308010+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:05.308010+0000","last_peered":"2024-03-19T00:36:05.308010+0000","last_clean":"2024-03-19T00:36:05.308010+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:05.308010+0000","last_undegraded":"2024-03-19T00:36:05.308010+0000","last_fullsized":"2024-03-19T00:36:05.308010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.233822+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:36:05.233822+0000","last_peered":"2024-03-19T00:36:05.233822+0000","last_clean":"2024-03-19T00:36:05.233822+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:36:05.233822+0000","last_undegraded":"2024-03-19T00:36:05.233822+0000","last_fullsized":"2024-03-19T00:36:05.233822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306947+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:05.306947+0000","last_peered":"2024-03-19T00:36:05.306947+0000","last_clean":"2024-03-19T00:36:05.306947+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:05.306947+0000","last_undegraded":"2024-03-19T00:36:05.306947+0000","last_fullsized":"2024-03-19T00:36:05.306947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306534+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:36:05.306534+0000","last_peered":"2024-03-19T00:36:05.306534+0000","last_clean":"2024-03-19T00:36:05.306534+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:36:05.306534+0000","last_undegraded":"2024-03-19T00:36:05.306534+0000","last_fullsized":"2024-03-19T00:36:05.306534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306649+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:36:05.306649+0000","last_peered":"2024-03-19T00:36:05.306649+0000","last_clean":"2024-03-19T00:36:05.306649+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:36:05.306649+0000","last_undegraded":"2024-03-19T00:36:05.306649+0000","last_fullsized":"2024-03-19T00:36:05.306649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.233866+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:05.233866+0000","last_peered":"2024-03-19T00:36:05.233866+0000","last_clean":"2024-03-19T00:36:05.233866+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:05.233866+0000","last_undegraded":"2024-03-19T00:36:05.233866+0000","last_fullsized":"2024-03-19T00:36:05.233866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.287406+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:24.287406+0000","last_peered":"2024-03-19T00:36:24.287406+0000","last_clean":"2024-03-19T00:36:24.287406+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:24.287406+0000","last_undegraded":"2024-03-19T00:36:24.287406+0000","last_fullsized":"2024-03-19T00:36:24.287406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758128+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:36:05.758128+0000","last_peered":"2024-03-19T00:36:05.758128+0000","last_clean":"2024-03-19T00:36:05.758128+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:36:05.758128+0000","last_undegraded":"2024-03-19T00:36:05.758128+0000","last_fullsized":"2024-03-19T00:36:05.758128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307326+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:05.307326+0000","last_peered":"2024-03-19T00:36:05.307326+0000","last_clean":"2024-03-19T00:36:05.307326+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:05.307326+0000","last_undegraded":"2024-03-19T00:36:05.307326+0000","last_fullsized":"2024-03-19T00:36:05.307326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307431+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:05.307431+0000","last_peered":"2024-03-19T00:36:05.307431+0000","last_clean":"2024-03-19T00:36:05.307431+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:05.307431+0000","last_undegraded":"2024-03-19T00:36:05.307431+0000","last_fullsized":"2024-03-19T00:36:05.307431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307050+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:36:05.307050+0000","last_peered":"2024-03-19T00:36:05.307050+0000","last_clean":"2024-03-19T00:36:05.307050+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:36:05.307050+0000","last_undegraded":"2024-03-19T00:36:05.307050+0000","last_fullsized":"2024-03-19T00:36:05.307050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.233950+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:05.233950+0000","last_peered":"2024-03-19T00:36:05.233950+0000","last_clean":"2024-03-19T00:36:05.233950+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:05.233950+0000","last_undegraded":"2024-03-19T00:36:05.233950+0000","last_fullsized":"2024-03-19T00:36:05.233950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'10","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:16.007959+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:36:16.007959+0000","last_peered":"2024-03-19T00:36:16.007959+0000","last_clean":"2024-03-19T00:36:16.007959+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:36:16.007959+0000","last_undegraded":"2024-03-19T00:36:16.007959+0000","last_fullsized":"2024-03-19T00:36:16.007959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3843,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307372+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:36:05.307372+0000","last_peered":"2024-03-19T00:36:05.307372+0000","last_clean":"2024-03-19T00:36:05.307372+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:36:05.307372+0000","last_undegraded":"2024-03-19T00:36:05.307372+0000","last_fullsized":"2024-03-19T00:36:05.307372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.308061+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:05.308061+0000","last_peered":"2024-03-19T00:36:05.308061+0000","last_clean":"2024-03-19T00:36:05.308061+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:05.308061+0000","last_undegraded":"2024-03-19T00:36:05.308061+0000","last_fullsized":"2024-03-19T00:36:05.308061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234609+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:05.234609+0000","last_peered":"2024-03-19T00:36:05.234609+0000","last_clean":"2024-03-19T00:36:05.234609+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:05.234609+0000","last_undegraded":"2024-03-19T00:36:05.234609+0000","last_fullsized":"2024-03-19T00:36:05.234609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306435+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:36:05.306435+0000","last_peered":"2024-03-19T00:36:05.306435+0000","last_clean":"2024-03-19T00:36:05.306435+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:36:05.306435+0000","last_undegraded":"2024-03-19T00:36:05.306435+0000","last_fullsized":"2024-03-19T00:36:05.306435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306701+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:05.306701+0000","last_peered":"2024-03-19T00:36:05.306701+0000","last_clean":"2024-03-19T00:36:05.306701+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:05.306701+0000","last_undegraded":"2024-03-19T00:36:05.306701+0000","last_fullsized":"2024-03-19T00:36:05.306701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307704+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:05.307704+0000","last_peered":"2024-03-19T00:36:05.307704+0000","last_clean":"2024-03-19T00:36:05.307704+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:05.307704+0000","last_undegraded":"2024-03-19T00:36:05.307704+0000","last_fullsized":"2024-03-19T00:36:05.307704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236299+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:36:05.236299+0000","last_peered":"2024-03-19T00:36:05.236299+0000","last_clean":"2024-03-19T00:36:05.236299+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:36:05.236299+0000","last_undegraded":"2024-03-19T00:36:05.236299+0000","last_fullsized":"2024-03-19T00:36:05.236299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234377+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:05.234377+0000","last_peered":"2024-03-19T00:36:05.234377+0000","last_clean":"2024-03-19T00:36:05.234377+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:05.234377+0000","last_undegraded":"2024-03-19T00:36:05.234377+0000","last_fullsized":"2024-03-19T00:36:05.234377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306188+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:36:05.306188+0000","last_peered":"2024-03-19T00:36:05.306188+0000","last_clean":"2024-03-19T00:36:05.306188+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:36:05.306188+0000","last_undegraded":"2024-03-19T00:36:05.306188+0000","last_fullsized":"2024-03-19T00:36:05.306188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236311+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:36:05.236311+0000","last_peered":"2024-03-19T00:36:05.236311+0000","last_clean":"2024-03-19T00:36:05.236311+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:36:05.236311+0000","last_undegraded":"2024-03-19T00:36:05.236311+0000","last_fullsized":"2024-03-19T00:36:05.236311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307526+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:05.307526+0000","last_peered":"2024-03-19T00:36:05.307526+0000","last_clean":"2024-03-19T00:36:05.307526+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:05.307526+0000","last_undegraded":"2024-03-19T00:36:05.307526+0000","last_fullsized":"2024-03-19T00:36:05.307526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.297155+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:25.297155+0000","last_peered":"2024-03-19T00:36:25.297155+0000","last_clean":"2024-03-19T00:36:25.297155+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:25.297155+0000","last_undegraded":"2024-03-19T00:36:25.297155+0000","last_fullsized":"2024-03-19T00:36:25.297155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306829+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:36:05.306829+0000","last_peered":"2024-03-19T00:36:05.306829+0000","last_clean":"2024-03-19T00:36:05.306829+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:36:05.306829+0000","last_undegraded":"2024-03-19T00:36:05.306829+0000","last_fullsized":"2024-03-19T00:36:05.306829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306715+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:05.306715+0000","last_peered":"2024-03-19T00:36:05.306715+0000","last_clean":"2024-03-19T00:36:05.306715+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:05.306715+0000","last_undegraded":"2024-03-19T00:36:05.306715+0000","last_fullsized":"2024-03-19T00:36:05.306715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307051+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:05.307051+0000","last_peered":"2024-03-19T00:36:05.307051+0000","last_clean":"2024-03-19T00:36:05.307051+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:05.307051+0000","last_undegraded":"2024-03-19T00:36:05.307051+0000","last_fullsized":"2024-03-19T00:36:05.307051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307117+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:36:05.307117+0000","last_peered":"2024-03-19T00:36:05.307117+0000","last_clean":"2024-03-19T00:36:05.307117+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:36:05.307117+0000","last_undegraded":"2024-03-19T00:36:05.307117+0000","last_fullsized":"2024-03-19T00:36:05.307117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306764+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:05.306764+0000","last_peered":"2024-03-19T00:36:05.306764+0000","last_clean":"2024-03-19T00:36:05.306764+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:05.306764+0000","last_undegraded":"2024-03-19T00:36:05.306764+0000","last_fullsized":"2024-03-19T00:36:05.306764+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234150+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:05.234150+0000","last_peered":"2024-03-19T00:36:05.234150+0000","last_clean":"2024-03-19T00:36:05.234150+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:05.234150+0000","last_undegraded":"2024-03-19T00:36:05.234150+0000","last_fullsized":"2024-03-19T00:36:05.234150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307360+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:36:05.307360+0000","last_peered":"2024-03-19T00:36:05.307360+0000","last_clean":"2024-03-19T00:36:05.307360+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:36:05.307360+0000","last_undegraded":"2024-03-19T00:36:05.307360+0000","last_fullsized":"2024-03-19T00:36:05.307360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306841+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:05.306841+0000","last_peered":"2024-03-19T00:36:05.306841+0000","last_clean":"2024-03-19T00:36:05.306841+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:05.306841+0000","last_undegraded":"2024-03-19T00:36:05.306841+0000","last_fullsized":"2024-03-19T00:36:05.306841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.235950+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:05.235950+0000","last_peered":"2024-03-19T00:36:05.235950+0000","last_clean":"2024-03-19T00:36:05.235950+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:05.235950+0000","last_undegraded":"2024-03-19T00:36:05.235950+0000","last_fullsized":"2024-03-19T00:36:05.235950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306506+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:36:05.306506+0000","last_peered":"2024-03-19T00:36:05.306506+0000","last_clean":"2024-03-19T00:36:05.306506+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:36:05.306506+0000","last_undegraded":"2024-03-19T00:36:05.306506+0000","last_fullsized":"2024-03-19T00:36:05.306506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307401+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:05.307401+0000","last_peered":"2024-03-19T00:36:05.307401+0000","last_clean":"2024-03-19T00:36:05.307401+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:05.307401+0000","last_undegraded":"2024-03-19T00:36:05.307401+0000","last_fullsized":"2024-03-19T00:36:05.307401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.308093+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:36:05.308093+0000","last_peered":"2024-03-19T00:36:05.308093+0000","last_clean":"2024-03-19T00:36:05.308093+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:36:05.308093+0000","last_undegraded":"2024-03-19T00:36:05.308093+0000","last_fullsized":"2024-03-19T00:36:05.308093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306332+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:36:05.306332+0000","last_peered":"2024-03-19T00:36:05.306332+0000","last_clean":"2024-03-19T00:36:05.306332+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:36:05.306332+0000","last_undegraded":"2024-03-19T00:36:05.306332+0000","last_fullsized":"2024-03-19T00:36:05.306332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236138+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:05.236138+0000","last_peered":"2024-03-19T00:36:05.236138+0000","last_clean":"2024-03-19T00:36:05.236138+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:05.236138+0000","last_undegraded":"2024-03-19T00:36:05.236138+0000","last_fullsized":"2024-03-19T00:36:05.236138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306809+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:36:05.306809+0000","last_peered":"2024-03-19T00:36:05.306809+0000","last_clean":"2024-03-19T00:36:05.306809+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:36:05.306809+0000","last_undegraded":"2024-03-19T00:36:05.306809+0000","last_fullsized":"2024-03-19T00:36:05.306809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307252+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:05.307252+0000","last_peered":"2024-03-19T00:36:05.307252+0000","last_clean":"2024-03-19T00:36:05.307252+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:05.307252+0000","last_undegraded":"2024-03-19T00:36:05.307252+0000","last_fullsized":"2024-03-19T00:36:05.307252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.235913+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:05.235913+0000","last_peered":"2024-03-19T00:36:05.235913+0000","last_clean":"2024-03-19T00:36:05.235913+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:05.235913+0000","last_undegraded":"2024-03-19T00:36:05.235913+0000","last_fullsized":"2024-03-19T00:36:05.235913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306923+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:05.306923+0000","last_peered":"2024-03-19T00:36:05.306923+0000","last_clean":"2024-03-19T00:36:05.306923+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:05.306923+0000","last_undegraded":"2024-03-19T00:36:05.306923+0000","last_fullsized":"2024-03-19T00:36:05.306923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306245+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:36:05.306245+0000","last_peered":"2024-03-19T00:36:05.306245+0000","last_clean":"2024-03-19T00:36:05.306245+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:36:05.306245+0000","last_undegraded":"2024-03-19T00:36:05.306245+0000","last_fullsized":"2024-03-19T00:36:05.306245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236293+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:05.236293+0000","last_peered":"2024-03-19T00:36:05.236293+0000","last_clean":"2024-03-19T00:36:05.236293+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:05.236293+0000","last_undegraded":"2024-03-19T00:36:05.236293+0000","last_fullsized":"2024-03-19T00:36:05.236293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306448+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:05.306448+0000","last_peered":"2024-03-19T00:36:05.306448+0000","last_clean":"2024-03-19T00:36:05.306448+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:05.306448+0000","last_undegraded":"2024-03-19T00:36:05.306448+0000","last_fullsized":"2024-03-19T00:36:05.306448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.288538+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:24.288538+0000","last_peered":"2024-03-19T00:36:24.288538+0000","last_clean":"2024-03-19T00:36:24.288538+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:24.288538+0000","last_undegraded":"2024-03-19T00:36:24.288538+0000","last_fullsized":"2024-03-19T00:36:24.288538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306964+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:36:05.306964+0000","last_peered":"2024-03-19T00:36:05.306964+0000","last_clean":"2024-03-19T00:36:05.306964+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:36:05.306964+0000","last_undegraded":"2024-03-19T00:36:05.306964+0000","last_fullsized":"2024-03-19T00:36:05.306964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306406+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:05.306406+0000","last_peered":"2024-03-19T00:36:05.306406+0000","last_clean":"2024-03-19T00:36:05.306406+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:05.306406+0000","last_undegraded":"2024-03-19T00:36:05.306406+0000","last_fullsized":"2024-03-19T00:36:05.306406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:25.295363+0000","last_change":"2024-03-19T00:35:59.209510+0000","last_active":"2024-03-19T00:36:25.295363+0000","last_peered":"2024-03-19T00:36:25.295363+0000","last_clean":"2024-03-19T00:36:25.295363+0000","last_became_active":"2024-03-19T00:35:59.209175+0000","last_became_peered":"2024-03-19T00:35:59.209175+0000","last_unstale":"2024-03-19T00:36:25.295363+0000","last_undegraded":"2024-03-19T00:36:25.295363+0000","last_fullsized":"2024-03-19T00:36:25.295363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:33:14.523970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307478+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:36:05.307478+0000","last_peered":"2024-03-19T00:36:05.307478+0000","last_clean":"2024-03-19T00:36:05.307478+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:36:05.307478+0000","last_undegraded":"2024-03-19T00:36:05.307478+0000","last_fullsized":"2024-03-19T00:36:05.307478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234249+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:05.234249+0000","last_peered":"2024-03-19T00:36:05.234249+0000","last_clean":"2024-03-19T00:36:05.234249+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:05.234249+0000","last_undegraded":"2024-03-19T00:36:05.234249+0000","last_fullsized":"2024-03-19T00:36:05.234249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307029+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:05.307029+0000","last_peered":"2024-03-19T00:36:05.307029+0000","last_clean":"2024-03-19T00:36:05.307029+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:05.307029+0000","last_undegraded":"2024-03-19T00:36:05.307029+0000","last_fullsized":"2024-03-19T00:36:05.307029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.758071+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:36:05.758071+0000","last_peered":"2024-03-19T00:36:05.758071+0000","last_clean":"2024-03-19T00:36:05.758071+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:36:05.758071+0000","last_undegraded":"2024-03-19T00:36:05.758071+0000","last_fullsized":"2024-03-19T00:36:05.758071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306540+0000","last_change":"2024-03-19T00:36:04.247824+0000","last_active":"2024-03-19T00:36:05.306540+0000","last_peered":"2024-03-19T00:36:05.306540+0000","last_clean":"2024-03-19T00:36:05.306540+0000","last_became_active":"2024-03-19T00:36:04.247287+0000","last_became_peered":"2024-03-19T00:36:04.247287+0000","last_unstale":"2024-03-19T00:36:05.306540+0000","last_undegraded":"2024-03-19T00:36:05.306540+0000","last_fullsized":"2024-03-19T00:36:05.306540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:08:39.283999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.288572+0000","last_change":"2024-03-19T00:35:59.204325+0000","last_active":"2024-03-19T00:36:24.288572+0000","last_peered":"2024-03-19T00:36:24.288572+0000","last_clean":"2024-03-19T00:36:24.288572+0000","last_became_active":"2024-03-19T00:35:59.203962+0000","last_became_peered":"2024-03-19T00:35:59.203962+0000","last_unstale":"2024-03-19T00:36:24.288572+0000","last_undegraded":"2024-03-19T00:36:24.288572+0000","last_fullsized":"2024-03-19T00:36:24.288572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:31.104229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306333+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:36:05.306333+0000","last_peered":"2024-03-19T00:36:05.306333+0000","last_clean":"2024-03-19T00:36:05.306333+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:36:05.306333+0000","last_undegraded":"2024-03-19T00:36:05.306333+0000","last_fullsized":"2024-03-19T00:36:05.306333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234535+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:05.234535+0000","last_peered":"2024-03-19T00:36:05.234535+0000","last_clean":"2024-03-19T00:36:05.234535+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:05.234535+0000","last_undegraded":"2024-03-19T00:36:05.234535+0000","last_fullsized":"2024-03-19T00:36:05.234535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306758+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:05.306758+0000","last_peered":"2024-03-19T00:36:05.306758+0000","last_clean":"2024-03-19T00:36:05.306758+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:05.306758+0000","last_undegraded":"2024-03-19T00:36:05.306758+0000","last_fullsized":"2024-03-19T00:36:05.306758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306559+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:36:05.306559+0000","last_peered":"2024-03-19T00:36:05.306559+0000","last_clean":"2024-03-19T00:36:05.306559+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:36:05.306559+0000","last_undegraded":"2024-03-19T00:36:05.306559+0000","last_fullsized":"2024-03-19T00:36:05.306559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306810+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:05.306810+0000","last_peered":"2024-03-19T00:36:05.306810+0000","last_clean":"2024-03-19T00:36:05.306810+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:05.306810+0000","last_undegraded":"2024-03-19T00:36:05.306810+0000","last_fullsized":"2024-03-19T00:36:05.306810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234405+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:36:05.234405+0000","last_peered":"2024-03-19T00:36:05.234405+0000","last_clean":"2024-03-19T00:36:05.234405+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:36:05.234405+0000","last_undegraded":"2024-03-19T00:36:05.234405+0000","last_fullsized":"2024-03-19T00:36:05.234405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307264+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:36:05.307264+0000","last_peered":"2024-03-19T00:36:05.307264+0000","last_clean":"2024-03-19T00:36:05.307264+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:36:05.307264+0000","last_undegraded":"2024-03-19T00:36:05.307264+0000","last_fullsized":"2024-03-19T00:36:05.307264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306494+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:05.306494+0000","last_peered":"2024-03-19T00:36:05.306494+0000","last_clean":"2024-03-19T00:36:05.306494+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:05.306494+0000","last_undegraded":"2024-03-19T00:36:05.306494+0000","last_fullsized":"2024-03-19T00:36:05.306494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:24.288830+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:24.288830+0000","last_peered":"2024-03-19T00:36:24.288830+0000","last_clean":"2024-03-19T00:36:24.288830+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:24.288830+0000","last_undegraded":"2024-03-19T00:36:24.288830+0000","last_fullsized":"2024-03-19T00:36:24.288830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236050+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:36:05.236050+0000","last_peered":"2024-03-19T00:36:05.236050+0000","last_clean":"2024-03-19T00:36:05.236050+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:36:05.236050+0000","last_undegraded":"2024-03-19T00:36:05.236050+0000","last_fullsized":"2024-03-19T00:36:05.236050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306043+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:05.306043+0000","last_peered":"2024-03-19T00:36:05.306043+0000","last_clean":"2024-03-19T00:36:05.306043+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:05.306043+0000","last_undegraded":"2024-03-19T00:36:05.306043+0000","last_fullsized":"2024-03-19T00:36:05.306043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234841+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:05.234841+0000","last_peered":"2024-03-19T00:36:05.234841+0000","last_clean":"2024-03-19T00:36:05.234841+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:05.234841+0000","last_undegraded":"2024-03-19T00:36:05.234841+0000","last_fullsized":"2024-03-19T00:36:05.234841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236433+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:36:05.236433+0000","last_peered":"2024-03-19T00:36:05.236433+0000","last_clean":"2024-03-19T00:36:05.236433+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:36:05.236433+0000","last_undegraded":"2024-03-19T00:36:05.236433+0000","last_fullsized":"2024-03-19T00:36:05.236433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757785+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:05.757785+0000","last_peered":"2024-03-19T00:36:05.757785+0000","last_clean":"2024-03-19T00:36:05.757785+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:05.757785+0000","last_undegraded":"2024-03-19T00:36:05.757785+0000","last_fullsized":"2024-03-19T00:36:05.757785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307680+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:05.307680+0000","last_peered":"2024-03-19T00:36:05.307680+0000","last_clean":"2024-03-19T00:36:05.307680+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:05.307680+0000","last_undegraded":"2024-03-19T00:36:05.307680+0000","last_fullsized":"2024-03-19T00:36:05.307680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236100+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:36:05.236100+0000","last_peered":"2024-03-19T00:36:05.236100+0000","last_clean":"2024-03-19T00:36:05.236100+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:36:05.236100+0000","last_undegraded":"2024-03-19T00:36:05.236100+0000","last_fullsized":"2024-03-19T00:36:05.236100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307845+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:05.307845+0000","last_peered":"2024-03-19T00:36:05.307845+0000","last_clean":"2024-03-19T00:36:05.307845+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:05.307845+0000","last_undegraded":"2024-03-19T00:36:05.307845+0000","last_fullsized":"2024-03-19T00:36:05.307845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307326+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:05.307326+0000","last_peered":"2024-03-19T00:36:05.307326+0000","last_clean":"2024-03-19T00:36:05.307326+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:05.307326+0000","last_undegraded":"2024-03-19T00:36:05.307326+0000","last_fullsized":"2024-03-19T00:36:05.307326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.236574+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:36:05.236574+0000","last_peered":"2024-03-19T00:36:05.236574+0000","last_clean":"2024-03-19T00:36:05.236574+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:36:05.236574+0000","last_undegraded":"2024-03-19T00:36:05.236574+0000","last_fullsized":"2024-03-19T00:36:05.236574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306679+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:36:05.306679+0000","last_peered":"2024-03-19T00:36:05.306679+0000","last_clean":"2024-03-19T00:36:05.306679+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:36:05.306679+0000","last_undegraded":"2024-03-19T00:36:05.306679+0000","last_fullsized":"2024-03-19T00:36:05.306679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234780+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:36:05.234780+0000","last_peered":"2024-03-19T00:36:05.234780+0000","last_clean":"2024-03-19T00:36:05.234780+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:36:05.234780+0000","last_undegraded":"2024-03-19T00:36:05.234780+0000","last_fullsized":"2024-03-19T00:36:05.234780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234220+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:36:05.234220+0000","last_peered":"2024-03-19T00:36:05.234220+0000","last_clean":"2024-03-19T00:36:05.234220+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:36:05.234220+0000","last_undegraded":"2024-03-19T00:36:05.234220+0000","last_fullsized":"2024-03-19T00:36:05.234220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.757948+0000","last_change":"2024-03-19T00:36:00.224831+0000","last_active":"2024-03-19T00:36:05.757948+0000","last_peered":"2024-03-19T00:36:05.757948+0000","last_clean":"2024-03-19T00:36:05.757948+0000","last_became_active":"2024-03-19T00:36:00.224454+0000","last_became_peered":"2024-03-19T00:36:00.224454+0000","last_unstale":"2024-03-19T00:36:05.757948+0000","last_undegraded":"2024-03-19T00:36:05.757948+0000","last_fullsized":"2024-03-19T00:36:05.757948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:44:13.635476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306853+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:36:05.306853+0000","last_peered":"2024-03-19T00:36:05.306853+0000","last_clean":"2024-03-19T00:36:05.306853+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:36:05.306853+0000","last_undegraded":"2024-03-19T00:36:05.306853+0000","last_fullsized":"2024-03-19T00:36:05.306853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306972+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:36:05.306972+0000","last_peered":"2024-03-19T00:36:05.306972+0000","last_clean":"2024-03-19T00:36:05.306972+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:36:05.306972+0000","last_undegraded":"2024-03-19T00:36:05.306972+0000","last_fullsized":"2024-03-19T00:36:05.306972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.234661+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:36:05.234661+0000","last_peered":"2024-03-19T00:36:05.234661+0000","last_clean":"2024-03-19T00:36:05.234661+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:36:05.234661+0000","last_undegraded":"2024-03-19T00:36:05.234661+0000","last_fullsized":"2024-03-19T00:36:05.234661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307497+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:36:05.307497+0000","last_peered":"2024-03-19T00:36:05.307497+0000","last_clean":"2024-03-19T00:36:05.307497+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:36:05.307497+0000","last_undegraded":"2024-03-19T00:36:05.307497+0000","last_fullsized":"2024-03-19T00:36:05.307497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307974+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:36:05.307974+0000","last_peered":"2024-03-19T00:36:05.307974+0000","last_clean":"2024-03-19T00:36:05.307974+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:36:05.307974+0000","last_undegraded":"2024-03-19T00:36:05.307974+0000","last_fullsized":"2024-03-19T00:36:05.307974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306666+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:36:05.306666+0000","last_peered":"2024-03-19T00:36:05.306666+0000","last_clean":"2024-03-19T00:36:05.306666+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:36:05.306666+0000","last_undegraded":"2024-03-19T00:36:05.306666+0000","last_fullsized":"2024-03-19T00:36:05.306666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306269+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:36:05.306269+0000","last_peered":"2024-03-19T00:36:05.306269+0000","last_clean":"2024-03-19T00:36:05.306269+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:36:05.306269+0000","last_undegraded":"2024-03-19T00:36:05.306269+0000","last_fullsized":"2024-03-19T00:36:05.306269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.307020+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:36:05.307020+0000","last_peered":"2024-03-19T00:36:05.307020+0000","last_clean":"2024-03-19T00:36:05.307020+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:36:05.307020+0000","last_undegraded":"2024-03-19T00:36:05.307020+0000","last_fullsized":"2024-03-19T00:36:05.307020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T00:36:05.306826+0000","last_change":"2024-03-19T00:36:00.222669+0000","last_active":"2024-03-19T00:36:05.306826+0000","last_peered":"2024-03-19T00:36:05.306826+0000","last_clean":"2024-03-19T00:36:05.306826+0000","last_became_active":"2024-03-19T00:36:00.222305+0000","last_became_peered":"2024-03-19T00:36:00.222305+0000","last_unstale":"2024-03-19T00:36:05.306826+0000","last_undegraded":"2024-03-19T00:36:05.306826+0000","last_fullsized":"2024-03-19T00:36:05.306826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:52.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4775,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":80,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":15519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1071620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":477953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":481092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":477319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1589248,"data_stored":1067765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:36:29.434 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-19T00:36:29.435 INFO:tasks.cephadm:Setting up client nodes... 2024-03-19T00:36:29.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-19T00:36:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:29 smithi044 ceph-mon[66366]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1023 B/s wr, 3 op/s 2024-03-19T00:36:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:29 smithi044 ceph-mon[66366]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:29 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1733690104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:29 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1075806257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:29 smithi204 ceph-mon[66572]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1023 B/s wr, 3 op/s 2024-03-19T00:36:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:29 smithi204 ceph-mon[66572]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:36:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:29 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1733690104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:36:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:29 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1075806257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:30 smithi007 ceph-mon[62854]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:36:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:30 smithi007 ceph-mon[62854]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-19T00:36:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:30 smithi007 ceph-mon[62854]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:36:30.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:30 smithi044 ceph-mon[66366]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:36:30.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:30 smithi044 ceph-mon[66366]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-19T00:36:30.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:30 smithi044 ceph-mon[66366]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:36:30.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:30 smithi204 ceph-mon[66572]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:36:30.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:30 smithi204 ceph-mon[66572]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-19T00:36:30.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:30 smithi204 ceph-mon[66572]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:36:31.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:31 smithi007 ceph-mon[62854]: pgmap v306: 129 pgs: 129 active+clean; 621 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 -- 172.21.15.7:0/158829707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 msgr2=0x7fd2c01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 --2- 172.21.15.7:0/158829707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c01028c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40099c0 tx=0x7fd2b4009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 -- 172.21.15.7:0/158829707 shutdown_connections 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 --2- 172.21.15.7:0/158829707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0102e00 0x7fd2c01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 --2- 172.21.15.7:0/158829707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0101260 0x7fd2c0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 --2- 172.21.15.7:0/158829707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c01028c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 -- 172.21.15.7:0/158829707 >> 172.21.15.7:0/158829707 conn(0x7fd2c00fc800 msgr2=0x7fd2c00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 -- 172.21.15.7:0/158829707 shutdown_connections 2024-03-19T00:36:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.602+0000 7fd2c830e700 1 -- 172.21.15.7:0/158829707 wait complete. 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 Processor -- start 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 -- start start 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0101260 0x7fd2c006ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0102e00 0x7fd2c019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c010ccf0 con 0x7fd2c0102460 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c010cb70 con 0x7fd2c0102e00 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c830e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c010ce70 con 0x7fd2c0101260 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c68ab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0102e00 0x7fd2c019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c019d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52650/0 (socket says 172.21.15.7:52650) 2024-03-19T00:36:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 -- 172.21.15.7:0/378542107 learned_addr learned my addr 172.21.15.7:0/378542107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c60aa700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0101260 0x7fd2c006ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 -- 172.21.15.7:0/378542107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0101260 msgr2=0x7fd2c006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0101260 0x7fd2c006ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 -- 172.21.15.7:0/378542107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0102e00 msgr2=0x7fd2c019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0102e00 0x7fd2c019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c58a9700 1 -- 172.21.15.7:0/378542107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c01a0010 con 0x7fd2c0102460 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c68ab700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0102e00 0x7fd2c019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.603+0000 7fd2c60aa700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0101260 0x7fd2c006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.604+0000 7fd2c58a9700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c019d100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40036f0 tx=0x7fd2b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.604+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4009e60 con 0x7fd2c0102460 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.604+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c01a02a0 con 0x7fd2c0102460 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.604+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2b40038c0 con 0x7fd2c0102460 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.604+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40055b0 con 0x7fd2c0102460 2024-03-19T00:36:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.604+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c01a0700 con 0x7fd2c0102460 2024-03-19T00:36:31.608 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:31 smithi044 ceph-mon[66366]: pgmap v306: 129 pgs: 129 active+clean; 621 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-03-19T00:36:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.605+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c0050820 con 0x7fd2c0102460 2024-03-19T00:36:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.606+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fd2b4003a70 con 0x7fd2c0102460 2024-03-19T00:36:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.606+0000 7fd2bf7fe700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd2a80a29d0 0x7fd2a80a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.607+0000 7fd2c60aa700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd2a80a29d0 0x7fd2a80a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.607+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd2b40cfc70 con 0x7fd2c0102460 2024-03-19T00:36:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.607+0000 7fd2c60aa700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd2a80a29d0 0x7fd2a80a4e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2c01022c0 tx=0x7fd2b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.611+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd2b4013070 con 0x7fd2c0102460 2024-03-19T00:36:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:31 smithi204 ceph-mon[66572]: pgmap v306: 129 pgs: 129 active+clean; 621 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-03-19T00:36:31.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.902+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd2c0062f20 con 0x7fd2c0102460 2024-03-19T00:36:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.903+0000 7fd2bf7fe700 1 -- 172.21.15.7:0/378542107 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd2b409ca00 con 0x7fd2c0102460 2024-03-19T00:36:31.904 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-03-19T00:36:31.905 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAP3vhlZ9DYNRAAVvVkv8joxeQw2VQTFOMm9Q== 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd2a80a29d0 msgr2=0x7fd2a80a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd2a80a29d0 0x7fd2a80a4e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2c01022c0 tx=0x7fd2b8009260 comp rx=0 tx=0).stop 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 msgr2=0x7fd2c019d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c019d100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40036f0 tx=0x7fd2b400f690 comp rx=0 tx=0).stop 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 shutdown_connections 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd2a80a29d0 0x7fd2a80a4e90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2c0102e00 0x7fd2c019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2c0101260 0x7fd2c006ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 --2- 172.21.15.7:0/378542107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c0102460 0x7fd2c019d100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 >> 172.21.15.7:0/378542107 conn(0x7fd2c00fc800 msgr2=0x7fd2c0105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 shutdown_connections 2024-03-19T00:36:31.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:31.906+0000 7fd2c830e700 1 -- 172.21.15.7:0/378542107 wait complete. 2024-03-19T00:36:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:32 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/378542107' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T00:36:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:32 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/378542107' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T00:36:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:32 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/378542107' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T00:36:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:32 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/378542107' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T00:36:32.691 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:36:32.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-19T00:36:32.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-19T00:36:32.737 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-19T00:36:32.737 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-19T00:36:32.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-19T00:36:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:32 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/378542107' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T00:36:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:32 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/378542107' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T00:36:33.024 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:34.176 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:33 smithi007 ceph-mon[62854]: pgmap v307: 129 pgs: 129 active+clean; 621 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-19T00:36:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:33 smithi204 ceph-mon[66572]: pgmap v307: 129 pgs: 129 active+clean; 621 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-19T00:36:34.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:33 smithi044 ceph-mon[66366]: pgmap v307: 129 pgs: 129 active+clean; 621 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 -- 172.21.15.7:0/3626445868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101bb0 msgr2=0x7f0d9c10e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3626445868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c10e090 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b0a0 tx=0x7f0d9400b3b0 comp rx=0 tx=0).stop 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 -- 172.21.15.7:0/3626445868 shutdown_connections 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3626445868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c10e090 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3626445868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c107250 0x7f0d9c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3626445868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c101210 0x7f0d9c101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 -- 172.21.15.7:0/3626445868 >> 172.21.15.7:0/3626445868 conn(0x7f0d9c078140 msgr2=0x7f0d9c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 -- 172.21.15.7:0/3626445868 shutdown_connections 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 -- 172.21.15.7:0/3626445868 wait complete. 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 Processor -- start 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.754+0000 7f0da2b17700 1 -- start start 2024-03-19T00:36:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da2b17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101210 0x7f0d9c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da2b17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da2b17700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c107250 0x7f0d9c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da2b17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c111350 con 0x7f0d9c101210 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da2b17700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1111d0 con 0x7f0d9c107250 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da2b17700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1114d0 con 0x7f0d9c101bb0 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da08b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101210 0x7f0d9c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0d9bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0d9bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:38808/0 (socket says 172.21.15.7:38808) 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c107250 0x7f0d9c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0d9bfff700 1 -- 172.21.15.7:0/3104666666 learned_addr learned my addr 172.21.15.7:0/3104666666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 -- 172.21.15.7:0/3104666666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c101bb0 msgr2=0x7f0d9c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 -- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101210 msgr2=0x7f0d9c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101210 0x7f0d9c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 -- 172.21.15.7:0/3104666666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c199040 con 0x7f0d9c107250 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da08b3700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101210 0x7f0d9c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0da10b4700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c107250 0x7f0d9c198860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b070 tx=0x7f0d94004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.755+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94021070 con 0x7f0d9c107250 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.756+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d94007e30 con 0x7f0d9c107250 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.756+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1992d0 con 0x7f0d9c107250 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.756+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d940045c0 con 0x7f0d9c107250 2024-03-19T00:36:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.756+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a54f0 con 0x7f0d9c107250 2024-03-19T00:36:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.757+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c102900 con 0x7f0d9c107250 2024-03-19T00:36:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.758+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f0d94004760 con 0x7f0d9c107250 2024-03-19T00:36:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.758+0000 7f0d99ffb700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0d8c0a29a0 0x7f0d8c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.758+0000 7f0da08b3700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0d8c0a29a0 0x7f0d8c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.758+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0d940cfba0 con 0x7f0d9c107250 2024-03-19T00:36:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.759+0000 7f0da08b3700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0d8c0a29a0 0x7f0d8c0a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009fd0 tx=0x7f0d90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:34.763+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0d9409c8d0 con 0x7f0d9c107250 2024-03-19T00:36:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.025+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0d9c1a56c0 con 0x7f0d9c107250 2024-03-19T00:36:35.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.029+0000 7f0d99ffb700 1 -- 172.21.15.7:0/3104666666 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262650 (secure 0 0 0) 0x7f0d940cfe20 con 0x7f0d9c107250 2024-03-19T00:36:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0d8c0a29a0 msgr2=0x7f0d8c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0d8c0a29a0 0x7f0d8c0a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009fd0 tx=0x7f0d90009260 comp rx=0 tx=0).stop 2024-03-19T00:36:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c107250 msgr2=0x7f0d9c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c107250 0x7f0d9c198860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b070 tx=0x7f0d94004d00 comp rx=0 tx=0).stop 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 shutdown_connections 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0d8c0a29a0 0x7f0d8c0a4e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d9c107250 0x7f0d9c198860 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9c101210 0x7f0d9c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 --2- 172.21.15.7:0/3104666666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d9c101bb0 0x7f0d9c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 >> 172.21.15.7:0/3104666666 conn(0x7f0d9c078140 msgr2=0x7f0d9c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 shutdown_connections 2024-03-19T00:36:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:35.032+0000 7f0da2b17700 1 -- 172.21.15.7:0/3104666666 wait complete. 2024-03-19T00:36:35.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1790452807},{"type":"v1","addr":"172.21.15.7:6801","nonce":1790452807}]},"active_addr":"172.21.15.7:6801/1790452807","active_change":"2024-03-19T00:28:55.916832+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1214490391}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":829005753}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1833517360}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3984607080}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":233579778}]}]} 2024-03-19T00:36:35.846 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-19T00:36:35.846 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-19T00:36:35.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T00:36:36.101 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:36.158 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:35 smithi044 ceph-mon[66366]: pgmap v308: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:36.159 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:35 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3104666666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T00:36:36.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:35 smithi007 ceph-mon[62854]: pgmap v308: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:36.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:35 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3104666666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T00:36:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:35 smithi204 ceph-mon[66572]: pgmap v308: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:35 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3104666666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T00:36:38.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 -- 172.21.15.7:0/3036309224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102360 msgr2=0x7f2fe81027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:38.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/3036309224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102360 0x7f2fe81027e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80099c0 tx=0x7f2fd8009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 -- 172.21.15.7:0/3036309224 shutdown_connections 2024-03-19T00:36:38.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/3036309224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8102d20 0x7f2fe810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/3036309224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8108340 0x7f2fe8108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/3036309224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102360 0x7f2fe81027e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 -- 172.21.15.7:0/3036309224 >> 172.21.15.7:0/3036309224 conn(0x7f2fe8079440 msgr2=0x7f2fe8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:38.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:37 smithi007 ceph-mon[62854]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:38.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 -- 172.21.15.7:0/3036309224 shutdown_connections 2024-03-19T00:36:38.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.070+0000 7f2fef7b0700 1 -- 172.21.15.7:0/3036309224 wait complete. 2024-03-19T00:36:38.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 Processor -- start 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 -- start start 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8102360 0x7f2fe819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 0x7f2fe819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8108340 0x7f2fe81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8112470 con 0x7f2fe8102d20 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81122f0 con 0x7f2fe8108340 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fef7b0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81125f0 con 0x7f2fe8102360 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 0x7f2fe819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 0x7f2fe819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50242/0 (socket says 172.21.15.7:50242) 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 -- 172.21.15.7:0/1945183830 learned_addr learned my addr 172.21.15.7:0/1945183830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:38.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fedd4d700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8108340 0x7f2fe81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fed54c700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8102360 0x7f2fe819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 -- 172.21.15.7:0/1945183830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8102360 msgr2=0x7f2fe819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8102360 0x7f2fe819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 -- 172.21.15.7:0/1945183830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8108340 msgr2=0x7f2fe81987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8108340 0x7f2fe81987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fecd4b700 1 -- 172.21.15.7:0/1945183830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe8198fb0 con 0x7f2fe8102d20 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.071+0000 7f2fedd4d700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8108340 0x7f2fe81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.072+0000 7f2fecd4b700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 0x7f2fe819eb90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006010 tx=0x7f2fd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.072+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8026070 con 0x7f2fe8102d20 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.072+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe8199240 con 0x7f2fe8102d20 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.072+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fd8004630 con 0x7f2fe8102d20 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.072+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81a54b0 con 0x7f2fe8102d20 2024-03-19T00:36:38.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.072+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800fb20 con 0x7f2fe8102d20 2024-03-19T00:36:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.073+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc005330 con 0x7f2fe8102d20 2024-03-19T00:36:38.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.074+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f2fd8029070 con 0x7f2fe8102d20 2024-03-19T00:36:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.075+0000 7f2fde7fc700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2fd40a2920 0x7f2fd40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.075+0000 7f2fed54c700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2fd40a2920 0x7f2fd40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.075+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2fd80cf920 con 0x7f2fe8102d20 2024-03-19T00:36:38.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.075+0000 7f2fed54c700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2fd40a2920 0x7f2fd40a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009be0 tx=0x7f2fe4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.078+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2fd8030080 con 0x7f2fe8102d20 2024-03-19T00:36:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:37 smithi044 ceph-mon[66366]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:38.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:37 smithi204 ceph-mon[66572]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.282+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fcc0058f0 con 0x7f2fe8102d20 2024-03-19T00:36:38.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.283+0000 7f2fde7fc700 1 -- 172.21.15.7:0/1945183830 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f2fd8013070 con 0x7f2fe8102d20 2024-03-19T00:36:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:38.284 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:38.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2fd40a2920 msgr2=0x7f2fd40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:38.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2fd40a2920 0x7f2fd40a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009be0 tx=0x7f2fe4009260 comp rx=0 tx=0).stop 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 msgr2=0x7f2fe819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 0x7f2fe819eb90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006010 tx=0x7f2fd8004270 comp rx=0 tx=0).stop 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 shutdown_connections 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2fd40a2920 0x7f2fd40a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe8108340 0x7f2fe81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8102360 0x7f2fe819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 --2- 172.21.15.7:0/1945183830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe8102d20 0x7f2fe819eb90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 >> 172.21.15.7:0/1945183830 conn(0x7f2fe8079440 msgr2=0x7f2fe8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 shutdown_connections 2024-03-19T00:36:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:38.285+0000 7f2fef7b0700 1 -- 172.21.15.7:0/1945183830 wait complete. 2024-03-19T00:36:39.059 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-19T00:36:39.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T00:36:39.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:38 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1945183830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:38 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1945183830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:39.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:38 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1945183830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:40.158 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:39 smithi007 ceph-mon[62854]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:40.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:39 smithi204 ceph-mon[66572]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:39 smithi044 ceph-mon[66366]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 -- 172.21.15.7:0/1549631651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 msgr2=0x7f17201087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 --2- 172.21.15.7:0/1549631651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 0x7f17201087f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b0a0 tx=0x7f171c00b3b0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 -- 172.21.15.7:0/1549631651 shutdown_connections 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 --2- 172.21.15.7:0/1549631651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f172010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 --2- 172.21.15.7:0/1549631651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 0x7f17201087f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 --2- 172.21.15.7:0/1549631651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1720102360 0x7f17201027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 -- 172.21.15.7:0/1549631651 >> 172.21.15.7:0/1549631651 conn(0x7f1720079440 msgr2=0x7f1720079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 -- 172.21.15.7:0/1549631651 shutdown_connections 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.153+0000 7f1727f17700 1 -- 172.21.15.7:0/1549631651 wait complete. 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 Processor -- start 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 -- start start 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1720102360 0x7f17201a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f17201a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 0x7f172019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1720112500 con 0x7f1720108410 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1720112380 con 0x7f1720102d20 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1727f17700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1720112680 con 0x7f1720102360 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f17201a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17264b4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 0x7f172019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f17201a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47796/0 (socket says 172.21.15.7:47796) 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 -- 172.21.15.7:0/1440232071 learned_addr learned my addr 172.21.15.7:0/1440232071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f1725cb3700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1720102360 0x7f17201a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 -- 172.21.15.7:0/1440232071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1720102360 msgr2=0x7f17201a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:41.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1720102360 0x7f17201a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 -- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 msgr2=0x7f172019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.154+0000 7f17254b2700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 0x7f172019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f17254b2700 1 -- 172.21.15.7:0/1440232071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172019b180 con 0x7f1720102d20 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f17254b2700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f17201a0da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f171000e9c0 tx=0x7f171000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171000cbd0 con 0x7f1720102d20 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f171000ee60 con 0x7f1720102d20 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172019b470 con 0x7f1720102d20 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17100055b0 con 0x7f1720102d20 2024-03-19T00:36:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.155+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201a7730 con 0x7f1720102d20 2024-03-19T00:36:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.156+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1720068fd0 con 0x7f1720102d20 2024-03-19T00:36:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.157+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f1710004750 con 0x7f1720102d20 2024-03-19T00:36:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.158+0000 7f1716ffd700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f170c0a29a0 0x7f170c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.158+0000 7f1725cb3700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f170c0a29a0 0x7f170c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.158+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f17100cf2a0 con 0x7f1720102d20 2024-03-19T00:36:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.158+0000 7f1725cb3700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f170c0a29a0 0x7f170c0a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f171c009fd0 tx=0x7f171c00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:41.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.161+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1710053740 con 0x7f1720102d20 2024-03-19T00:36:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.365+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f172019bf50 con 0x7f1720102d20 2024-03-19T00:36:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.368+0000 7f1716ffd700 1 -- 172.21.15.7:0/1440232071 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23971 (secure 0 0 0) 0x7f171009bf40 con 0x7f1720102d20 2024-03-19T00:36:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:36:41.369 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:04.597542+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f170c0a29a0 msgr2=0x7f170c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f170c0a29a0 0x7f170c0a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f171c009fd0 tx=0x7f171c00b700 comp rx=0 tx=0).stop 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 msgr2=0x7f17201a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f17201a0da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f171000e9c0 tx=0x7f171000ecd0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 shutdown_connections 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f170c0a29a0 0x7f170c0a4e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1720108410 0x7f172019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1720102360 0x7f17201a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 --2- 172.21.15.7:0/1440232071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1720102d20 0x7f17201a0da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 >> 172.21.15.7:0/1440232071 conn(0x7f1720079440 msgr2=0x7f17200748b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 shutdown_connections 2024-03-19T00:36:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:41.370+0000 7f1727f17700 1 -- 172.21.15.7:0/1440232071 wait complete. 2024-03-19T00:36:42.108 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:41 smithi007 ceph-mon[62854]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2024-03-19T00:36:42.109 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:41 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1440232071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:42.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-19T00:36:42.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-19T00:36:42.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-19T00:36:42.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-19T00:36:42.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-19T00:36:42.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-19T00:36:42.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-19T00:36:42.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-19T00:36:42.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-19T00:36:42.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-19T00:36:42.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-19T00:36:42.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-19T00:36:42.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:41 smithi204 ceph-mon[66572]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2024-03-19T00:36:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:41 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1440232071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:41 smithi044 ceph-mon[66366]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2024-03-19T00:36:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:41 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1440232071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:36:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:36:44.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:43 smithi007 ceph-mon[62854]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-19T00:36:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:43 smithi204 ceph-mon[66572]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-19T00:36:44.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:43 smithi044 ceph-mon[66366]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-19T00:36:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:45 smithi007 ceph-mon[62854]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-19T00:36:46.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:45 smithi204 ceph-mon[66572]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-19T00:36:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:45 smithi044 ceph-mon[66366]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-19T00:36:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:47 smithi044 ceph-mon[66366]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-19T00:36:48.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:47 smithi007 ceph-mon[62854]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-19T00:36:48.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:47 smithi204 ceph-mon[66572]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-19T00:36:50.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:49 smithi007 ceph-mon[62854]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-19T00:36:50.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:49 smithi204 ceph-mon[66572]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-19T00:36:50.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:49 smithi044 ceph-mon[66366]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-19T00:36:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:51 smithi007 ceph-mon[62854]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-19T00:36:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:51 smithi204 ceph-mon[66572]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-19T00:36:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:51 smithi044 ceph-mon[66366]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-19T00:36:52.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 -- 172.21.15.7:0/1638794540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 msgr2=0x7fda90101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 --2- 172.21.15.7:0/1638794540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda90101870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda840099c0 tx=0x7fda84009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 -- 172.21.15.7:0/1638794540 shutdown_connections 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 --2- 172.21.15.7:0/1638794540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda90106570 0x7fda9010e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 --2- 172.21.15.7:0/1638794540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda90101870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 --2- 172.21.15.7:0/1638794540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda90101e40 0x7fda90105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 -- 172.21.15.7:0/1638794540 >> 172.21.15.7:0/1638794540 conn(0x7fda90078310 msgr2=0x7fda90078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 -- 172.21.15.7:0/1638794540 shutdown_connections 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.641+0000 7fda97019700 1 -- 172.21.15.7:0/1638794540 wait complete. 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 Processor -- start 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 -- start start 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda9019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda90101e40 0x7fda9019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:52.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda90106570 0x7fda90198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda90111430 con 0x7fda90101490 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda901112b0 con 0x7fda90101e40 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda97019700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda901115b0 con 0x7fda90106570 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda94db5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda9019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda94db5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda9019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58018/0 (socket says 172.21.15.7:58018) 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda94db5700 1 -- 172.21.15.7:0/159088676 learned_addr learned my addr 172.21.15.7:0/159088676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda955b6700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda90106570 0x7fda90198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.642+0000 7fda8ffff700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda90101e40 0x7fda9019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda94db5700 1 -- 172.21.15.7:0/159088676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda90106570 msgr2=0x7fda90198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda94db5700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda90106570 0x7fda90198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda94db5700 1 -- 172.21.15.7:0/159088676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda90101e40 msgr2=0x7fda9019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda94db5700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda90101e40 0x7fda9019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda94db5700 1 -- 172.21.15.7:0/159088676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda90198fe0 con 0x7fda90101490 2024-03-19T00:36:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda94db5700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda9019e680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fda840098e0 tx=0x7fda840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84026070 con 0x7fda90101490 2024-03-19T00:36:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda8400f460 con 0x7fda90101490 2024-03-19T00:36:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda90199270 con 0x7fda90101490 2024-03-19T00:36:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84009e60 con 0x7fda90101490 2024-03-19T00:36:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.643+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda90102e60 con 0x7fda90101490 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.644+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda90062a60 con 0x7fda90101490 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.645+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fda84029070 con 0x7fda90101490 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8dffb700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda800a29a0 0x7fda800a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fda840cfa10 con 0x7fda90101490 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8ffff700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda800a29a0 0x7fda800a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8dffb700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] conn(0x7fda800ad660 0x7fda800afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 --> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda8400f7b0 con 0x7fda800ad660 2024-03-19T00:36:52.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fda840cfe60 con 0x7fda90101490 2024-03-19T00:36:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda955b6700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] conn(0x7fda800ad660 0x7fda800afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.646+0000 7fda8ffff700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda800a29a0 0x7fda800a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda900ffcc0 tx=0x7fda7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.647+0000 7fda955b6700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] conn(0x7fda800ad660 0x7fda800afaa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.647+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== osd.0 v2:172.21.15.7:6802/2757458659 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda8400f7b0 con 0x7fda800ad660 2024-03-19T00:36:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.663+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 --> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda90062e60 con 0x7fda800ad660 2024-03-19T00:36:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda8dffb700 1 -- 172.21.15.7:0/159088676 <== osd.0 v2:172.21.15.7:6802/2757458659 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fda90062e60 con 0x7fda800ad660 2024-03-19T00:36:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] conn(0x7fda800ad660 msgr2=0x7fda800afaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] conn(0x7fda800ad660 0x7fda800afaa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda800a29a0 msgr2=0x7fda800a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda800a29a0 0x7fda800a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda900ffcc0 tx=0x7fda7c009260 comp rx=0 tx=0).stop 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 msgr2=0x7fda9019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 --2- 172.21.15.7:0/159088676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda90101490 0x7fda9019e680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fda840098e0 tx=0x7fda840045e0 comp rx=0 tx=0).stop 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda955b6700 1 -- 172.21.15.7:0/159088676 reap_dead start 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 shutdown_connections 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 >> 172.21.15.7:0/159088676 conn(0x7fda90078310 msgr2=0x7fda90100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 shutdown_connections 2024-03-19T00:36:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:52.664+0000 7fda97019700 1 -- 172.21.15.7:0/159088676 wait complete. 2024-03-19T00:36:54.168 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:53 smithi007 ceph-mon[62854]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-19T00:36:54.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:53 smithi204 ceph-mon[66572]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-19T00:36:54.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:53 smithi044 ceph-mon[66366]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-19T00:36:55.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 -- 172.21.15.7:0/537932653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 msgr2=0x7f68d0106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 --2- 172.21.15.7:0/537932653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d0106f50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f68b80099c0 tx=0x7f68b8009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 -- 172.21.15.7:0/537932653 shutdown_connections 2024-03-19T00:36:55.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 --2- 172.21.15.7:0/537932653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d0107630 0x7f68d010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 --2- 172.21.15.7:0/537932653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0102550 0x7f68d0102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 --2- 172.21.15.7:0/537932653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d0106f50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 -- 172.21.15.7:0/537932653 >> 172.21.15.7:0/537932653 conn(0x7f68d00fddc0 msgr2=0x7f68d01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 -- 172.21.15.7:0/537932653 shutdown_connections 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 -- 172.21.15.7:0/537932653 wait complete. 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.338+0000 7f68d5b61700 1 Processor -- start 2024-03-19T00:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 -- start start 2024-03-19T00:36:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d0102550 0x7f68d019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0107630 0x7f68d01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01124a0 con 0x7f68d0102f00 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0112320 con 0x7f68d0107630 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68d5b61700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0112620 con 0x7f68d0102550 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39390/0 (socket says 172.21.15.7:39390) 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 -- 172.21.15.7:0/2218867700 learned_addr learned my addr 172.21.15.7:0/2218867700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68cffff700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0107630 0x7f68d01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68cf7fe700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d0102550 0x7f68d019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 -- 172.21.15.7:0/2218867700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d0102550 msgr2=0x7f68d019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68d0102550 0x7f68d019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 -- 172.21.15.7:0/2218867700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0107630 msgr2=0x7f68d01987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0107630 0x7f68d01987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68ceffd700 1 -- 172.21.15.7:0/2218867700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d0198fa0 con 0x7f68d0102f00 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.339+0000 7f68cffff700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0107630 0x7f68d01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.340+0000 7f68ceffd700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d019eb80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005f50 tx=0x7f68b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.340+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8026070 con 0x7f68d0102f00 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.340+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b8003c70 con 0x7f68d0102f00 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.340+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d0199230 con 0x7f68d0102f00 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.340+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b800f9c0 con 0x7f68d0102f00 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.340+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d01a54a0 con 0x7f68d0102f00 2024-03-19T00:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.341+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68d0062aa0 con 0x7f68d0102f00 2024-03-19T00:36:55.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.342+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f68b8029070 con 0x7f68d0102f00 2024-03-19T00:36:55.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68ccff9700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68bc0a2920 0x7f68bc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68cf7fe700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68bc0a2920 0x7f68bc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f68b80cfa50 con 0x7f68d0102f00 2024-03-19T00:36:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68ccff9700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] conn(0x7f68bc0ad5e0 0x7f68bc0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 --> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68bc0b00f0 con 0x7f68bc0ad5e0 2024-03-19T00:36:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f68b80cfe00 con 0x7f68d0102f00 2024-03-19T00:36:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.343+0000 7f68cffff700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] conn(0x7f68bc0ad5e0 0x7f68bc0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.344+0000 7f68cf7fe700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68bc0a2920 0x7f68bc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009da0 tx=0x7f68c000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.344+0000 7f68cffff700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] conn(0x7f68bc0ad5e0 0x7f68bc0afa20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.345+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== osd.6 v2:172.21.15.7:6818/3051876259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68bc0b00f0 con 0x7f68bc0ad5e0 2024-03-19T00:36:55.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.367+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 --> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68d0062f20 con 0x7f68bc0ad5e0 2024-03-19T00:36:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.367+0000 7f68ccff9700 1 -- 172.21.15.7:0/2218867700 <== osd.6 v2:172.21.15.7:6818/3051876259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f68d0062f20 con 0x7f68bc0ad5e0 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.367+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] conn(0x7f68bc0ad5e0 msgr2=0x7f68bc0afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.367+0000 7f68d5b61700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] conn(0x7f68bc0ad5e0 0x7f68bc0afa20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68bc0a2920 msgr2=0x7f68bc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f68bc0a2920 0x7f68bc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009da0 tx=0x7f68c000b040 comp rx=0 tx=0).stop 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 msgr2=0x7f68d019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 --2- 172.21.15.7:0/2218867700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0102f00 0x7f68d019eb80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005f50 tx=0x7f68b8004270 comp rx=0 tx=0).stop 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68cffff700 1 -- 172.21.15.7:0/2218867700 reap_dead start 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 shutdown_connections 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 >> 172.21.15.7:0/2218867700 conn(0x7f68d00fddc0 msgr2=0x7f68d0100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 shutdown_connections 2024-03-19T00:36:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.368+0000 7f68d5b61700 1 -- 172.21.15.7:0/2218867700 wait complete. 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 -- 172.21.15.7:0/3251246357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 msgr2=0x7f29c8102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 --2- 172.21.15.7:0/3251246357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8102810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009990 tx=0x7f29bc009ca0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 -- 172.21.15.7:0/3251246357 shutdown_connections 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 --2- 172.21.15.7:0/3251246357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c8102d50 0x7f29c810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 --2- 172.21.15.7:0/3251246357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c8108470 0x7f29c8108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 --2- 172.21.15.7:0/3251246357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8102810 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 -- 172.21.15.7:0/3251246357 >> 172.21.15.7:0/3251246357 conn(0x7f29c80793d0 msgr2=0x7f29c80797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 -- 172.21.15.7:0/3251246357 shutdown_connections 2024-03-19T00:36:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.893+0000 7f29cecb1700 1 -- 172.21.15.7:0/3251246357 wait complete. 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 Processor -- start 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 -- start start 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c8102d50 0x7f29c8074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c8108470 0x7f29c8075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112480 con 0x7f29c8102390 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112300 con 0x7f29c8108470 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cecb1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112600 con 0x7f29c8102d50 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cca4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cca4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39416/0 (socket says 172.21.15.7:39416) 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cca4d700 1 -- 172.21.15.7:0/1251512862 learned_addr learned my addr 172.21.15.7:0/1251512862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29cd24e700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c8108470 0x7f29c8075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.894+0000 7f29c7fff700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c8102d50 0x7f29c8074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cca4d700 1 -- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c8102d50 msgr2=0x7f29c8074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cca4d700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29c8102d50 0x7f29c8074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cca4d700 1 -- 172.21.15.7:0/1251512862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c8108470 msgr2=0x7f29c8075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cca4d700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c8108470 0x7f29c8075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cca4d700 1 -- 172.21.15.7:0/1251512862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c81a6ee0 con 0x7f29c8102390 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cd24e700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c8108470 0x7f29c8075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cca4d700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8074610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29b800eb30 tx=0x7f29b800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b800cbd0 con 0x7f29c8102390 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29b80044d0 con 0x7f29c8102390 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c81a71d0 con 0x7f29c8102390 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8003df0 con 0x7f29c8102390 2024-03-19T00:36:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.895+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c81a76e0 con 0x7f29c8102390 2024-03-19T00:36:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.896+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29c8062a20 con 0x7f29c8102390 2024-03-19T00:36:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.897+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f29b8003f90 con 0x7f29c8102390 2024-03-19T00:36:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c5ffb700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f29b00a2920 0x7f29b00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c7fff700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f29b00a2920 0x7f29b00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f29b80cf210 con 0x7f29c8102390 2024-03-19T00:36:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c5ffb700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] conn(0x7f29b00ad5e0 0x7f29b00afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 --> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29b00b00f0 con 0x7f29b00ad5e0 2024-03-19T00:36:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f29b80cf660 con 0x7f29c8102390 2024-03-19T00:36:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29c7fff700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f29b00a2920 0x7f29b00a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0098e0 tx=0x7f29bc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29cd24e700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] conn(0x7f29b00ad5e0 0x7f29b00afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.898+0000 7f29cd24e700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] conn(0x7f29b00ad5e0 0x7f29b00afa20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.900+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== osd.11 v2:172.21.15.204:6824/849720251 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29b00b00f0 con 0x7f29b00ad5e0 2024-03-19T00:36:55.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.916+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 --> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29c8062ea0 con 0x7f29b00ad5e0 2024-03-19T00:36:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.916+0000 7f29c5ffb700 1 -- 172.21.15.7:0/1251512862 <== osd.11 v2:172.21.15.204:6824/849720251 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f29c8062ea0 con 0x7f29b00ad5e0 2024-03-19T00:36:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] conn(0x7f29b00ad5e0 msgr2=0x7f29b00afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] conn(0x7f29b00ad5e0 0x7f29b00afa20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f29b00a2920 msgr2=0x7f29b00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f29b00a2920 0x7f29b00a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0098e0 tx=0x7f29bc005960 comp rx=0 tx=0).stop 2024-03-19T00:36:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 msgr2=0x7f29c8074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 --2- 172.21.15.7:0/1251512862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c8102390 0x7f29c8074610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29b800eb30 tx=0x7f29b800ee40 comp rx=0 tx=0).stop 2024-03-19T00:36:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cd24e700 1 -- 172.21.15.7:0/1251512862 reap_dead start 2024-03-19T00:36:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 shutdown_connections 2024-03-19T00:36:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 >> 172.21.15.7:0/1251512862 conn(0x7f29c80793d0 msgr2=0x7f29c8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 shutdown_connections 2024-03-19T00:36:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:55.917+0000 7f29cecb1700 1 -- 172.21.15.7:0/1251512862 wait complete. 2024-03-19T00:36:56.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:55 smithi007 ceph-mon[62854]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-19T00:36:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:55 smithi204 ceph-mon[66572]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-19T00:36:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:55 smithi044 ceph-mon[66366]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]: dispatch 2024-03-19T00:36:57.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:56 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-19T00:36:57.230 INFO:teuthology.orchestra.run.smithi007.stdout:343597383705 2024-03-19T00:36:57.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]: dispatch 2024-03-19T00:36:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:56 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-03-19T00:36:57.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-19T00:36:57.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]: dispatch 2024-03-19T00:36:57.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:56 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-19T00:36:58.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 -- 172.21.15.7:0/2133309001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 msgr2=0x7f9de8108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 --2- 172.21.15.7:0/2133309001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 0x7f9de8108720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9de00099c0 tx=0x7f9de0009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 -- 172.21.15.7:0/2133309001 shutdown_connections 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 --2- 172.21.15.7:0/2133309001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de8102d20 0x7f9de810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 --2- 172.21.15.7:0/2133309001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 0x7f9de8108720 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 --2- 172.21.15.7:0/2133309001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de8102360 0x7f9de81027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 -- 172.21.15.7:0/2133309001 >> 172.21.15.7:0/2133309001 conn(0x7f9de8079440 msgr2=0x7f9de8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 -- 172.21.15.7:0/2133309001 shutdown_connections 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.009+0000 7f9def939700 1 -- 172.21.15.7:0/2133309001 wait complete. 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 Processor -- start 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 -- start start 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de8102360 0x7f9de819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de8102d20 0x7f9de819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 0x7f9de8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8112470 con 0x7f9de8108340 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81122f0 con 0x7f9de8102360 2024-03-19T00:36:58.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9def939700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81125f0 con 0x7f9de8102d20 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9deded6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 0x7f9de8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9ded6d5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de8102360 0x7f9de819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9deded6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 0x7f9de8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39430/0 (socket says 172.21.15.7:39430) 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9deded6700 1 -- 172.21.15.7:0/3191410328 learned_addr learned my addr 172.21.15.7:0/3191410328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.010+0000 7f9deced4700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de8102d20 0x7f9de819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9ded6d5700 1 -- 172.21.15.7:0/3191410328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de8102d20 msgr2=0x7f9de819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9ded6d5700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de8102d20 0x7f9de819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9ded6d5700 1 -- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 msgr2=0x7f9de8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9ded6d5700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de8108340 0x7f9de8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9ded6d5700 1 -- 172.21.15.7:0/3191410328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de8198ff0 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9ded6d5700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de8102360 0x7f9de819e640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9de00098e0 tx=0x7f9de0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0026070 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de0003c70 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de8199280 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81a53f0 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.011+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de000f9c0 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.012+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9de8062a60 con 0x7f9de8102360 2024-03-19T00:36:58.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.013+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f9de0029070 con 0x7f9de8102360 2024-03-19T00:36:58.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9de67fc700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9dd00a2920 0x7f9dd00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9deced4700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9dd00a2920 0x7f9dd00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9de00cfe40 con 0x7f9de8102360 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9de67fc700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] conn(0x7f9dd00ada80 0x7f9dd00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 --> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dd00b05f0 con 0x7f9dd00ada80 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9de00d0290 con 0x7f9de8102360 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9deded6700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] conn(0x7f9dd00ada80 0x7f9dd00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.014+0000 7f9deced4700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9dd00a2920 0x7f9dd00a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9de8199ed0 tx=0x7f9ddc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.015+0000 7f9deded6700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] conn(0x7f9dd00ada80 0x7f9dd00aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:58.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.015+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== osd.9 v2:172.21.15.7:6826/1022710322 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9dd00b05f0 con 0x7f9dd00ada80 2024-03-19T00:36:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.033+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 --> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9de8062ee0 con 0x7f9dd00ada80 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-19T00:36:58.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-03-19T00:36:58.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-19T00:36:58.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]': finished 2024-03-19T00:36:58.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-19T00:36:58.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:57 smithi007 ceph-mon[62854]: osdmap e90: 12 total, 12 up, 12 in 2024-03-19T00:36:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.033+0000 7f9de67fc700 1 -- 172.21.15.7:0/3191410328 <== osd.9 v2:172.21.15.7:6826/1022710322 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9de8062ee0 con 0x7f9dd00ada80 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.033+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] conn(0x7f9dd00ada80 msgr2=0x7f9dd00aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] conn(0x7f9dd00ada80 0x7f9dd00aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9dd00a2920 msgr2=0x7f9dd00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9dd00a2920 0x7f9dd00a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9de8199ed0 tx=0x7f9ddc009260 comp rx=0 tx=0).stop 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de8102360 msgr2=0x7f9de819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 --2- 172.21.15.7:0/3191410328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de8102360 0x7f9de819e640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9de00098e0 tx=0x7f9de0004270 comp rx=0 tx=0).stop 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9deded6700 1 -- 172.21.15.7:0/3191410328 reap_dead start 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 shutdown_connections 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 >> 172.21.15.7:0/3191410328 conn(0x7f9de8079440 msgr2=0x7f9de8101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 shutdown_connections 2024-03-19T00:36:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.034+0000 7f9def939700 1 -- 172.21.15.7:0/3191410328 wait complete. 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-19T00:36:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-03-19T00:36:58.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-19T00:36:58.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]': finished 2024-03-19T00:36:58.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-19T00:36:58.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:57 smithi204 ceph-mon[66572]: osdmap e90: 12 total, 12 up, 12 in 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-19T00:36:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-03-19T00:36:58.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-19T00:36:58.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]': finished 2024-03-19T00:36:58.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-19T00:36:58.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:57 smithi044 ceph-mon[66366]: osdmap e90: 12 total, 12 up, 12 in 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 -- 172.21.15.7:0/4263266502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 msgr2=0x7f09d4105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 --2- 172.21.15.7:0/4263266502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4105e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0099c0 tx=0x7f09bc009cd0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 -- 172.21.15.7:0/4263266502 shutdown_connections 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 --2- 172.21.15.7:0/4263266502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09d41064f0 0x7f09d410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 --2- 172.21.15.7:0/4263266502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d41014a0 0x7f09d4101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 --2- 172.21.15.7:0/4263266502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4105e10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.449+0000 7f09d9d84700 1 -- 172.21.15.7:0/4263266502 >> 172.21.15.7:0/4263266502 conn(0x7f09d4077260 msgr2=0x7f09d4077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.450+0000 7f09d9d84700 1 -- 172.21.15.7:0/4263266502 shutdown_connections 2024-03-19T00:36:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.450+0000 7f09d9d84700 1 -- 172.21.15.7:0/4263266502 wait complete. 2024-03-19T00:36:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.450+0000 7f09d9d84700 1 Processor -- start 2024-03-19T00:36:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.450+0000 7f09d9d84700 1 -- start start 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09d41014a0 0x7f09d4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d41064f0 0x7f09d41a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4111380 con 0x7f09d4101e50 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4111200 con 0x7f09d41014a0 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4111500 con 0x7f09d41064f0 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09caffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09caffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39452/0 (socket says 172.21.15.7:39452) 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09caffd700 1 -- 172.21.15.7:0/3918257832 learned_addr learned my addr 172.21.15.7:0/3918257832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d41064f0 0x7f09d41a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d37fe700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09d41014a0 0x7f09d4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 -- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09d41014a0 msgr2=0x7f09d4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09d41014a0 0x7f09d4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 -- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 msgr2=0x7f09d4199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 -- 172.21.15.7:0/3918257832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41a4ed0 con 0x7f09d41064f0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09caffd700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d4101e50 0x7f09d4199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d3fff700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d41064f0 0x7f09d41a4990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00efb0 tx=0x7f09cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc003a00 con 0x7f09d41064f0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.451+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d41a50a0 con 0x7f09d41064f0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.452+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09cc00ce50 con 0x7f09d41064f0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.452+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0047c0 con 0x7f09d41064f0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.452+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d41a55e0 con 0x7f09d41064f0 2024-03-19T00:36:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.452+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09d4062a20 con 0x7f09d41064f0 2024-03-19T00:36:58.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.454+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f09cc012050 con 0x7f09d41064f0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.454+0000 7f09d17fa700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09c00a2920 0x7f09c00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.454+0000 7f09d37fe700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09c00a2920 0x7f09c00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.454+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f09cc0ced00 con 0x7f09d41064f0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.455+0000 7f09d17fa700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] conn(0x7f09c00ada80 0x7f09c00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.455+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 --> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09c00b05f0 con 0x7f09c00ada80 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.455+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f09cc0cf0b0 con 0x7f09d41064f0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.455+0000 7f09d37fe700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09c00a2920 0x7f09c00a4de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009c60 tx=0x7f09c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.455+0000 7f09caffd700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] conn(0x7f09c00ada80 0x7f09c00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.455+0000 7f09caffd700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] conn(0x7f09c00ada80 0x7f09c00aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:36:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.456+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== osd.4 v2:172.21.15.44:6808/3332813314 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09c00b05f0 con 0x7f09c00ada80 2024-03-19T00:36:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.472+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 --> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09d4062e20 con 0x7f09c00ada80 2024-03-19T00:36:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.472+0000 7f09d17fa700 1 -- 172.21.15.7:0/3918257832 <== osd.4 v2:172.21.15.44:6808/3332813314 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09d4062e20 con 0x7f09c00ada80 2024-03-19T00:36:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] conn(0x7f09c00ada80 msgr2=0x7f09c00aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] conn(0x7f09c00ada80 0x7f09c00aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09c00a2920 msgr2=0x7f09c00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f09c00a2920 0x7f09c00a4de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009c60 tx=0x7f09c4009260 comp rx=0 tx=0).stop 2024-03-19T00:36:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d41064f0 msgr2=0x7f09d41a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:36:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 --2- 172.21.15.7:0/3918257832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d41064f0 0x7f09d41a4990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00efb0 tx=0x7f09cc00c3a0 comp rx=0 tx=0).stop 2024-03-19T00:36:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d3fff700 1 -- 172.21.15.7:0/3918257832 reap_dead start 2024-03-19T00:36:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 shutdown_connections 2024-03-19T00:36:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 >> 172.21.15.7:0/3918257832 conn(0x7f09d4077260 msgr2=0x7f09d40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:36:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 shutdown_connections 2024-03-19T00:36:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:36:58.473+0000 7f09d9d84700 1 -- 172.21.15.7:0/3918257832 wait complete. 2024-03-19T00:36:58.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:58 smithi007 ceph-mon[62854]: osdmap e91: 12 total, 12 up, 12 in 2024-03-19T00:36:59.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:58 smithi204 ceph-mon[66572]: osdmap e91: 12 total, 12 up, 12 in 2024-03-19T00:36:59.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:58 smithi044 ceph-mon[66366]: osdmap e91: 12 total, 12 up, 12 in 2024-03-19T00:37:00.041 INFO:teuthology.orchestra.run.smithi007.stdout:34359738447 2024-03-19T00:37:00.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-19T00:37:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:36:59 smithi007 ceph-mon[62854]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-19T00:37:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:36:59 smithi204 ceph-mon[66572]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-19T00:37:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:36:59 smithi044 ceph-mon[66366]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-19T00:37:01.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:00 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:37:01.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:00 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:37:01.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:00 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 -- 172.21.15.7:0/1290223233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406b460 msgr2=0x7f0664106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 --2- 172.21.15.7:0/1290223233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406b460 0x7f0664106f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f064c0099c0 tx=0x7f064c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 -- 172.21.15.7:0/1290223233 shutdown_connections 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 --2- 172.21.15.7:0/1290223233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0664107690 0x7f066410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 --2- 172.21.15.7:0/1290223233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f066406aab0 0x7f066406ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 --2- 172.21.15.7:0/1290223233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406b460 0x7f0664106f60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.422+0000 7f0668a10700 1 -- 172.21.15.7:0/1290223233 >> 172.21.15.7:0/1290223233 conn(0x7f0664077260 msgr2=0x7f0664077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 -- 172.21.15.7:0/1290223233 shutdown_connections 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 -- 172.21.15.7:0/1290223233 wait complete. 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 Processor -- start 2024-03-19T00:37:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 -- start start 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406aab0 0x7f06641a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f066406b460 0x7f06641a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.423+0000 7f0668a10700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0664107690 0x7f066419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0668a10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06641124b0 con 0x7f066406aab0 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0668a10700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0664112330 con 0x7f066406b460 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0668a10700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0664112630 con 0x7f0664107690 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406aab0 0x7f06641a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406aab0 0x7f06641a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39478/0 (socket says 172.21.15.7:39478) 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0661d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f066406b460 0x7f06641a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 -- 172.21.15.7:0/2353659080 learned_addr learned my addr 172.21.15.7:0/2353659080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0662d9d700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0664107690 0x7f066419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 -- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0664107690 msgr2=0x7f066419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0664107690 0x7f066419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 -- 172.21.15.7:0/2353659080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f066406b460 msgr2=0x7f06641a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f066406b460 0x7f06641a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 -- 172.21.15.7:0/2353659080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066419b1f0 con 0x7f066406aab0 2024-03-19T00:37:01.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f066259c700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406aab0 0x7f06641a08d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f065c007f10 tx=0x7f065c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:01.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00dbd0 con 0x7f066406aab0 2024-03-19T00:37:01.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f065c00dd70 con 0x7f066406aab0 2024-03-19T00:37:01.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066419b4e0 con 0x7f066406aab0 2024-03-19T00:37:01.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c005960 con 0x7f066406aab0 2024-03-19T00:37:01.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.424+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06641a77a0 con 0x7f066406aab0 2024-03-19T00:37:01.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.425+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f066410dfb0 con 0x7f066406aab0 2024-03-19T00:37:01.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f065c01a020 con 0x7f066406aab0 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f065b7fe700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f06500a2920 0x7f06500a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f0661d9b700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f06500a2920 0x7f06500a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f065c0cedc0 con 0x7f066406aab0 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f065b7fe700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] conn(0x7f06500ada80 0x7f06500aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 --> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f065c00be50 con 0x7f06500ada80 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.427+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f065c0cf210 con 0x7f066406aab0 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.428+0000 7f0662d9d700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] conn(0x7f06500ada80 0x7f06500aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.428+0000 7f0661d9b700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f06500a2920 0x7f06500a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f064c009990 tx=0x7f064c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.428+0000 7f0662d9d700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] conn(0x7f06500ada80 0x7f06500aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.429+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== osd.5 v2:172.21.15.204:6808/3713019537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f065c00be50 con 0x7f06500ada80 2024-03-19T00:37:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.448+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 --> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0664103a40 con 0x7f06500ada80 2024-03-19T00:37:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.448+0000 7f065b7fe700 1 -- 172.21.15.7:0/2353659080 <== osd.5 v2:172.21.15.204:6808/3713019537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0664103a40 con 0x7f06500ada80 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] conn(0x7f06500ada80 msgr2=0x7f06500aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] conn(0x7f06500ada80 0x7f06500aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f06500a2920 msgr2=0x7f06500a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f06500a2920 0x7f06500a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f064c009990 tx=0x7f064c005fb0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406aab0 msgr2=0x7f06641a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 --2- 172.21.15.7:0/2353659080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f066406aab0 0x7f06641a08d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f065c007f10 tx=0x7f065c00d3b0 comp rx=0 tx=0).stop 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0662d9d700 1 -- 172.21.15.7:0/2353659080 reap_dead start 2024-03-19T00:37:01.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 shutdown_connections 2024-03-19T00:37:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 >> 172.21.15.7:0/2353659080 conn(0x7f0664077260 msgr2=0x7f066410d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 shutdown_connections 2024-03-19T00:37:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.449+0000 7f0668a10700 1 -- 172.21.15.7:0/2353659080 wait complete. 2024-03-19T00:37:02.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:01 smithi007 ceph-mon[62854]: pgmap v323: 129 pgs: 2 activating, 127 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 -- 172.21.15.7:0/3567859952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee81072a0 msgr2=0x7f4ee8107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/3567859952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee81072a0 0x7f4ee8107680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009990 tx=0x7f4ee0009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 -- 172.21.15.7:0/3567859952 shutdown_connections 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/3567859952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee8101bb0 0x7f4ee810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/3567859952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee81072a0 0x7f4ee8107680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/3567859952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee81011f0 0x7f4ee8101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 -- 172.21.15.7:0/3567859952 >> 172.21.15.7:0/3567859952 conn(0x7f4ee8078220 msgr2=0x7f4ee8078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 -- 172.21.15.7:0/3567859952 shutdown_connections 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.636+0000 7f4eee0cf700 1 -- 172.21.15.7:0/3567859952 wait complete. 2024-03-19T00:37:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 Processor -- start 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 -- start start 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee81011f0 0x7f4ee8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee8101bb0 0x7f4ee8199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee81072a0 0x7f4ee81a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8111390 con 0x7f4ee8101bb0 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8111210 con 0x7f4ee81072a0 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.637+0000 7f4eee0cf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8111510 con 0x7f4ee81011f0 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee8101bb0 0x7f4ee8199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee81072a0 0x7f4ee81a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee8101bb0 0x7f4ee8199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39496/0 (socket says 172.21.15.7:39496) 2024-03-19T00:37:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 -- 172.21.15.7:0/523204232 learned_addr learned my addr 172.21.15.7:0/523204232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee81011f0 0x7f4ee8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 -- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee81011f0 msgr2=0x7f4ee8199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee81011f0 0x7f4ee8199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 -- 172.21.15.7:0/523204232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee81072a0 msgr2=0x7f4ee81a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee81072a0 0x7f4ee81a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 -- 172.21.15.7:0/523204232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee81a4cd0 con 0x7f4ee8101bb0 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee6ffd700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee8101bb0 0x7f4ee8199c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000c9b0 tx=0x7f4ed000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0007b00 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed000ce50 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81a4f60 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0018660 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.638+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a5410 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.639+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ee80629a0 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f4ed000f450 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee4ff9700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ed40a2920 0x7f4ed40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee77fe700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ed40a2920 0x7f4ed40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4ed00cf2f0 con 0x7f4ee8101bb0 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee4ff9700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] conn(0x7f4ed40ada80 0x7f4ed40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 --> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ed0015e70 con 0x7f4ed40ada80 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.641+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4ed00cf740 con 0x7f4ee8101bb0 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.642+0000 7f4ee77fe700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ed40a2920 0x7f4ed40a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000b5c0 tx=0x7f4ee0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.642+0000 7f4ee7fff700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] conn(0x7f4ed40ada80 0x7f4ed40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.642+0000 7f4ee7fff700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] conn(0x7f4ed40ada80 0x7f4ed40aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.643+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== osd.2 v2:172.21.15.204:6800/281046450 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ed0015e70 con 0x7f4ed40ada80 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.659+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 --> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ee8062e90 con 0x7f4ed40ada80 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.659+0000 7f4ee4ff9700 1 -- 172.21.15.7:0/523204232 <== osd.2 v2:172.21.15.204:6800/281046450 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ee8062e90 con 0x7f4ed40ada80 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] conn(0x7f4ed40ada80 msgr2=0x7f4ed40aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] conn(0x7f4ed40ada80 0x7f4ed40aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ed40a2920 msgr2=0x7f4ed40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ed40a2920 0x7f4ed40a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000b5c0 tx=0x7f4ee0005960 comp rx=0 tx=0).stop 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee8101bb0 msgr2=0x7f4ee8199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 --2- 172.21.15.7:0/523204232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee8101bb0 0x7f4ee8199c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000c9b0 tx=0x7f4ed000ccc0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4ee7fff700 1 -- 172.21.15.7:0/523204232 reap_dead start 2024-03-19T00:37:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 shutdown_connections 2024-03-19T00:37:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 >> 172.21.15.7:0/523204232 conn(0x7f4ee8078220 msgr2=0x7f4ee8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 shutdown_connections 2024-03-19T00:37:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.660+0000 7f4eee0cf700 1 -- 172.21.15.7:0/523204232 wait complete. 2024-03-19T00:37:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:01 smithi204 ceph-mon[66572]: pgmap v323: 129 pgs: 2 activating, 127 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:02.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:01 smithi044 ceph-mon[66366]: pgmap v323: 129 pgs: 2 activating, 127 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 -- 172.21.15.7:0/1802692122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102360 msgr2=0x7f87681027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 --2- 172.21.15.7:0/1802692122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102360 0x7f87681027e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f875c0099c0 tx=0x7f875c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 -- 172.21.15.7:0/1802692122 shutdown_connections 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 --2- 172.21.15.7:0/1802692122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768102d20 0x7f876810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 --2- 172.21.15.7:0/1802692122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8768108340 0x7f8768108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 --2- 172.21.15.7:0/1802692122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102360 0x7f87681027e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 -- 172.21.15.7:0/1802692122 >> 172.21.15.7:0/1802692122 conn(0x7f8768079440 msgr2=0x7f8768079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 -- 172.21.15.7:0/1802692122 shutdown_connections 2024-03-19T00:37:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.864+0000 7f876cbc0700 1 -- 172.21.15.7:0/1802692122 wait complete. 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 Processor -- start 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 -- start start 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8768102360 0x7f87681a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102d20 0x7f87681a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768108340 0x7f876819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8768112470 con 0x7f8768102d20 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87681122f0 con 0x7f8768102360 2024-03-19T00:37:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876cbc0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87681125f0 con 0x7f8768108340 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f876659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8768102360 0x7f87681a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f8766d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768108340 0x7f876819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f8766d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768108340 0x7f876819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:47278/0 (socket says 172.21.15.7:47278) 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f8766d9d700 1 -- 172.21.15.7:0/324372046 learned_addr learned my addr 172.21.15.7:0/324372046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.865+0000 7f8765d9b700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102d20 0x7f87681a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f8766d9d700 1 -- 172.21.15.7:0/324372046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8768102360 msgr2=0x7f87681a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f8766d9d700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8768102360 0x7f87681a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f8766d9d700 1 -- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102d20 msgr2=0x7f87681a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f8766d9d700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8768102d20 0x7f87681a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f8766d9d700 1 -- 172.21.15.7:0/324372046 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876819b150 con 0x7f8768108340 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f8766d9d700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768108340 0x7f876819a9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f876000ef30 tx=0x7f876000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87600092e0 con 0x7f8768108340 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876819b3e0 con 0x7f8768108340 2024-03-19T00:37:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8760004bb0 con 0x7f8768108340 2024-03-19T00:37:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760004020 con 0x7f8768108340 2024-03-19T00:37:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.866+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87681a7700 con 0x7f8768108340 2024-03-19T00:37:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.867+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87680762b0 con 0x7f8768108340 2024-03-19T00:37:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.868+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f876001b030 con 0x7f8768108340 2024-03-19T00:37:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.869+0000 7f87577fe700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87500a2920 0x7f87500a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.869+0000 7f876659c700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87500a2920 0x7f87500a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.869+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f87600ceb40 con 0x7f8768108340 2024-03-19T00:37:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.869+0000 7f87577fe700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] conn(0x7f87500ada80 0x7f87500aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.869+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 --> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87500b05f0 con 0x7f87500ada80 2024-03-19T00:37:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.869+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f87600ceef0 con 0x7f8768108340 2024-03-19T00:37:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.870+0000 7f8765d9b700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] conn(0x7f87500ada80 0x7f87500aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.870+0000 7f876659c700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87500a2920 0x7f87500a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f876819bb30 tx=0x7f8758009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.870+0000 7f8765d9b700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] conn(0x7f87500ada80 0x7f87500aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.871+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== osd.8 v2:172.21.15.204:6816/1654194341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87500b05f0 con 0x7f87500ada80 2024-03-19T00:37:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.888+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 --> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87681039f0 con 0x7f87500ada80 2024-03-19T00:37:02.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.889+0000 7f87577fe700 1 -- 172.21.15.7:0/324372046 <== osd.8 v2:172.21.15.204:6816/1654194341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f87681039f0 con 0x7f87500ada80 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.889+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] conn(0x7f87500ada80 msgr2=0x7f87500aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.889+0000 7f876cbc0700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] conn(0x7f87500ada80 0x7f87500aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.889+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87500a2920 msgr2=0x7f87500a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87500a2920 0x7f87500a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f876819bb30 tx=0x7f8758009260 comp rx=0 tx=0).stop 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768108340 msgr2=0x7f876819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 --2- 172.21.15.7:0/324372046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8768108340 0x7f876819a9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f876000ef30 tx=0x7f876000c3a0 comp rx=0 tx=0).stop 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f8766d9d700 1 -- 172.21.15.7:0/324372046 reap_dead start 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 shutdown_connections 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 >> 172.21.15.7:0/324372046 conn(0x7f8768079440 msgr2=0x7f8768074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 shutdown_connections 2024-03-19T00:37:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:02.890+0000 7f876cbc0700 1 -- 172.21.15.7:0/324372046 wait complete. 2024-03-19T00:37:03.592 INFO:teuthology.orchestra.run.smithi007.stdout:287762808868 2024-03-19T00:37:03.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-19T00:37:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:03 smithi007 ceph-mon[62854]: pgmap v324: 129 pgs: 1 activating, 128 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:03 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:03 smithi204 ceph-mon[66572]: pgmap v324: 129 pgs: 1 activating, 128 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:03 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:03 smithi044 ceph-mon[66366]: pgmap v324: 129 pgs: 1 activating, 128 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:03 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:04.247 INFO:teuthology.orchestra.run.smithi007.stdout:197568495667 2024-03-19T00:37:04.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-19T00:37:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 -- 172.21.15.7:0/1035715053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 msgr2=0x7fc428106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1035715053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 0x7fc428106f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0099c0 tx=0x7fc41c009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 -- 172.21.15.7:0/1035715053 shutdown_connections 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1035715053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428107630 0x7fc42810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1035715053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 0x7fc428102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1035715053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 0x7fc428106f50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 -- 172.21.15.7:0/1035715053 >> 172.21.15.7:0/1035715053 conn(0x7fc4280fddc0 msgr2=0x7fc4281001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 -- 172.21.15.7:0/1035715053 shutdown_connections 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.320+0000 7fc42e69c700 1 -- 172.21.15.7:0/1035715053 wait complete. 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 Processor -- start 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 -- start start 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 0x7fc42819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 0x7fc42819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc427fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 0x7fc42819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc4277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 0x7fc42819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc427fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 0x7fc42819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35514/0 (socket says 172.21.15.7:35514) 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc427fff700 1 -- 172.21.15.7:0/1697970194 learned_addr learned my addr 172.21.15.7:0/1697970194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428107630 0x7fc428198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281124c0 con 0x7fc428102f00 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112340 con 0x7fc428102550 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.321+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112640 con 0x7fc428107630 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc42cc39700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428107630 0x7fc428198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc427fff700 1 -- 172.21.15.7:0/1697970194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428107630 msgr2=0x7fc428198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc427fff700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428107630 0x7fc428198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc427fff700 1 -- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 msgr2=0x7fc42819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc427fff700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 0x7fc42819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc427fff700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc428199020 con 0x7fc428102550 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc4277fe700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc428102f00 0x7fc42819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc427fff700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 0x7fc42819e6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e9a0 tx=0x7fc41800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41800cbd0 con 0x7fc428102550 2024-03-19T00:37:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc41800cd70 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc428199310 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418018680 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.322+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4281a5520 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.323+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc428062aa0 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.325+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fc418010430 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.325+0000 7fc4257fa700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc4100a2920 0x7fc4100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.325+0000 7fc4277fe700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc4100a2920 0x7fc4100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.325+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc4180cfa00 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.326+0000 7fc4257fa700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] conn(0x7fc4100ada80 0x7fc4100aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.326+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc418023e50 con 0x7fc4100ada80 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.326+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fc4180cfdb0 con 0x7fc428102550 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.326+0000 7fc42cc39700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] conn(0x7fc4100ada80 0x7fc4100aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.326+0000 7fc4277fe700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc4100a2920 0x7fc4100a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc41c000c00 tx=0x7fc41c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.326+0000 7fc42cc39700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] conn(0x7fc4100ada80 0x7fc4100aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.327+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== osd.3 v2:172.21.15.7:6810/375379923 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc418023e50 con 0x7fc4100ada80 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.343+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 --> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc428062f20 con 0x7fc4100ada80 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc4257fa700 1 -- 172.21.15.7:0/1697970194 <== osd.3 v2:172.21.15.7:6810/375379923 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc428062f20 con 0x7fc4100ada80 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] conn(0x7fc4100ada80 msgr2=0x7fc4100aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] conn(0x7fc4100ada80 0x7fc4100aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc4100a2920 msgr2=0x7fc4100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc4100a2920 0x7fc4100a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc41c000c00 tx=0x7fc41c005fb0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 msgr2=0x7fc42819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 --2- 172.21.15.7:0/1697970194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc428102550 0x7fc42819e6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e9a0 tx=0x7fc41800ecb0 comp rx=0 tx=0).stop 2024-03-19T00:37:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42cc39700 1 -- 172.21.15.7:0/1697970194 reap_dead start 2024-03-19T00:37:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 shutdown_connections 2024-03-19T00:37:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 >> 172.21.15.7:0/1697970194 conn(0x7fc4280fddc0 msgr2=0x7fc428100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 shutdown_connections 2024-03-19T00:37:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:01.344+0000 7fc42e69c700 1 -- 172.21.15.7:0/1697970194 wait complete. 2024-03-19T00:37:05.416 INFO:teuthology.orchestra.run.smithi007.stdout:107374182467 2024-03-19T00:37:05.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-19T00:37:05.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:05 smithi007 ceph-mon[62854]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:05 smithi204 ceph-mon[66572]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:05 smithi044 ceph-mon[66366]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:06.101 INFO:teuthology.orchestra.run.smithi007.stdout:253403070506 2024-03-19T00:37:06.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-19T00:37:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 -- 172.21.15.7:0/2386633180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 msgr2=0x7f25b006a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 --2- 172.21.15.7:0/2386633180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b006a420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f25a00099c0 tx=0x7f25a0009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 -- 172.21.15.7:0/2386633180 shutdown_connections 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 --2- 172.21.15.7:0/2386633180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b006a960 0x7f25b010f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 --2- 172.21.15.7:0/2386633180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0108360 0x7f25b0108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 --2- 172.21.15.7:0/2386633180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b006a420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 -- 172.21.15.7:0/2386633180 >> 172.21.15.7:0/2386633180 conn(0x7f25b0078140 msgr2=0x7f25b0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 -- 172.21.15.7:0/2386633180 shutdown_connections 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.638+0000 7f25b6509700 1 -- 172.21.15.7:0/2386633180 wait complete. 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 Processor -- start 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 -- start start 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b01012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b006a960 0x7f25b0101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0108360 0x7f25b0101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0112470 con 0x7f25b0069fc0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01122f0 con 0x7f25b006a960 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.639+0000 7f25b6509700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01125f0 con 0x7f25b0108360 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b01012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b01012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35236/0 (socket says 172.21.15.7:35236) 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 -- 172.21.15.7:0/1291017573 learned_addr learned my addr 172.21.15.7:0/1291017573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25af7fe700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b006a960 0x7f25b0101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25b4aa6700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0108360 0x7f25b0101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 -- 172.21.15.7:0/1291017573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0108360 msgr2=0x7f25b0101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0108360 0x7f25b0101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 -- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b006a960 msgr2=0x7f25b0101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b006a960 0x7f25b0101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 -- 172.21.15.7:0/1291017573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b01a6f10 con 0x7f25b0069fc0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25affff700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b01012e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f25a400e9c0 tx=0x7f25a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a400cbd0 con 0x7f25b0069fc0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b01a7200 con 0x7f25b0069fc0 2024-03-19T00:37:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a400ee50 con 0x7f25b0069fc0 2024-03-19T00:37:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4005550 con 0x7f25b0069fc0 2024-03-19T00:37:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.640+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b01a7710 con 0x7f25b0069fc0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.641+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25b0062a20 con 0x7f25b0069fc0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.642+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f25a4004750 con 0x7f25b0069fc0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25ad7fa700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25980a2920 0x7f25980a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25af7fe700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25980a2920 0x7f25980a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f25a40cf580 con 0x7f25b0069fc0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25ad7fa700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] conn(0x7f25980ada80 0x7f25980aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 --> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25a401aec0 con 0x7f25980ada80 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f25a40cf9d0 con 0x7f25b0069fc0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.643+0000 7f25b4aa6700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] conn(0x7f25980ada80 0x7f25980aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.644+0000 7f25af7fe700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25980a2920 0x7f25980a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f25b0102d70 tx=0x7f25a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.644+0000 7f25b4aa6700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] conn(0x7f25980ada80 0x7f25980aff20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.645+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== osd.10 v2:172.21.15.44:6824/443402515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25a401aec0 con 0x7f25980ada80 2024-03-19T00:37:06.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.667+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 --> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f25b0062ea0 con 0x7f25980ada80 2024-03-19T00:37:06.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.667+0000 7f25ad7fa700 1 -- 172.21.15.7:0/1291017573 <== osd.10 v2:172.21.15.44:6824/443402515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f25b0062ea0 con 0x7f25980ada80 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] conn(0x7f25980ada80 msgr2=0x7f25980aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] conn(0x7f25980ada80 0x7f25980aff20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25980a2920 msgr2=0x7f25980a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f25980a2920 0x7f25980a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f25b0102d70 tx=0x7f25a0005fb0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 msgr2=0x7f25b01012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 --2- 172.21.15.7:0/1291017573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b0069fc0 0x7f25b01012e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f25a400e9c0 tx=0x7f25a400ecd0 comp rx=0 tx=0).stop 2024-03-19T00:37:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b4aa6700 1 -- 172.21.15.7:0/1291017573 reap_dead start 2024-03-19T00:37:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 shutdown_connections 2024-03-19T00:37:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 >> 172.21.15.7:0/1291017573 conn(0x7f25b0078140 msgr2=0x7f25b00fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 shutdown_connections 2024-03-19T00:37:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:06.668+0000 7f25b6509700 1 -- 172.21.15.7:0/1291017573 wait complete. 2024-03-19T00:37:07.098 INFO:teuthology.orchestra.run.smithi007.stdout:141733920829 2024-03-19T00:37:07.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-19T00:37:07.332 INFO:teuthology.orchestra.run.smithi007.stdout:77309411399 2024-03-19T00:37:07.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-19T00:37:07.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:07 smithi007 ceph-mon[62854]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 225 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:07.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:07 smithi204 ceph-mon[66572]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 225 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:07 smithi044 ceph-mon[66366]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 225 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 -- 172.21.15.7:0/25083934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c1014a0 msgr2=0x7f759c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 --2- 172.21.15.7:0/25083934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c1014a0 0x7f759c101880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f758c009990 tx=0x7f758c009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 -- 172.21.15.7:0/25083934 shutdown_connections 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 --2- 172.21.15.7:0/25083934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c106530 0x7f759c10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 --2- 172.21.15.7:0/25083934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c1014a0 0x7f759c101880 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 --2- 172.21.15.7:0/25083934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 0x7f759c105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 -- 172.21.15.7:0/25083934 >> 172.21.15.7:0/25083934 conn(0x7f759c077230 msgr2=0x7f759c077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 -- 172.21.15.7:0/25083934 shutdown_connections 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.871+0000 7f75a25cd700 1 -- 172.21.15.7:0/25083934 wait complete. 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 Processor -- start 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 -- start start 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c1014a0 0x7f759c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 0x7f759c103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c106530 0x7f759c103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1113d0 con 0x7f759c106530 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f759c111250 con 0x7f759c101e50 2024-03-19T00:37:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a25cd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f759c111550 con 0x7f759c1014a0 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.872+0000 7f75a0b6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c106530 0x7f759c103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f759b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 0x7f759c103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c106530 0x7f759c103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35252/0 (socket says 172.21.15.7:35252) 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f759b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 0x7f759c103430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43956/0 (socket says 172.21.15.7:43956) 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 -- 172.21.15.7:0/1595047122 learned_addr learned my addr 172.21.15.7:0/1595047122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f759bfff700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c1014a0 0x7f759c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 -- 172.21.15.7:0/1595047122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c1014a0 msgr2=0x7f759c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c1014a0 0x7f759c102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 -- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 msgr2=0x7f759c103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 0x7f759c103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 -- 172.21.15.7:0/1595047122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c1a6f40 con 0x7f759c106530 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f759b7fe700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c101e50 0x7f759c103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f759bfff700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f759c1014a0 0x7f759c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a0b6a700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c106530 0x7f759c103970 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f759400ef30 tx=0x7f759400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75940092e0 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c1a71d0 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7594004bb0 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594004020 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.873+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1a7710 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.874+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f759c0629e0 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.875+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f759401b030 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f75997fa700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f75840a2920 0x7f75840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f759bfff700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f75840a2920 0x7f75840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f75940cf340 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f75997fa700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] conn(0x7f75840ada80 0x7f75840aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 --> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f759401fe50 con 0x7f75840ada80 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f75940cf6f0 con 0x7f759c106530 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f759b7fe700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] conn(0x7f75840ada80 0x7f75840aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.876+0000 7f759bfff700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f75840a2920 0x7f75840a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f758c00b2e0 tx=0x7f758c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.877+0000 7f759b7fe700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] conn(0x7f75840ada80 0x7f75840aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.878+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== osd.1 v2:172.21.15.44:6800/274970163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f759401fe50 con 0x7f75840ada80 2024-03-19T00:37:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.894+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 --> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f759c062e60 con 0x7f75840ada80 2024-03-19T00:37:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.894+0000 7f75997fa700 1 -- 172.21.15.7:0/1595047122 <== osd.1 v2:172.21.15.44:6800/274970163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f759c062e60 con 0x7f75840ada80 2024-03-19T00:37:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.894+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] conn(0x7f75840ada80 msgr2=0x7f75840aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.894+0000 7f75a25cd700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] conn(0x7f75840ada80 0x7f75840aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f75840a2920 msgr2=0x7f75840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f75840a2920 0x7f75840a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f758c00b2e0 tx=0x7f758c01a040 comp rx=0 tx=0).stop 2024-03-19T00:37:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c106530 msgr2=0x7f759c103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 --2- 172.21.15.7:0/1595047122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f759c106530 0x7f759c103970 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f759400ef30 tx=0x7f759400c3a0 comp rx=0 tx=0).stop 2024-03-19T00:37:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a0b6a700 1 -- 172.21.15.7:0/1595047122 reap_dead start 2024-03-19T00:37:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 shutdown_connections 2024-03-19T00:37:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 >> 172.21.15.7:0/1595047122 conn(0x7f759c077230 msgr2=0x7f759c06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 shutdown_connections 2024-03-19T00:37:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.895+0000 7f75a25cd700 1 -- 172.21.15.7:0/1595047122 wait complete. 2024-03-19T00:37:08.590 INFO:teuthology.orchestra.run.smithi007.stdout:163208757304 2024-03-19T00:37:08.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-19T00:37:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2597847818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54100900 msgr2=0x7fbd54100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2597847818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54100900 0x7fbd54100d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009990 tx=0x7fbd44009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2597847818 shutdown_connections 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2597847818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 0x7fbd5410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2597847818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54107380 0x7fbd54107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2597847818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54100900 0x7fbd54100d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2597847818 >> 172.21.15.7:0/2597847818 conn(0x7fbd54077810 msgr2=0x7fbd540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2597847818 shutdown_connections 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2597847818 wait complete. 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 Processor -- start 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 -- start start 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.290+0000 7fbd5b74a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54100900 0x7fbd5419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 0x7fbd5419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54107380 0x7fbd541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541125f0 con 0x7fbd54100900 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54112470 con 0x7fbd54107380 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54112770 con 0x7fbd541012c0 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd594e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54100900 0x7fbd5419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 0x7fbd5419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd594e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54100900 0x7fbd5419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35242/0 (socket says 172.21.15.7:35242) 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 0x7fbd5419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:46588/0 (socket says 172.21.15.7:46588) 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd59ce7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54107380 0x7fbd541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd594e6700 1 -- 172.21.15.7:0/2575851258 learned_addr learned my addr 172.21.15.7:0/2575851258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 -- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54107380 msgr2=0x7fbd541988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54107380 0x7fbd541988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 -- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54100900 msgr2=0x7fbd5419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54100900 0x7fbd5419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 -- 172.21.15.7:0/2575851258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd54199050 con 0x7fbd541012c0 2024-03-19T00:37:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd594e6700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd54100900 0x7fbd5419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd59ce7700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd54107380 0x7fbd541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd58ce5700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 0x7fbd5419ec70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd44006010 tx=0x7fbd440057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44027070 con 0x7fbd541012c0 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd44022470 con 0x7fbd541012c0 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd541992e0 con 0x7fbd541012c0 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400b610 con 0x7fbd541012c0 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.291+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd541a5600 con 0x7fbd541012c0 2024-03-19T00:37:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.292+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd540629e0 con 0x7fbd541012c0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.293+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fbd4402a070 con 0x7fbd541012c0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd4a7fc700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd400a2920 0x7fbd400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd594e6700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd400a2920 0x7fbd400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd440cffe0 con 0x7fbd541012c0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd4a7fc700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] conn(0x7fbd400ad950 0x7fbd400afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 --> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd400b04c0 con 0x7fbd400ad950 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fbd440d0430 con 0x7fbd541012c0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd59ce7700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] conn(0x7fbd400ad950 0x7fbd400afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.294+0000 7fbd594e6700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd400a2920 0x7fbd400a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd54199a30 tx=0x7fbd50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.295+0000 7fbd59ce7700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] conn(0x7fbd400ad950 0x7fbd400afdf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.296+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== osd.7 v2:172.21.15.44:6816/48320224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd400b04c0 con 0x7fbd400ad950 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.319+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 --> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd54062de0 con 0x7fbd400ad950 2024-03-19T00:37:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.319+0000 7fbd4a7fc700 1 -- 172.21.15.7:0/2575851258 <== osd.7 v2:172.21.15.44:6816/48320224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbd54062de0 con 0x7fbd400ad950 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.319+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] conn(0x7fbd400ad950 msgr2=0x7fbd400afdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.319+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] conn(0x7fbd400ad950 0x7fbd400afdf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.319+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd400a2920 msgr2=0x7fbd400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.319+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd400a2920 0x7fbd400a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd54199a30 tx=0x7fbd50009260 comp rx=0 tx=0).stop 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 msgr2=0x7fbd5419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd5b74a700 1 --2- 172.21.15.7:0/2575851258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd541012c0 0x7fbd5419ec70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd44006010 tx=0x7fbd440057a0 comp rx=0 tx=0).stop 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd59ce7700 1 -- 172.21.15.7:0/2575851258 reap_dead start 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 shutdown_connections 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 >> 172.21.15.7:0/2575851258 conn(0x7fbd54077810 msgr2=0x7fbd54105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 shutdown_connections 2024-03-19T00:37:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:07.320+0000 7fbd5b74a700 1 -- 172.21.15.7:0/2575851258 wait complete. 2024-03-19T00:37:09.455 INFO:teuthology.orchestra.run.smithi007.stdout:219043332143 2024-03-19T00:37:09.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-19T00:37:09.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:09 smithi007 ceph-mon[62854]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 203 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:09 smithi204 ceph-mon[66572]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 203 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:09 smithi044 ceph-mon[66366]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 203 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:09.885 INFO:teuthology.orchestra.run.smithi007.stdout:55834574926 2024-03-19T00:37:09.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-19T00:37:10.006 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:10.010 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:10.472 INFO:teuthology.orchestra.run.smithi007.stdout:309237645344 2024-03-19T00:37:10.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-19T00:37:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:11.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:11 smithi007 ceph-mon[62854]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:11 smithi204 ceph-mon[66572]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:11.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:11 smithi044 ceph-mon[66366]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T00:37:13.116 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:13.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:13 smithi204 ceph-mon[66572]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-03-19T00:37:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:13 smithi044 ceph-mon[66366]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-03-19T00:37:13.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:13 smithi007 ceph-mon[62854]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 -- 172.21.15.7:0/1067263588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 msgr2=0x7f69e0107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1067263588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 0x7f69e0107630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69d00098f0 tx=0x7f69d0009c00 comp rx=0 tx=0).stop 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 -- 172.21.15.7:0/1067263588 shutdown_connections 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1067263588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101bb0 0x7f69e010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1067263588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 0x7f69e0107630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1067263588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101210 0x7f69e0101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 -- 172.21.15.7:0/1067263588 >> 172.21.15.7:0/1067263588 conn(0x7f69e0078140 msgr2=0x7f69e0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 -- 172.21.15.7:0/1067263588 shutdown_connections 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 -- 172.21.15.7:0/1067263588 wait complete. 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 Processor -- start 2024-03-19T00:37:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.883+0000 7f69e66fe700 1 -- start start 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e66fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101210 0x7f69e0103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e66fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101bb0 0x7f69e0103850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e66fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 0x7f69e0103d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e66fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0111360 con 0x7f69e0101bb0 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e66fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01111e0 con 0x7f69e0107250 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e66fe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01114e0 con 0x7f69e0101210 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101bb0 0x7f69e0103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101210 0x7f69e0103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101bb0 0x7f69e0103850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58656/0 (socket says 172.21.15.7:58656) 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69df7fe700 1 -- 172.21.15.7:0/1955537779 learned_addr learned my addr 172.21.15.7:0/1955537779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69e4c9b700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 0x7f69e0103d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 -- 172.21.15.7:0/1955537779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 msgr2=0x7f69e0103d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 0x7f69e0103d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 -- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101bb0 msgr2=0x7f69e0103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101bb0 0x7f69e0103850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 -- 172.21.15.7:0/1955537779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e01a6ee0 con 0x7f69e0101210 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.884+0000 7f69dffff700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101210 0x7f69e0103310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009230 tx=0x7f69d000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.885+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0005040 con 0x7f69e0101210 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.885+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69d0005690 con 0x7f69e0101210 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.885+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e01a7170 con 0x7f69e0101210 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.885+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00038e0 con 0x7f69e0101210 2024-03-19T00:37:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.885+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e01a75c0 con 0x7f69e0101210 2024-03-19T00:37:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.886+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e0068fd0 con 0x7f69e0101210 2024-03-19T00:37:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.887+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f69d000b7d0 con 0x7f69e0101210 2024-03-19T00:37:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.887+0000 7f69dd7fa700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2920 0x7f69c80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.887+0000 7f69df7fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2920 0x7f69c80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.887+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f69d00cfed0 con 0x7f69e0101210 2024-03-19T00:37:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.888+0000 7f69df7fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2920 0x7f69c80a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f69e006b0a0 tx=0x7f69d4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:14.891+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f69d0013030 con 0x7f69e0101210 2024-03-19T00:37:15.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.101+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f69e01a7960 con 0x7f69e0101210 2024-03-19T00:37:15.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.101+0000 7f69dd7fa700 1 -- 172.21.15.7:0/1955537779 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f69d009cb40 con 0x7f69e0101210 2024-03-19T00:37:15.103 INFO:teuthology.orchestra.run.smithi007.stdout:343597383708 2024-03-19T00:37:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2920 msgr2=0x7f69c80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2920 0x7f69c80a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f69e006b0a0 tx=0x7f69d4009310 comp rx=0 tx=0).stop 2024-03-19T00:37:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101210 msgr2=0x7f69e0103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101210 0x7f69e0103310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009230 tx=0x7f69d000fa00 comp rx=0 tx=0).stop 2024-03-19T00:37:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 shutdown_connections 2024-03-19T00:37:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f69c80a2920 0x7f69c80a4de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e0107250 0x7f69e0103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69e0101210 0x7f69e0103310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 --2- 172.21.15.7:0/1955537779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0101bb0 0x7f69e0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 >> 172.21.15.7:0/1955537779 conn(0x7f69e0078140 msgr2=0x7f69e01056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 shutdown_connections 2024-03-19T00:37:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.103+0000 7f69e66fe700 1 -- 172.21.15.7:0/1955537779 wait complete. 2024-03-19T00:37:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 -- 172.21.15.7:0/4014812052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102d50 msgr2=0x7fbebc10f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 --2- 172.21.15.7:0/4014812052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102d50 0x7fbebc10f1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b0a0 tx=0x7fbeb400b3b0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 -- 172.21.15.7:0/4014812052 shutdown_connections 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 --2- 172.21.15.7:0/4014812052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102d50 0x7fbebc10f1b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 --2- 172.21.15.7:0/4014812052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc108470 0x7fbebc108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 --2- 172.21.15.7:0/4014812052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc102390 0x7fbebc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 -- 172.21.15.7:0/4014812052 >> 172.21.15.7:0/4014812052 conn(0x7fbebc0793d0 msgr2=0x7fbebc0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 -- 172.21.15.7:0/4014812052 shutdown_connections 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.532+0000 7fbec44c5700 1 -- 172.21.15.7:0/4014812052 wait complete. 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 Processor -- start 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 -- start start 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 0x7fbebc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc102d50 0x7fbebc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 0x7fbebc1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc112480 con 0x7fbebc108470 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc112300 con 0x7fbebc102390 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec44c5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc112600 con 0x7fbebc102d50 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec2a62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 0x7fbebc1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec2261700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 0x7fbebc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec1a60700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc102d50 0x7fbebc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec2a62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 0x7fbebc1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58686/0 (socket says 172.21.15.7:58686) 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec2a62700 1 -- 172.21.15.7:0/1164050669 learned_addr learned my addr 172.21.15.7:0/1164050669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.533+0000 7fbec2261700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 0x7fbebc19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44406/0 (socket says 172.21.15.7:44406) 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2261700 1 -- 172.21.15.7:0/1164050669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc102d50 msgr2=0x7fbebc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2261700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc102d50 0x7fbebc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2261700 1 -- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 msgr2=0x7fbebc1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2261700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 0x7fbebc1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2261700 1 -- 172.21.15.7:0/1164050669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc198fa0 con 0x7fbebc102390 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2a62700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 0x7fbebc1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec1a60700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc102d50 0x7fbebc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec2261700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 0x7fbebc19e640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800e9f0 tx=0x7fbeb800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800c9c0 con 0x7fbebc102390 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc199290 con 0x7fbebc102390 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeb800cb60 con 0x7fbebc102390 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8018680 con 0x7fbebc102390 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.534+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc1a54a0 con 0x7fbebc102390 2024-03-19T00:37:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.535+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbebc103a50 con 0x7fbebc102390 2024-03-19T00:37:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.536+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fbeb8010430 con 0x7fbebc102390 2024-03-19T00:37:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.537+0000 7fbeb37fe700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbea80a2920 0x7fbea80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.537+0000 7fbec1a60700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbea80a2920 0x7fbea80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.537+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbeb80cefb0 con 0x7fbebc102390 2024-03-19T00:37:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.537+0000 7fbec1a60700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbea80a2920 0x7fbea80a4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbebc079960 tx=0x7fbeac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.541+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbeb809bb90 con 0x7fbebc102390 2024-03-19T00:37:15.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:15 smithi007 ceph-mon[62854]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-19T00:37:15.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1955537779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T00:37:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.750+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fbebc02dbb0 con 0x7fbebc102390 2024-03-19T00:37:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.750+0000 7fbeb37fe700 1 -- 172.21.15.7:0/1164050669 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbeb809b490 con 0x7fbebc102390 2024-03-19T00:37:15.753 INFO:teuthology.orchestra.run.smithi007.stdout:287762808871 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbea80a2920 msgr2=0x7fbea80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbea80a2920 0x7fbea80a4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbebc079960 tx=0x7fbeac009260 comp rx=0 tx=0).stop 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 msgr2=0x7fbebc19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 0x7fbebc19e640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800e9f0 tx=0x7fbeb800ed00 comp rx=0 tx=0).stop 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 shutdown_connections 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbea80a2920 0x7fbea80a4de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc108470 0x7fbebc1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc102390 0x7fbebc19e640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 --2- 172.21.15.7:0/1164050669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbebc102d50 0x7fbebc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 >> 172.21.15.7:0/1164050669 conn(0x7fbebc0793d0 msgr2=0x7fbebc100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 shutdown_connections 2024-03-19T00:37:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:15.753+0000 7fbec44c5700 1 -- 172.21.15.7:0/1164050669 wait complete. 2024-03-19T00:37:15.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:15 smithi204 ceph-mon[66572]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-19T00:37:15.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1955537779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T00:37:15.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:15 smithi044 ceph-mon[66366]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-19T00:37:15.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1955537779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T00:37:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1164050669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T00:37:16.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1164050669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T00:37:16.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1164050669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T00:37:17.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:17 smithi204 ceph-mon[66572]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:17.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:17 smithi007 ceph-mon[62854]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:17.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:17 smithi044 ceph-mon[66366]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:18.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383708 for osd.11 2024-03-19T00:37:18.516 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 -- 172.21.15.7:0/923996284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 msgr2=0x7fa3a4106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 --2- 172.21.15.7:0/923996284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4106f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa394009990 tx=0x7fa394009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 -- 172.21.15.7:0/923996284 shutdown_connections 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 --2- 172.21.15.7:0/923996284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3a4107620 0x7fa3a410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 --2- 172.21.15.7:0/923996284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a406aab0 0x7fa3a406ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 --2- 172.21.15.7:0/923996284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4106f40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 -- 172.21.15.7:0/923996284 >> 172.21.15.7:0/923996284 conn(0x7fa3a4077260 msgr2=0x7fa3a4077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 -- 172.21.15.7:0/923996284 shutdown_connections 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 -- 172.21.15.7:0/923996284 wait complete. 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 Processor -- start 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.176+0000 7fa3ab724700 1 -- start start 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3a406aab0 0x7fa3a4101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a4107620 0x7fa3a4101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112470 con 0x7fa3a406b460 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41122f0 con 0x7fa3a406aab0 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41125f0 con 0x7fa3a4107620 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58700/0 (socket says 172.21.15.7:58700) 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 -- 172.21.15.7:0/3317333082 learned_addr learned my addr 172.21.15.7:0/3317333082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a9cc1700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a4107620 0x7fa3a4101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a94c0700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3a406aab0 0x7fa3a4101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 -- 172.21.15.7:0/3317333082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a4107620 msgr2=0x7fa3a4101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a4107620 0x7fa3a4101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 -- 172.21.15.7:0/3317333082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3a406aab0 msgr2=0x7fa3a4101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3a406aab0 0x7fa3a4101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 -- 172.21.15.7:0/3317333082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a41a6e90 con 0x7fa3a406b460 2024-03-19T00:37:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3a8cbf700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4101850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa394000c00 tx=0x7fa394004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394027070 con 0x7fa3a406b460 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa394022470 con 0x7fa3a406b460 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a41a7120 con 0x7fa3a406b460 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394017650 con 0x7fa3a406b460 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.177+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41a7570 con 0x7fa3a406b460 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.178+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a4068fd0 con 0x7fa3a406b460 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stdout:34359738452 2024-03-19T00:37:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.179+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fa39402a070 con 0x7fa3a406b460 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.180+0000 7fa39a7fc700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3900a2920 0x7fa3900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.180+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa3940d0160 con 0x7fa3a406b460 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.180+0000 7fa3a94c0700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3900a2920 0x7fa3900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.180+0000 7fa3a94c0700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3900a2920 0x7fa3900a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009c60 tx=0x7fa3a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.184+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa394031070 con 0x7fa3a406b460 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.390+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa3a402dd70 con 0x7fa3a406b460 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.390+0000 7fa39a7fc700 1 -- 172.21.15.7:0/3317333082 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa394013070 con 0x7fa3a406b460 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3900a2920 msgr2=0x7fa3900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3900a2920 0x7fa3900a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009c60 tx=0x7fa3a0009260 comp rx=0 tx=0).stop 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 msgr2=0x7fa3a4101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4101850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa394000c00 tx=0x7fa394004790 comp rx=0 tx=0).stop 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 shutdown_connections 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3900a2920 0x7fa3900a4de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3a4107620 0x7fa3a4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3a406aab0 0x7fa3a4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 --2- 172.21.15.7:0/3317333082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a406b460 0x7fa3a4101850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 >> 172.21.15.7:0/3317333082 conn(0x7fa3a4077260 msgr2=0x7fa3a40ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 shutdown_connections 2024-03-19T00:37:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:18.392+0000 7fa3ab724700 1 -- 172.21.15.7:0/3317333082 wait complete. 2024-03-19T00:37:18.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3317333082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T00:37:18.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:18 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3317333082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T00:37:18.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:18 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3317333082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T00:37:19.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808868 got 287762808871 for osd.9 2024-03-19T00:37:19.273 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:19 smithi204 ceph-mon[66572]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:19.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:19 smithi044 ceph-mon[66366]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:19.898 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:19 smithi007 ceph-mon[62854]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:19.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738452 for osd.0 2024-03-19T00:37:19.926 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:19.930 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:20.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 -- 172.21.15.7:0/1594967110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101210 msgr2=0x7fb0dc101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/1594967110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101210 0x7fb0dc101670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009990 tx=0x7fb0cc009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 -- 172.21.15.7:0/1594967110 shutdown_connections 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/1594967110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/1594967110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 0x7fb0dc107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/1594967110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101210 0x7fb0dc101670 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 -- 172.21.15.7:0/1594967110 >> 172.21.15.7:0/1594967110 conn(0x7fb0dc078140 msgr2=0x7fb0dc078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 -- 172.21.15.7:0/1594967110 shutdown_connections 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.128+0000 7fb0e0d80700 1 -- 172.21.15.7:0/1594967110 wait complete. 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 Processor -- start 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 -- start start 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101210 0x7fb0dc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 0x7fb0dc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc111380 con 0x7fb0dc101bb0 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc111200 con 0x7fb0dc107250 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0e0d80700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc111500 con 0x7fb0dc101210 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58730/0 (socket says 172.21.15.7:58730) 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0d9d9b700 1 -- 172.21.15.7:0/202361504 learned_addr learned my addr 172.21.15.7:0/202361504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0da59c700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101210 0x7fb0dc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.129+0000 7fb0dad9d700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 0x7fb0dc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0d9d9b700 1 -- 172.21.15.7:0/202361504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101210 msgr2=0x7fb0dc19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0d9d9b700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101210 0x7fb0dc19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0d9d9b700 1 -- 172.21.15.7:0/202361504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 msgr2=0x7fb0dc198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0d9d9b700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 0x7fb0dc198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0d9d9b700 1 -- 172.21.15.7:0/202361504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc199010 con 0x7fb0dc101bb0 2024-03-19T00:37:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0da59c700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101210 0x7fb0dc19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0dad9d700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 0x7fb0dc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0d9d9b700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc19eba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc027070 con 0x7fb0dc101bb0 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0cc022470 con 0x7fb0dc101bb0 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc1992a0 con 0x7fb0dc101bb0 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc00b610 con 0x7fb0dc101bb0 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.130+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc1a54c0 con 0x7fb0dc101bb0 2024-03-19T00:37:20.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.131+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0dc102900 con 0x7fb0dc101bb0 2024-03-19T00:37:20.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.134+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fb0cc02a070 con 0x7fb0dc101bb0 2024-03-19T00:37:20.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.135+0000 7fb0cb7fe700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb0c40a2920 0x7fb0c40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.135+0000 7fb0da59c700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb0c40a2920 0x7fb0c40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.135+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb0cc0d00a0 con 0x7fb0dc101bb0 2024-03-19T00:37:20.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.135+0000 7fb0da59c700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb0c40a2920 0x7fb0c40a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009680 tx=0x7fb0d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:20.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.138+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb0cc013070 con 0x7fb0dc101bb0 2024-03-19T00:37:20.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.346+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb0dc068fd0 con 0x7fb0dc101bb0 2024-03-19T00:37:20.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.346+0000 7fb0cb7fe700 1 -- 172.21.15.7:0/202361504 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0dc068fd0 con 0x7fb0dc101bb0 2024-03-19T00:37:20.347 INFO:teuthology.orchestra.run.smithi007.stdout:219043332145 2024-03-19T00:37:20.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb0c40a2920 msgr2=0x7fb0c40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb0c40a2920 0x7fb0c40a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009680 tx=0x7fb0d0009260 comp rx=0 tx=0).stop 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 msgr2=0x7fb0dc19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc19eba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc004790 comp rx=0 tx=0).stop 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 shutdown_connections 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb0c40a2920 0x7fb0c40a4de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0dc107250 0x7fb0dc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0dc101210 0x7fb0dc19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 --2- 172.21.15.7:0/202361504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc101bb0 0x7fb0dc19eba0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 >> 172.21.15.7:0/202361504 conn(0x7fb0dc078140 msgr2=0x7fb0dc06a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:20.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 shutdown_connections 2024-03-19T00:37:20.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.349+0000 7fb0e0d80700 1 -- 172.21.15.7:0/202361504 wait complete. 2024-03-19T00:37:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2858805796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101210 msgr2=0x7f7c68101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2858805796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101210 0x7f7c68101670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009990 tx=0x7f7c58009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2858805796 shutdown_connections 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2858805796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101bb0 0x7f7c6810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2858805796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c68107250 0x7f7c68107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2858805796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101210 0x7f7c68101670 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2858805796 >> 172.21.15.7:0/2858805796 conn(0x7f7c68078140 msgr2=0x7f7c68078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2858805796 shutdown_connections 2024-03-19T00:37:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.544+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2858805796 wait complete. 2024-03-19T00:37:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 Processor -- start 2024-03-19T00:37:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 -- start start 2024-03-19T00:37:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101210 0x7f7c6819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 0x7f7c6819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c68107250 0x7f7c68198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111380 con 0x7f7c68101bb0 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111200 con 0x7f7c68107250 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.545+0000 7f7c6fd99700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111500 con 0x7f7c68101210 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6d334700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 0x7f7c6819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101210 0x7f7c6819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6d334700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 0x7f7c6819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58748/0 (socket says 172.21.15.7:58748) 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6d334700 1 -- 172.21.15.7:0/2819200364 learned_addr learned my addr 172.21.15.7:0/2819200364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6e336700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c68107250 0x7f7c68198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 -- 172.21.15.7:0/2819200364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c68107250 msgr2=0x7f7c68198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c68107250 0x7f7c68198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 -- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 msgr2=0x7f7c6819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 0x7f7c6819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 -- 172.21.15.7:0/2819200364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c68199010 con 0x7f7c68101210 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6d334700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 0x7f7c6819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6db35700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101210 0x7f7c6819e660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9f0 tx=0x7f7c6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400cb70 con 0x7f7c68101210 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c68199300 con 0x7f7c68101210 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c6400ee90 con 0x7f7c68101210 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003d40 con 0x7f7c68101210 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.546+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681a54c0 con 0x7f7c68101210 2024-03-19T00:37:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.547+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c68102900 con 0x7f7c68101210 2024-03-19T00:37:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.549+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f7c64044940 con 0x7f7c68101210 2024-03-19T00:37:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.550+0000 7f7c5effd700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7c540a2920 0x7f7c540a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.550+0000 7f7c6d334700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7c540a2920 0x7f7c540a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.550+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c640ceff0 con 0x7f7c68101210 2024-03-19T00:37:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.550+0000 7f7c6d334700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7c540a2920 0x7f7c540a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c68078740 tx=0x7f7c5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.553+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7c6409bbb0 con 0x7f7c68101210 2024-03-19T00:37:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.759+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7c68068fd0 con 0x7f7c68101210 2024-03-19T00:37:20.761 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:20 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/202361504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T00:37:20.761 INFO:teuthology.orchestra.run.smithi007.stdout:163208757308 2024-03-19T00:37:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.759+0000 7f7c5effd700 1 -- 172.21.15.7:0/2819200364 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c64009420 con 0x7f7c68101210 2024-03-19T00:37:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7c540a2920 msgr2=0x7f7c540a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7c540a2920 0x7f7c540a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c68078740 tx=0x7f7c5801a040 comp rx=0 tx=0).stop 2024-03-19T00:37:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101210 msgr2=0x7f7c6819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101210 0x7f7c6819e660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9f0 tx=0x7f7c6400ed00 comp rx=0 tx=0).stop 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 shutdown_connections 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7c540a2920 0x7f7c540a4de0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c68107250 0x7f7c68198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c68101210 0x7f7c6819e660 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9f0 tx=0x7f7c6400ed00 comp rx=0 tx=0).stop 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 --2- 172.21.15.7:0/2819200364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68101bb0 0x7f7c6819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 >> 172.21.15.7:0/2819200364 conn(0x7f7c68078140 msgr2=0x7f7c6806a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 shutdown_connections 2024-03-19T00:37:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:20.762+0000 7f7c6fd99700 1 -- 172.21.15.7:0/2819200364 wait complete. 2024-03-19T00:37:20.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:20 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/202361504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T00:37:20.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:20 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/202361504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T00:37:21.697 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332143 got 219043332145 for osd.7 2024-03-19T00:37:21.698 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:21 smithi204 ceph-mon[66572]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-19T00:37:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:21 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2819200364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T00:37:21.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:21 smithi007 ceph-mon[62854]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-19T00:37:21.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:21 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2819200364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T00:37:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:21 smithi044 ceph-mon[66366]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-19T00:37:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:21 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2819200364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T00:37:23.526 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:23 smithi007 ceph-mon[62854]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:23 smithi204 ceph-mon[66572]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:23.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:23 smithi044 ceph-mon[66366]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 -- 172.21.15.7:0/1579282912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 msgr2=0x7f435010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 --2- 172.21.15.7:0/1579282912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435010e050 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f434800b0a0 tx=0x7f434800b3b0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 -- 172.21.15.7:0/1579282912 shutdown_connections 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 --2- 172.21.15.7:0/1579282912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435010e050 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 --2- 172.21.15.7:0/1579282912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 0x7f4350107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 --2- 172.21.15.7:0/1579282912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 0x7f4350101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 -- 172.21.15.7:0/1579282912 >> 172.21.15.7:0/1579282912 conn(0x7f4350078220 msgr2=0x7f4350078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 -- 172.21.15.7:0/1579282912 shutdown_connections 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.668+0000 7f4357e12700 1 -- 172.21.15.7:0/1579282912 wait complete. 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 Processor -- start 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 -- start start 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 0x7f435019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 0x7f4350198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4350111360 con 0x7f4350101bb0 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43501111e0 con 0x7f43501072a0 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4357e12700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43501114e0 con 0x7f43501011f0 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f43553ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f43553ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33384/0 (socket says 172.21.15.7:33384) 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f43553ad700 1 -- 172.21.15.7:0/102348351 learned_addr learned my addr 172.21.15.7:0/102348351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f4355bae700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 0x7f435019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:25.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.669+0000 7f43563af700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 0x7f4350198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43553ad700 1 -- 172.21.15.7:0/102348351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 msgr2=0x7f435019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43553ad700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 0x7f435019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43553ad700 1 -- 172.21.15.7:0/102348351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 msgr2=0x7f4350198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43553ad700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 0x7f4350198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43553ad700 1 -- 172.21.15.7:0/102348351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4350198ed0 con 0x7f4350101bb0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f4355bae700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 0x7f435019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43563af700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 0x7f4350198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f43553ad700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435019eb10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9a0 tx=0x7f434000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434000ca10 con 0x7f4350101bb0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434000cbb0 con 0x7f4350101bb0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43501991c0 con 0x7f4350101bb0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340018680 con 0x7f4350101bb0 2024-03-19T00:37:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.670+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501a53f0 con 0x7f4350101bb0 2024-03-19T00:37:25.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.671+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4350068fd0 con 0x7f4350101bb0 2024-03-19T00:37:25.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.672+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f4340010430 con 0x7f4350101bb0 2024-03-19T00:37:25.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.673+0000 7f4346ffd700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f433c0a2920 0x7f433c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:25.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.673+0000 7f4355bae700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f433c0a2920 0x7f433c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:25.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.673+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f43400cf2f0 con 0x7f4350101bb0 2024-03-19T00:37:25.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.673+0000 7f4355bae700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f433c0a2920 0x7f433c0a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f434c009fd0 tx=0x7f434c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.677+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f434009bed0 con 0x7f4350101bb0 2024-03-19T00:37:25.686 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:25 smithi204 ceph-mon[66572]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:25.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:25 smithi044 ceph-mon[66366]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.894+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f43501a5620 con 0x7f4350101bb0 2024-03-19T00:37:25.895 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:25 smithi007 ceph-mon[62854]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:25.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.894+0000 7f4346ffd700 1 -- 172.21.15.7:0/102348351 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f434009b7d0 con 0x7f4350101bb0 2024-03-19T00:37:25.895 INFO:teuthology.orchestra.run.smithi007.stdout:77309411404 2024-03-19T00:37:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f433c0a2920 msgr2=0x7f433c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f433c0a2920 0x7f433c0a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f434c009fd0 tx=0x7f434c009260 comp rx=0 tx=0).stop 2024-03-19T00:37:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 msgr2=0x7f435019eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435019eb10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9a0 tx=0x7f434000ecb0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 shutdown_connections 2024-03-19T00:37:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f433c0a2920 0x7f433c0a4de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43501072a0 0x7f4350198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43501011f0 0x7f435019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 --2- 172.21.15.7:0/102348351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350101bb0 0x7f435019eb10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 >> 172.21.15.7:0/102348351 conn(0x7f4350078220 msgr2=0x7f435006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 shutdown_connections 2024-03-19T00:37:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:25.896+0000 7f4357e12700 1 -- 172.21.15.7:0/102348351 wait complete. 2024-03-19T00:37:26.150 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757308 for osd.5 2024-03-19T00:37:26.150 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:26.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/102348351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T00:37:26.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/102348351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T00:37:27.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/102348351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T00:37:27.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:27 smithi007 ceph-mon[62854]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.586+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/1043639722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 msgr2=0x7fbd44101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.586+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/1043639722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd44101880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098f0 tx=0x7fbd40009c00 comp rx=0 tx=0).stop 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.586+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/1043639722 shutdown_connections 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.586+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/1043639722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd441064f0 0x7fbd4410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.586+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/1043639722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd44101880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.586+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/1043639722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd44105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/1043639722 >> 172.21.15.7:0/1043639722 conn(0x7fbd44077260 msgr2=0x7fbd44077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/1043639722 shutdown_connections 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/1043639722 wait complete. 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 Processor -- start 2024-03-19T00:37:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- start start 2024-03-19T00:37:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd4419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd4419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd441064f0 0x7fbd441977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44111380 con 0x7fbd44101e50 2024-03-19T00:37:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44111200 con 0x7fbd441014a0 2024-03-19T00:37:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.587+0000 7fbd4c2fe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44111500 con 0x7fbd441064f0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd49899700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd4419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd4419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd49899700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd4419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33402/0 (socket says 172.21.15.7:33402) 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd49899700 1 -- 172.21.15.7:0/232159419 learned_addr learned my addr 172.21.15.7:0/232159419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a89b700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd441064f0 0x7fbd441977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 -- 172.21.15.7:0/232159419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd441064f0 msgr2=0x7fbd441977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd441064f0 0x7fbd441977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 -- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 msgr2=0x7fbd4419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd4419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 -- 172.21.15.7:0/232159419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd44198080 con 0x7fbd441014a0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd49899700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd4419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4a09a700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd4419d680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd4000fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40005040 con 0x7fbd441014a0 2024-03-19T00:37:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd40005690 con 0x7fbd441014a0 2024-03-19T00:37:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd44198310 con 0x7fbd441014a0 2024-03-19T00:37:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400038e0 con 0x7fbd441014a0 2024-03-19T00:37:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.588+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd441a4430 con 0x7fbd441014a0 2024-03-19T00:37:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.589+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd441028f0 con 0x7fbd441014a0 2024-03-19T00:37:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.590+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fbd4000b7d0 con 0x7fbd441014a0 2024-03-19T00:37:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.591+0000 7fbd3b7fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd300a2920 0x7fbd300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.591+0000 7fbd49899700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd300a2920 0x7fbd300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.591+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd400cf830 con 0x7fbd441014a0 2024-03-19T00:37:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.591+0000 7fbd49899700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd300a2920 0x7fbd300a4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009750 tx=0x7fbd34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.600+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd40013030 con 0x7fbd441014a0 2024-03-19T00:37:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.806+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbd44109100 con 0x7fbd441014a0 2024-03-19T00:37:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.807+0000 7fbd3b7fe700 1 -- 172.21.15.7:0/232159419 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd4009c4a0 con 0x7fbd441014a0 2024-03-19T00:37:27.809 INFO:teuthology.orchestra.run.smithi007.stdout:197568495673 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd300a2920 msgr2=0x7fbd300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd300a2920 0x7fbd300a4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009750 tx=0x7fbd34009310 comp rx=0 tx=0).stop 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 msgr2=0x7fbd4419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd4419d680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000c00 tx=0x7fbd4000fa20 comp rx=0 tx=0).stop 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 shutdown_connections 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd300a2920 0x7fbd300a4de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd441064f0 0x7fbd441977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd441014a0 0x7fbd4419d680 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 --2- 172.21.15.7:0/232159419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd44101e50 0x7fbd4419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 >> 172.21.15.7:0/232159419 conn(0x7fbd44077260 msgr2=0x7fbd441051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 shutdown_connections 2024-03-19T00:37:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:27.809+0000 7fbd4c2fe700 1 -- 172.21.15.7:0/232159419 wait complete. 2024-03-19T00:37:27.855 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:27 smithi044 ceph-mon[66366]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:28.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:27 smithi204 ceph-mon[66572]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:28.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:28 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/232159419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T00:37:28.675 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495673 for osd.6 2024-03-19T00:37:28.676 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:28.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:28 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/232159419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T00:37:29.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:28 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/232159419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T00:37:29.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411404 for osd.2 2024-03-19T00:37:29.157 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:29.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:29 smithi044 ceph-mon[66366]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:29.871 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:29 smithi007 ceph-mon[62854]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:29 smithi204 ceph-mon[66572]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:31.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 -- 172.21.15.7:0/4247851659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 msgr2=0x7f0458105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 --2- 172.21.15.7:0/4247851659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f0458105e40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f044c009990 tx=0x7f044c009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 -- 172.21.15.7:0/4247851659 shutdown_connections 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 --2- 172.21.15.7:0/4247851659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458106520 0x7f045810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 --2- 172.21.15.7:0/4247851659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101460 0x7f0458101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 --2- 172.21.15.7:0/4247851659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f0458105e40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 -- 172.21.15.7:0/4247851659 >> 172.21.15.7:0/4247851659 conn(0x7f0458078180 msgr2=0x7f0458078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 -- 172.21.15.7:0/4247851659 shutdown_connections 2024-03-19T00:37:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.025+0000 7f045cee8700 1 -- 172.21.15.7:0/4247851659 wait complete. 2024-03-19T00:37:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 Processor -- start 2024-03-19T00:37:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 -- start start 2024-03-19T00:37:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458101460 0x7f045819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f045819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458106520 0x7f0458197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0458111390 con 0x7f0458101e10 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0458111210 con 0x7f0458101460 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045cee8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0458111510 con 0x7f0458106520 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f0455d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f045819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f0455d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f045819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33418/0 (socket says 172.21.15.7:33418) 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f0455d9b700 1 -- 172.21.15.7:0/2509518152 learned_addr learned my addr 172.21.15.7:0/2509518152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f045659c700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458101460 0x7f045819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.026+0000 7f0456d9d700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458106520 0x7f0458197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f0455d9b700 1 -- 172.21.15.7:0/2509518152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458106520 msgr2=0x7f0458197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f0455d9b700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458106520 0x7f0458197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f0455d9b700 1 -- 172.21.15.7:0/2509518152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458101460 msgr2=0x7f045819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f0455d9b700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458101460 0x7f045819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f0455d9b700 1 -- 172.21.15.7:0/2509518152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0458197ff0 con 0x7f0458101e10 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f045659c700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458101460 0x7f045819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f0455d9b700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f045819db80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f044c000c00 tx=0x7f044c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c027070 con 0x7f0458101e10 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0458198280 con 0x7f0458101e10 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f044c022470 con 0x7f0458101e10 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00b610 con 0x7f0458101e10 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.027+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045806a9b0 con 0x7f0458101e10 2024-03-19T00:37:31.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.028+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045806a260 con 0x7f0458101e10 2024-03-19T00:37:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.029+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f044c02a070 con 0x7f0458101e10 2024-03-19T00:37:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.030+0000 7f04477fe700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f04400a4b30 0x7f04400a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.030+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f044c0cfbb0 con 0x7f0458101e10 2024-03-19T00:37:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.030+0000 7f045659c700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f04400a4b30 0x7f04400a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.030+0000 7f045659c700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f04400a4b30 0x7f04400a6ff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0448003e30 tx=0x7f044800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.035+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f044c013070 con 0x7f0458101e10 2024-03-19T00:37:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.247+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0458198c40 con 0x7f0458101e10 2024-03-19T00:37:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.247+0000 7f04477fe700 1 -- 172.21.15.7:0/2509518152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f044c09c820 con 0x7f0458101e10 2024-03-19T00:37:31.249 INFO:teuthology.orchestra.run.smithi007.stdout:55834574930 2024-03-19T00:37:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f04400a4b30 msgr2=0x7f04400a6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f04400a4b30 0x7f04400a6ff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0448003e30 tx=0x7f044800b040 comp rx=0 tx=0).stop 2024-03-19T00:37:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 msgr2=0x7f045819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f045819db80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f044c000c00 tx=0x7f044c004790 comp rx=0 tx=0).stop 2024-03-19T00:37:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 shutdown_connections 2024-03-19T00:37:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f04400a4b30 0x7f04400a6ff0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458106520 0x7f0458197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0458101460 0x7f045819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 --2- 172.21.15.7:0/2509518152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0458101e10 0x7f045819db80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 >> 172.21.15.7:0/2509518152 conn(0x7f0458078180 msgr2=0x7f04580ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 shutdown_connections 2024-03-19T00:37:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.250+0000 7f045cee8700 1 -- 172.21.15.7:0/2509518152 wait complete. 2024-03-19T00:37:31.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:31 smithi007 ceph-mon[62854]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:31.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:31 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2509518152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T00:37:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 -- 172.21.15.7:0/209673969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78102e20 msgr2=0x7fed78106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 --2- 172.21.15.7:0/209673969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78102e20 0x7fed78106e70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed600099c0 tx=0x7fed60009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 -- 172.21.15.7:0/209673969 shutdown_connections 2024-03-19T00:37:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 --2- 172.21.15.7:0/209673969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78107550 0x7fed7810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 --2- 172.21.15.7:0/209673969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed78102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 --2- 172.21.15.7:0/209673969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78102e20 0x7fed78106e70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 -- 172.21.15.7:0/209673969 >> 172.21.15.7:0/209673969 conn(0x7fed780fdd20 msgr2=0x7fed78100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:31.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 -- 172.21.15.7:0/209673969 shutdown_connections 2024-03-19T00:37:31.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.814+0000 7fed7d200700 1 -- 172.21.15.7:0/209673969 wait complete. 2024-03-19T00:37:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 Processor -- start 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 -- start start 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed7819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78102e20 0x7fed7819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78107550 0x7fed78198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed781123e0 con 0x7fed78107550 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed78112260 con 0x7fed78102470 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.815+0000 7fed7d200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed78112560 con 0x7fed78102e20 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed76d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed7819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78107550 0x7fed78198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78102e20 0x7fed7819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed76d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed7819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35148/0 (socket says 172.21.15.7:35148) 2024-03-19T00:37:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed76d9d700 1 -- 172.21.15.7:0/791400287 learned_addr learned my addr 172.21.15.7:0/791400287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 -- 172.21.15.7:0/791400287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 msgr2=0x7fed7819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed7819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 -- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78107550 msgr2=0x7fed78198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78107550 0x7fed78198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 -- 172.21.15.7:0/791400287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed78199040 con 0x7fed78102e20 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed76d9d700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed7819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7659c700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78102e20 0x7fed7819eb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed600098e0 tx=0x7fed60004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60026070 con 0x7fed78102e20 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed60003c70 con 0x7fed78102e20 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed781992d0 con 0x7fed78102e20 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6000f9c0 con 0x7fed78102e20 2024-03-19T00:37:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.816+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed781a53f0 con 0x7fed78102e20 2024-03-19T00:37:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.817+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78103950 con 0x7fed78102e20 2024-03-19T00:37:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.818+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fed60029070 con 0x7fed78102e20 2024-03-19T00:37:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.819+0000 7fed6ffff700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed640a2920 0x7fed640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.819+0000 7fed76d9d700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed640a2920 0x7fed640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:31.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.819+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fed600cf660 con 0x7fed78102e20 2024-03-19T00:37:31.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.819+0000 7fed76d9d700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed640a2920 0x7fed640a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fed68009c10 tx=0x7fed6800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:31.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:31.822+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fed60013070 con 0x7fed78102e20 2024-03-19T00:37:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:31 smithi044 ceph-mon[66366]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:31 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2509518152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T00:37:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:31 smithi204 ceph-mon[66572]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:31 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2509518152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T00:37:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.031+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fed78199c40 con 0x7fed78102e20 2024-03-19T00:37:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.031+0000 7fed6ffff700 1 -- 172.21.15.7:0/791400287 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed6009c3f0 con 0x7fed78102e20 2024-03-19T00:37:32.032 INFO:teuthology.orchestra.run.smithi007.stdout:107374182473 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed640a2920 msgr2=0x7fed640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed640a2920 0x7fed640a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fed68009c10 tx=0x7fed6800b040 comp rx=0 tx=0).stop 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78102e20 msgr2=0x7fed7819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78102e20 0x7fed7819eb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed600098e0 tx=0x7fed60004270 comp rx=0 tx=0).stop 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 shutdown_connections 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed640a2920 0x7fed640a4de0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78107550 0x7fed78198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed78102470 0x7fed7819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 --2- 172.21.15.7:0/791400287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed78102e20 0x7fed7819eb80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 >> 172.21.15.7:0/791400287 conn(0x7fed780fdd20 msgr2=0x7fed78100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 shutdown_connections 2024-03-19T00:37:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:32.033+0000 7fed7d200700 1 -- 172.21.15.7:0/791400287 wait complete. 2024-03-19T00:37:32.442 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574930 for osd.1 2024-03-19T00:37:32.442 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:32.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:32 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/791400287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T00:37:32.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:32 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/791400287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T00:37:33.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:32 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/791400287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T00:37:33.718 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:33.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:33 smithi007 ceph-mon[62854]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:33.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:33 smithi044 ceph-mon[66366]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:33.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 -- 172.21.15.7:0/1292517836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101bb0 msgr2=0x7f197c10e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 --2- 172.21.15.7:0/1292517836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101bb0 0x7f197c10e090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f197400b0a0 tx=0x7f197400b3b0 comp rx=0 tx=0).stop 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 -- 172.21.15.7:0/1292517836 shutdown_connections 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 --2- 172.21.15.7:0/1292517836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101bb0 0x7f197c10e090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 --2- 172.21.15.7:0/1292517836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 0x7f197c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 --2- 172.21.15.7:0/1292517836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101210 0x7f197c101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 -- 172.21.15.7:0/1292517836 >> 172.21.15.7:0/1292517836 conn(0x7f197c078140 msgr2=0x7f197c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 -- 172.21.15.7:0/1292517836 shutdown_connections 2024-03-19T00:37:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.917+0000 7f1982e77700 1 -- 172.21.15.7:0/1292517836 wait complete. 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 Processor -- start 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 -- start start 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 0x7f197c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101bb0 0x7f197c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 0x7f197c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111380 con 0x7f197c101210 2024-03-19T00:37:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111200 con 0x7f197c107250 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1982e77700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111500 con 0x7f197c101bb0 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1980c13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 0x7f197c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1980c13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 0x7f197c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33462/0 (socket says 172.21.15.7:33462) 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1980c13700 1 -- 172.21.15.7:0/864139858 learned_addr learned my addr 172.21.15.7:0/864139858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f197bfff700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101bb0 0x7f197c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.918+0000 7f1981414700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 0x7f197c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1980c13700 1 -- 172.21.15.7:0/864139858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101bb0 msgr2=0x7f197c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1980c13700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101bb0 0x7f197c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1980c13700 1 -- 172.21.15.7:0/864139858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 msgr2=0x7f197c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1980c13700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 0x7f197c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1980c13700 1 -- 172.21.15.7:0/864139858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c198f90 con 0x7f197c101210 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1981414700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 0x7f197c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f197bfff700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101bb0 0x7f197c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1980c13700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 0x7f197c19e630 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9a0 tx=0x7f197000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000ca10 con 0x7f197c101210 2024-03-19T00:37:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f197000cbb0 con 0x7f197c101210 2024-03-19T00:37:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c199280 con 0x7f197c101210 2024-03-19T00:37:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970018680 con 0x7f197c101210 2024-03-19T00:37:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.919+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c1a5490 con 0x7f197c101210 2024-03-19T00:37:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.920+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197c102900 con 0x7f197c101210 2024-03-19T00:37:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.921+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f1970010430 con 0x7f197c101210 2024-03-19T00:37:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.922+0000 7f1979ffb700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f196c0a2920 0x7f196c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.922+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f19700cf2d0 con 0x7f197c101210 2024-03-19T00:37:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.922+0000 7f197bfff700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f196c0a2920 0x7f196c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.923+0000 7f197bfff700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f196c0a2920 0x7f196c0a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f197c0786d0 tx=0x7f1968009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:33.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:33.926+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f197009be50 con 0x7f197c101210 2024-03-19T00:37:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:33 smithi204 ceph-mon[66572]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.132+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f197c1a56c0 con 0x7f197c101210 2024-03-19T00:37:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.133+0000 7f1979ffb700 1 -- 172.21.15.7:0/864139858 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f197009b750 con 0x7f197c101210 2024-03-19T00:37:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:141733920836 2024-03-19T00:37:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f196c0a2920 msgr2=0x7f196c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f196c0a2920 0x7f196c0a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f197c0786d0 tx=0x7f1968009260 comp rx=0 tx=0).stop 2024-03-19T00:37:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 msgr2=0x7f197c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 0x7f197c19e630 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9a0 tx=0x7f197000ecb0 comp rx=0 tx=0).stop 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 shutdown_connections 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f196c0a2920 0x7f196c0a4de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f197c107250 0x7f197c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f197c101210 0x7f197c19e630 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 --2- 172.21.15.7:0/864139858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f197c101bb0 0x7f197c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 >> 172.21.15.7:0/864139858 conn(0x7f197c078140 msgr2=0x7f197c06a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 shutdown_connections 2024-03-19T00:37:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:34.135+0000 7f1982e77700 1 -- 172.21.15.7:0/864139858 wait complete. 2024-03-19T00:37:34.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182473 for osd.3 2024-03-19T00:37:34.479 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:34.706 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:34 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/864139858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T00:37:34.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:34 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/864139858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T00:37:35.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:34 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/864139858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T00:37:35.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 -- 172.21.15.7:0/3155312129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed4101bb0 msgr2=0x7f4ed410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:35.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 --2- 172.21.15.7:0/3155312129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed4101bb0 0x7f4ed410e050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b0a0 tx=0x7f4ecc00b3b0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 -- 172.21.15.7:0/3155312129 shutdown_connections 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 --2- 172.21.15.7:0/3155312129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed4101bb0 0x7f4ed410e050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 --2- 172.21.15.7:0/3155312129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed41072a0 0x7f4ed4107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 --2- 172.21.15.7:0/3155312129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41011f0 0x7f4ed4101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 -- 172.21.15.7:0/3155312129 >> 172.21.15.7:0/3155312129 conn(0x7f4ed4078220 msgr2=0x7f4ed4078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 -- 172.21.15.7:0/3155312129 shutdown_connections 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 -- 172.21.15.7:0/3155312129 wait complete. 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 Processor -- start 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.604+0000 7f4edb014700 1 -- start start 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 0x7f4ed41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4101bb0 0x7f4ed41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41072a0 0x7f4ed419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4111380 con 0x7f4ed41011f0 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4111200 con 0x7f4ed41072a0 2024-03-19T00:37:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4111500 con 0x7f4ed4101bb0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 0x7f4ed41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 0x7f4ed41a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35130/0 (socket says 172.21.15.7:35130) 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 -- 172.21.15.7:0/3370913655 learned_addr learned my addr 172.21.15.7:0/3370913655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed3fff700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4101bb0 0x7f4ed41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed95b1700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41072a0 0x7f4ed419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 -- 172.21.15.7:0/3370913655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4101bb0 msgr2=0x7f4ed41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4101bb0 0x7f4ed41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 -- 172.21.15.7:0/3370913655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41072a0 msgr2=0x7f4ed419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41072a0 0x7f4ed419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 -- 172.21.15.7:0/3370913655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed419b080 con 0x7f4ed41011f0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed95b1700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41072a0 0x7f4ed419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed8db0700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 0x7f4ed41a0760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800e9c0 tx=0x7f4ec800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800cbd0 con 0x7f4ed41011f0 2024-03-19T00:37:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed419b370 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ec800ee60 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.605+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80055b0 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.606+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41a7630 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.606+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed4068fd0 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.608+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f4ec8004750 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.608+0000 7f4ed1ffb700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ec40a2920 0x7f4ec40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.608+0000 7f4ed3fff700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ec40a2920 0x7f4ec40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.608+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4ec80cf220 con 0x7f4ed41011f0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.609+0000 7f4ed3fff700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ec40a2920 0x7f4ec40a4de0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009680 tx=0x7f4ec0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.613+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4ec809bda0 con 0x7f4ed41011f0 2024-03-19T00:37:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.820+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f4ed41a78c0 con 0x7f4ed41011f0 2024-03-19T00:37:35.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:35 smithi007 ceph-mon[62854]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.820+0000 7f4ed1ffb700 1 -- 172.21.15.7:0/3370913655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ec809b6a0 con 0x7f4ed41011f0 2024-03-19T00:37:35.822 INFO:teuthology.orchestra.run.smithi007.stdout:253403070512 2024-03-19T00:37:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ec40a2920 msgr2=0x7f4ec40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ec40a2920 0x7f4ec40a4de0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009680 tx=0x7f4ec0009260 comp rx=0 tx=0).stop 2024-03-19T00:37:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 msgr2=0x7f4ed41a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 0x7f4ed41a0760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800e9c0 tx=0x7f4ec800ecd0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 shutdown_connections 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4ec40a2920 0x7f4ec40a4de0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed41072a0 0x7f4ed419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed41011f0 0x7f4ed41a0760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 --2- 172.21.15.7:0/3370913655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ed4101bb0 0x7f4ed41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 >> 172.21.15.7:0/3370913655 conn(0x7f4ed4078220 msgr2=0x7f4ed406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.822+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 shutdown_connections 2024-03-19T00:37:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:35.823+0000 7f4edb014700 1 -- 172.21.15.7:0/3370913655 wait complete. 2024-03-19T00:37:35.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:35 smithi044 ceph-mon[66366]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:36.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:35 smithi204 ceph-mon[66572]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:36.596 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070512 for osd.8 2024-03-19T00:37:36.596 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:36.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:36 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3370913655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T00:37:36.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:36 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3370913655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T00:37:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:36 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3370913655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T00:37:37.234 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920836 for osd.4 2024-03-19T00:37:37.234 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:37 smithi044 ceph-mon[66366]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:37.859 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:37 smithi007 ceph-mon[62854]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 -- 172.21.15.7:0/4235325601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c108430 msgr2=0x7f158c108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 --2- 172.21.15.7:0/4235325601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c108430 0x7f158c108810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1588009990 tx=0x7f1588009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 -- 172.21.15.7:0/4235325601 shutdown_connections 2024-03-19T00:37:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 --2- 172.21.15.7:0/4235325601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c102d80 0x7f158c10f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 --2- 172.21.15.7:0/4235325601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c108430 0x7f158c108810 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 --2- 172.21.15.7:0/4235325601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c1023c0 0x7f158c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 -- 172.21.15.7:0/4235325601 >> 172.21.15.7:0/4235325601 conn(0x7f158c0793e0 msgr2=0x7f158c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 -- 172.21.15.7:0/4235325601 shutdown_connections 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.983+0000 7f1594676700 1 -- 172.21.15.7:0/4235325601 wait complete. 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 Processor -- start 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 -- start start 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 0x7f158c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c102d80 0x7f158c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c108430 0x7f158c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112500 con 0x7f158c1023c0 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112380 con 0x7f158c108430 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1594676700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112680 con 0x7f158c102d80 2024-03-19T00:37:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 0x7f158c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 0x7f158c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35148/0 (socket says 172.21.15.7:35148) 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 -- 172.21.15.7:0/3420079121 learned_addr learned my addr 172.21.15.7:0/3420079121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1591c11700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c102d80 0x7f158c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592c13700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c108430 0x7f158c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 -- 172.21.15.7:0/3420079121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c102d80 msgr2=0x7f158c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c102d80 0x7f158c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 -- 172.21.15.7:0/3420079121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c108430 msgr2=0x7f158c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c108430 0x7f158c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592412700 1 -- 172.21.15.7:0/3420079121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c198fc0 con 0x7f158c1023c0 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1591c11700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c102d80 0x7f158c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.984+0000 7f1592c13700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c108430 0x7f158c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.985+0000 7f1592412700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 0x7f158c19e6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f15880057d0 tx=0x7f1588005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.985+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588026070 con 0x7f158c1023c0 2024-03-19T00:37:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.985+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c199250 con 0x7f158c1023c0 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.985+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1588021410 con 0x7f158c1023c0 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.985+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158800b610 con 0x7f158c1023c0 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.985+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a5570 con 0x7f158c1023c0 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.986+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c068fd0 con 0x7f158c1023c0 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.987+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f1588029070 con 0x7f158c1023c0 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.988+0000 7f15837fe700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15780a2920 0x7f15780a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.988+0000 7f1591c11700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15780a2920 0x7f15780a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.988+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f15880cf980 con 0x7f158c1023c0 2024-03-19T00:37:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.988+0000 7f1591c11700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15780a2920 0x7f15780a4de0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f158c079a90 tx=0x7f157c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:37.996+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1588013070 con 0x7f158c1023c0 2024-03-19T00:37:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:37 smithi204 ceph-mon[66572]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.204+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f158c02dbb0 con 0x7f158c1023c0 2024-03-19T00:37:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.205+0000 7f15837fe700 1 -- 172.21.15.7:0/3420079121 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f158809c680 con 0x7f158c1023c0 2024-03-19T00:37:38.206 INFO:teuthology.orchestra.run.smithi007.stdout:309237645350 2024-03-19T00:37:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15780a2920 msgr2=0x7f15780a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15780a2920 0x7f15780a4de0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f158c079a90 tx=0x7f157c009260 comp rx=0 tx=0).stop 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 msgr2=0x7f158c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 0x7f158c19e6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f15880057d0 tx=0x7f1588005800 comp rx=0 tx=0).stop 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 shutdown_connections 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15780a2920 0x7f15780a4de0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f158c108430 0x7f158c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c1023c0 0x7f158c19e6a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 --2- 172.21.15.7:0/3420079121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f158c102d80 0x7f158c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 >> 172.21.15.7:0/3420079121 conn(0x7f158c0793e0 msgr2=0x7f158c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 shutdown_connections 2024-03-19T00:37:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:38.207+0000 7f1594676700 1 -- 172.21.15.7:0/3420079121 wait complete. 2024-03-19T00:37:38.927 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:38.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:38.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:38 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3420079121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T00:37:38.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:37:38.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:37:38.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:38.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645344 got 309237645350 for osd.10 2024-03-19T00:37:38.952 DEBUG:teuthology.parallel:result is None 2024-03-19T00:37:38.952 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-19T00:37:38.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-19T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:38 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3420079121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:38 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3420079121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T00:37:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:37:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:37:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:37:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:39 smithi007 ceph-mon[62854]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:39 smithi044 ceph-mon[66366]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:40.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:39 smithi204 ceph-mon[66572]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1580629776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8108340 msgr2=0x7f0ab8108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1580629776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8108340 0x7f0ab8108720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40099c0 tx=0x7f0ab4009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1580629776 shutdown_connections 2024-03-19T00:37:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1580629776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab8102d20 0x7f0ab810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1580629776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8108340 0x7f0ab8108720 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1580629776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ab8102360 0x7f0ab81027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1580629776 >> 172.21.15.7:0/1580629776 conn(0x7f0ab8079440 msgr2=0x7f0ab8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1580629776 shutdown_connections 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.945+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1580629776 wait complete. 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 Processor -- start 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 -- start start 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 0x7f0ab8074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ab8102d20 0x7f0ab8074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab8108340 0x7f0ab8075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab81124a0 con 0x7f0ab8102360 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8112320 con 0x7f0ab8102d20 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abfc8a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8112620 con 0x7f0ab8108340 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abda26700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 0x7f0ab8074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abda26700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 0x7f0ab8074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35168/0 (socket says 172.21.15.7:35168) 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abda26700 1 -- 172.21.15.7:0/1881852693 learned_addr learned my addr 172.21.15.7:0/1881852693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.946+0000 7f0abd225700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ab8102d20 0x7f0ab8074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abe227700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab8108340 0x7f0ab8075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abda26700 1 -- 172.21.15.7:0/1881852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab8108340 msgr2=0x7f0ab8075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abda26700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab8108340 0x7f0ab8075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abda26700 1 -- 172.21.15.7:0/1881852693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ab8102d20 msgr2=0x7f0ab8074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abda26700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ab8102d20 0x7f0ab8074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abda26700 1 -- 172.21.15.7:0/1881852693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab81a6e70 con 0x7f0ab8102360 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abda26700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 0x7f0ab8074610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4026070 con 0x7f0ab8102360 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4004630 con 0x7f0ab8102360 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab81a7100 con 0x7f0ab8102360 2024-03-19T00:37:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400fb20 con 0x7f0ab8102360 2024-03-19T00:37:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.947+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab81a75b0 con 0x7f0ab8102360 2024-03-19T00:37:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.948+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab8068fd0 con 0x7f0ab8102360 2024-03-19T00:37:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.949+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f0ab4029070 con 0x7f0ab8102360 2024-03-19T00:37:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.950+0000 7f0aaeffd700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aa40a29a0 0x7f0aa40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.950+0000 7f0abd225700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aa40a29a0 0x7f0aa40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.950+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0ab40cfb50 con 0x7f0ab8102360 2024-03-19T00:37:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.950+0000 7f0abd225700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aa40a29a0 0x7f0aa40a4e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8079ab0 tx=0x7f0aa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:40.953+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ab4030080 con 0x7f0ab8102360 2024-03-19T00:37:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.154+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ab8062750 con 0x7f0aa40a29a0 2024-03-19T00:37:41.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.164+0000 7f0aaeffd700 1 -- 172.21.15.7:0/1881852693 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340360 (secure 0 0 0) 0x7f0ab8062750 con 0x7f0aa40a29a0 2024-03-19T00:37:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:41.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aa40a29a0 msgr2=0x7f0aa40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:41.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aa40a29a0 0x7f0aa40a4e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8079ab0 tx=0x7f0aa8009260 comp rx=0 tx=0).stop 2024-03-19T00:37:41.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 msgr2=0x7f0ab8074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:41.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 0x7f0ab8074610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab4004270 comp rx=0 tx=0).stop 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 shutdown_connections 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aa40a29a0 0x7f0aa40a4e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab8108340 0x7f0ab8075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8102360 0x7f0ab8074610 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 --2- 172.21.15.7:0/1881852693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ab8102d20 0x7f0ab8074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 >> 172.21.15.7:0/1881852693 conn(0x7f0ab8079440 msgr2=0x7f0ab8101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 shutdown_connections 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:41.169+0000 7f0abfc8a700 1 -- 172.21.15.7:0/1881852693 wait complete. 2024-03-19T00:37:41.174 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:37:41.922 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-19T00:37:40.024160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658351,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":593,"num_read_kb":415,"num_write":345,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404912,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536912,"statfs":{"total":1151940427776,"available":1151525797888,"internally_reserved":0,"allocated":14434304,"data_stored":7960763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002854"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946530+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:36:57.946530+0000","last_peered":"2024-03-19T00:36:57.946530+0000","last_clean":"2024-03-19T00:36:57.946530+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:36:57.946530+0000","last_undegraded":"2024-03-19T00:36:57.946530+0000","last_fullsized":"2024-03-19T00:36:57.946530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947489+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:36:57.947489+0000","last_peered":"2024-03-19T00:36:57.947489+0000","last_clean":"2024-03-19T00:36:57.947489+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:36:57.947489+0000","last_undegraded":"2024-03-19T00:36:57.947489+0000","last_fullsized":"2024-03-19T00:36:57.947489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947223+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:36:57.947223+0000","last_peered":"2024-03-19T00:36:57.947223+0000","last_clean":"2024-03-19T00:36:57.947223+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:36:57.947223+0000","last_undegraded":"2024-03-19T00:36:57.947223+0000","last_fullsized":"2024-03-19T00:36:57.947223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944781+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:36:57.944781+0000","last_peered":"2024-03-19T00:36:57.944781+0000","last_clean":"2024-03-19T00:36:57.944781+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:36:57.944781+0000","last_undegraded":"2024-03-19T00:36:57.944781+0000","last_fullsized":"2024-03-19T00:36:57.944781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947569+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:36:57.947569+0000","last_peered":"2024-03-19T00:36:57.947569+0000","last_clean":"2024-03-19T00:36:57.947569+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:36:57.947569+0000","last_undegraded":"2024-03-19T00:36:57.947569+0000","last_fullsized":"2024-03-19T00:36:57.947569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297097+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:36:58.297097+0000","last_peered":"2024-03-19T00:36:58.297097+0000","last_clean":"2024-03-19T00:36:58.297097+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:36:58.297097+0000","last_undegraded":"2024-03-19T00:36:58.297097+0000","last_fullsized":"2024-03-19T00:36:58.297097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092426+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:36:58.092426+0000","last_peered":"2024-03-19T00:36:58.092426+0000","last_clean":"2024-03-19T00:36:58.092426+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:36:58.092426+0000","last_undegraded":"2024-03-19T00:36:58.092426+0000","last_fullsized":"2024-03-19T00:36:58.092426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946148+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:36:57.946148+0000","last_peered":"2024-03-19T00:36:57.946148+0000","last_clean":"2024-03-19T00:36:57.946148+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:36:57.946148+0000","last_undegraded":"2024-03-19T00:36:57.946148+0000","last_fullsized":"2024-03-19T00:36:57.946148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946720+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:36:57.946720+0000","last_peered":"2024-03-19T00:36:57.946720+0000","last_clean":"2024-03-19T00:36:57.946720+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:36:57.946720+0000","last_undegraded":"2024-03-19T00:36:57.946720+0000","last_fullsized":"2024-03-19T00:36:57.946720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297440+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:36:58.297440+0000","last_peered":"2024-03-19T00:36:58.297440+0000","last_clean":"2024-03-19T00:36:58.297440+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:36:58.297440+0000","last_undegraded":"2024-03-19T00:36:58.297440+0000","last_fullsized":"2024-03-19T00:36:58.297440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.093053+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:36:58.093053+0000","last_peered":"2024-03-19T00:36:58.093053+0000","last_clean":"2024-03-19T00:36:58.093053+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:36:58.093053+0000","last_undegraded":"2024-03-19T00:36:58.093053+0000","last_fullsized":"2024-03-19T00:36:58.093053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297533+0000","last_change":"2024-03-19T00:36:58.297533+0000","last_active":"2024-03-19T00:36:58.297533+0000","last_peered":"2024-03-19T00:36:58.297533+0000","last_clean":"2024-03-19T00:36:58.297533+0000","last_became_active":"2024-03-19T00:36:58.297218+0000","last_became_peered":"2024-03-19T00:36:58.297218+0000","last_unstale":"2024-03-19T00:36:58.297533+0000","last_undegraded":"2024-03-19T00:36:58.297533+0000","last_fullsized":"2024-03-19T00:36:58.297533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:06.256765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018519+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:36:58.018519+0000","last_peered":"2024-03-19T00:36:58.018519+0000","last_clean":"2024-03-19T00:36:58.018519+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:36:58.018519+0000","last_undegraded":"2024-03-19T00:36:58.018519+0000","last_fullsized":"2024-03-19T00:36:58.018519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946782+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:36:57.946782+0000","last_peered":"2024-03-19T00:36:57.946782+0000","last_clean":"2024-03-19T00:36:57.946782+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:36:57.946782+0000","last_undegraded":"2024-03-19T00:36:57.946782+0000","last_fullsized":"2024-03-19T00:36:57.946782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946540+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:36:57.946540+0000","last_peered":"2024-03-19T00:36:57.946540+0000","last_clean":"2024-03-19T00:36:57.946540+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:36:57.946540+0000","last_undegraded":"2024-03-19T00:36:57.946540+0000","last_fullsized":"2024-03-19T00:36:57.946540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946136+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:36:57.946136+0000","last_peered":"2024-03-19T00:36:57.946136+0000","last_clean":"2024-03-19T00:36:57.946136+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:36:57.946136+0000","last_undegraded":"2024-03-19T00:36:57.946136+0000","last_fullsized":"2024-03-19T00:36:57.946136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946442+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:36:57.946442+0000","last_peered":"2024-03-19T00:36:57.946442+0000","last_clean":"2024-03-19T00:36:57.946442+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:36:57.946442+0000","last_undegraded":"2024-03-19T00:36:57.946442+0000","last_fullsized":"2024-03-19T00:36:57.946442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944003+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:36:57.944003+0000","last_peered":"2024-03-19T00:36:57.944003+0000","last_clean":"2024-03-19T00:36:57.944003+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:36:57.944003+0000","last_undegraded":"2024-03-19T00:36:57.944003+0000","last_fullsized":"2024-03-19T00:36:57.944003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945235+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:36:57.945235+0000","last_peered":"2024-03-19T00:36:57.945235+0000","last_clean":"2024-03-19T00:36:57.945235+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:36:57.945235+0000","last_undegraded":"2024-03-19T00:36:57.945235+0000","last_fullsized":"2024-03-19T00:36:57.945235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944493+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:57.944493+0000","last_peered":"2024-03-19T00:36:57.944493+0000","last_clean":"2024-03-19T00:36:57.944493+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:57.944493+0000","last_undegraded":"2024-03-19T00:36:57.944493+0000","last_fullsized":"2024-03-19T00:36:57.944493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943824+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:57.943824+0000","last_peered":"2024-03-19T00:36:57.943824+0000","last_clean":"2024-03-19T00:36:57.943824+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:57.943824+0000","last_undegraded":"2024-03-19T00:36:57.943824+0000","last_fullsized":"2024-03-19T00:36:57.943824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947402+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:36:57.947402+0000","last_peered":"2024-03-19T00:36:57.947402+0000","last_clean":"2024-03-19T00:36:57.947402+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:36:57.947402+0000","last_undegraded":"2024-03-19T00:36:57.947402+0000","last_fullsized":"2024-03-19T00:36:57.947402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943659+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:57.943659+0000","last_peered":"2024-03-19T00:36:57.943659+0000","last_clean":"2024-03-19T00:36:57.943659+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:57.943659+0000","last_undegraded":"2024-03-19T00:36:57.943659+0000","last_fullsized":"2024-03-19T00:36:57.943659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947499+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:57.947499+0000","last_peered":"2024-03-19T00:36:57.947499+0000","last_clean":"2024-03-19T00:36:57.947499+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:57.947499+0000","last_undegraded":"2024-03-19T00:36:57.947499+0000","last_fullsized":"2024-03-19T00:36:57.947499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947563+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:36:57.947563+0000","last_peered":"2024-03-19T00:36:57.947563+0000","last_clean":"2024-03-19T00:36:57.947563+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:36:57.947563+0000","last_undegraded":"2024-03-19T00:36:57.947563+0000","last_fullsized":"2024-03-19T00:36:57.947563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943869+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:57.943869+0000","last_peered":"2024-03-19T00:36:57.943869+0000","last_clean":"2024-03-19T00:36:57.943869+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:57.943869+0000","last_undegraded":"2024-03-19T00:36:57.943869+0000","last_fullsized":"2024-03-19T00:36:57.943869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947966+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:57.947966+0000","last_peered":"2024-03-19T00:36:57.947966+0000","last_clean":"2024-03-19T00:36:57.947966+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:57.947966+0000","last_undegraded":"2024-03-19T00:36:57.947966+0000","last_fullsized":"2024-03-19T00:36:57.947966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946545+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:36:57.946545+0000","last_peered":"2024-03-19T00:36:57.946545+0000","last_clean":"2024-03-19T00:36:57.946545+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:36:57.946545+0000","last_undegraded":"2024-03-19T00:36:57.946545+0000","last_fullsized":"2024-03-19T00:36:57.946545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943744+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:57.943744+0000","last_peered":"2024-03-19T00:36:57.943744+0000","last_clean":"2024-03-19T00:36:57.943744+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:57.943744+0000","last_undegraded":"2024-03-19T00:36:57.943744+0000","last_fullsized":"2024-03-19T00:36:57.943744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948143+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:57.948143+0000","last_peered":"2024-03-19T00:36:57.948143+0000","last_clean":"2024-03-19T00:36:57.948143+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:57.948143+0000","last_undegraded":"2024-03-19T00:36:57.948143+0000","last_fullsized":"2024-03-19T00:36:57.948143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948938+0000","last_change":"2024-03-19T00:36:57.948938+0000","last_active":"2024-03-19T00:36:57.948938+0000","last_peered":"2024-03-19T00:36:57.948938+0000","last_clean":"2024-03-19T00:36:57.948938+0000","last_became_active":"2024-03-19T00:36:57.948519+0000","last_became_peered":"2024-03-19T00:36:57.948519+0000","last_unstale":"2024-03-19T00:36:57.948938+0000","last_undegraded":"2024-03-19T00:36:57.948938+0000","last_fullsized":"2024-03-19T00:36:57.948938+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:33:49.099555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.298764+0000","last_change":"2024-03-19T00:36:58.298764+0000","last_active":"2024-03-19T00:36:58.298764+0000","last_peered":"2024-03-19T00:36:58.298764+0000","last_clean":"2024-03-19T00:36:58.298764+0000","last_became_active":"2024-03-19T00:36:58.298265+0000","last_became_peered":"2024-03-19T00:36:58.298265+0000","last_unstale":"2024-03-19T00:36:58.298764+0000","last_undegraded":"2024-03-19T00:36:58.298764+0000","last_fullsized":"2024-03-19T00:36:58.298764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:45.983861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:37.381147+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:37:37.381147+0000","last_peered":"2024-03-19T00:37:37.381147+0000","last_clean":"2024-03-19T00:37:37.381147+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:37:37.381147+0000","last_undegraded":"2024-03-19T00:37:37.381147+0000","last_fullsized":"2024-03-19T00:37:37.381147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947887+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:36:57.947887+0000","last_peered":"2024-03-19T00:36:57.947887+0000","last_clean":"2024-03-19T00:36:57.947887+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:36:57.947887+0000","last_undegraded":"2024-03-19T00:36:57.947887+0000","last_fullsized":"2024-03-19T00:36:57.947887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342937+0000","last_change":"2024-03-19T00:36:58.342937+0000","last_active":"2024-03-19T00:36:58.342937+0000","last_peered":"2024-03-19T00:36:58.342937+0000","last_clean":"2024-03-19T00:36:58.342937+0000","last_became_active":"2024-03-19T00:36:58.342474+0000","last_became_peered":"2024-03-19T00:36:58.342474+0000","last_unstale":"2024-03-19T00:36:58.342937+0000","last_undegraded":"2024-03-19T00:36:58.342937+0000","last_fullsized":"2024-03-19T00:36:58.342937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:55.531546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297043+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:58.297043+0000","last_peered":"2024-03-19T00:36:58.297043+0000","last_clean":"2024-03-19T00:36:58.297043+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:58.297043+0000","last_undegraded":"2024-03-19T00:36:58.297043+0000","last_fullsized":"2024-03-19T00:36:58.297043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342182+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:36:58.342182+0000","last_peered":"2024-03-19T00:36:58.342182+0000","last_clean":"2024-03-19T00:36:58.342182+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:36:58.342182+0000","last_undegraded":"2024-03-19T00:36:58.342182+0000","last_fullsized":"2024-03-19T00:36:58.342182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.950687+0000","last_change":"2024-03-19T00:36:57.950687+0000","last_active":"2024-03-19T00:36:57.950687+0000","last_peered":"2024-03-19T00:36:57.950687+0000","last_clean":"2024-03-19T00:36:57.950687+0000","last_became_active":"2024-03-19T00:36:57.950151+0000","last_became_peered":"2024-03-19T00:36:57.950151+0000","last_unstale":"2024-03-19T00:36:57.950687+0000","last_undegraded":"2024-03-19T00:36:57.950687+0000","last_fullsized":"2024-03-19T00:36:57.950687+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:41.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018228+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:58.018228+0000","last_peered":"2024-03-19T00:36:58.018228+0000","last_clean":"2024-03-19T00:36:58.018228+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:58.018228+0000","last_undegraded":"2024-03-19T00:36:58.018228+0000","last_fullsized":"2024-03-19T00:36:58.018228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946096+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:36:57.946096+0000","last_peered":"2024-03-19T00:36:57.946096+0000","last_clean":"2024-03-19T00:36:57.946096+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:36:57.946096+0000","last_undegraded":"2024-03-19T00:36:57.946096+0000","last_fullsized":"2024-03-19T00:36:57.946096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947422+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:57.947422+0000","last_peered":"2024-03-19T00:36:57.947422+0000","last_clean":"2024-03-19T00:36:57.947422+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:57.947422+0000","last_undegraded":"2024-03-19T00:36:57.947422+0000","last_fullsized":"2024-03-19T00:36:57.947422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341344+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:58.341344+0000","last_peered":"2024-03-19T00:36:58.341344+0000","last_clean":"2024-03-19T00:36:58.341344+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:58.341344+0000","last_undegraded":"2024-03-19T00:36:58.341344+0000","last_fullsized":"2024-03-19T00:36:58.341344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943255+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:36:57.943255+0000","last_peered":"2024-03-19T00:36:57.943255+0000","last_clean":"2024-03-19T00:36:57.943255+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:36:57.943255+0000","last_undegraded":"2024-03-19T00:36:57.943255+0000","last_fullsized":"2024-03-19T00:36:57.943255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946477+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:57.946477+0000","last_peered":"2024-03-19T00:36:57.946477+0000","last_clean":"2024-03-19T00:36:57.946477+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:57.946477+0000","last_undegraded":"2024-03-19T00:36:57.946477+0000","last_fullsized":"2024-03-19T00:36:57.946477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:37.380682+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:37:37.380682+0000","last_peered":"2024-03-19T00:37:37.380682+0000","last_clean":"2024-03-19T00:37:37.380682+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:37:37.380682+0000","last_undegraded":"2024-03-19T00:37:37.380682+0000","last_fullsized":"2024-03-19T00:37:37.380682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946907+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:36:57.946907+0000","last_peered":"2024-03-19T00:36:57.946907+0000","last_clean":"2024-03-19T00:36:57.946907+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:36:57.946907+0000","last_undegraded":"2024-03-19T00:36:57.946907+0000","last_fullsized":"2024-03-19T00:36:57.946907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943152+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:57.943152+0000","last_peered":"2024-03-19T00:36:57.943152+0000","last_clean":"2024-03-19T00:36:57.943152+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:57.943152+0000","last_undegraded":"2024-03-19T00:36:57.943152+0000","last_fullsized":"2024-03-19T00:36:57.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092828+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:58.092828+0000","last_peered":"2024-03-19T00:36:58.092828+0000","last_clean":"2024-03-19T00:36:58.092828+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:58.092828+0000","last_undegraded":"2024-03-19T00:36:58.092828+0000","last_fullsized":"2024-03-19T00:36:58.092828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947451+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:36:57.947451+0000","last_peered":"2024-03-19T00:36:57.947451+0000","last_clean":"2024-03-19T00:36:57.947451+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:36:57.947451+0000","last_undegraded":"2024-03-19T00:36:57.947451+0000","last_fullsized":"2024-03-19T00:36:57.947451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944083+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:57.944083+0000","last_peered":"2024-03-19T00:36:57.944083+0000","last_clean":"2024-03-19T00:36:57.944083+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:57.944083+0000","last_undegraded":"2024-03-19T00:36:57.944083+0000","last_fullsized":"2024-03-19T00:36:57.944083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341822+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:58.341822+0000","last_peered":"2024-03-19T00:36:58.341822+0000","last_clean":"2024-03-19T00:36:58.341822+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:58.341822+0000","last_undegraded":"2024-03-19T00:36:58.341822+0000","last_fullsized":"2024-03-19T00:36:58.341822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946646+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:36:57.946646+0000","last_peered":"2024-03-19T00:36:57.946646+0000","last_clean":"2024-03-19T00:36:57.946646+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:36:57.946646+0000","last_undegraded":"2024-03-19T00:36:57.946646+0000","last_fullsized":"2024-03-19T00:36:57.946646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943258+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:57.943258+0000","last_peered":"2024-03-19T00:36:57.943258+0000","last_clean":"2024-03-19T00:36:57.943258+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:57.943258+0000","last_undegraded":"2024-03-19T00:36:57.943258+0000","last_fullsized":"2024-03-19T00:36:57.943258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":214,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:38.381759+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:37:38.381759+0000","last_peered":"2024-03-19T00:37:38.381759+0000","last_clean":"2024-03-19T00:37:38.381759+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:37:38.381759+0000","last_undegraded":"2024-03-19T00:37:38.381759+0000","last_fullsized":"2024-03-19T00:37:38.381759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63357,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":154,"num_read_kb":139,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947770+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:36:57.947770+0000","last_peered":"2024-03-19T00:36:57.947770+0000","last_clean":"2024-03-19T00:36:57.947770+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:36:57.947770+0000","last_undegraded":"2024-03-19T00:36:57.947770+0000","last_fullsized":"2024-03-19T00:36:57.947770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342057+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:58.342057+0000","last_peered":"2024-03-19T00:36:58.342057+0000","last_clean":"2024-03-19T00:36:58.342057+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:58.342057+0000","last_undegraded":"2024-03-19T00:36:58.342057+0000","last_fullsized":"2024-03-19T00:36:58.342057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945034+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:57.945034+0000","last_peered":"2024-03-19T00:36:57.945034+0000","last_clean":"2024-03-19T00:36:57.945034+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:57.945034+0000","last_undegraded":"2024-03-19T00:36:57.945034+0000","last_fullsized":"2024-03-19T00:36:57.945034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018317+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:36:58.018317+0000","last_peered":"2024-03-19T00:36:58.018317+0000","last_clean":"2024-03-19T00:36:58.018317+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:36:58.018317+0000","last_undegraded":"2024-03-19T00:36:58.018317+0000","last_fullsized":"2024-03-19T00:36:58.018317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945351+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:57.945351+0000","last_peered":"2024-03-19T00:36:57.945351+0000","last_clean":"2024-03-19T00:36:57.945351+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:57.945351+0000","last_undegraded":"2024-03-19T00:36:57.945351+0000","last_fullsized":"2024-03-19T00:36:57.945351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341835+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:58.341835+0000","last_peered":"2024-03-19T00:36:58.341835+0000","last_clean":"2024-03-19T00:36:58.341835+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:58.341835+0000","last_undegraded":"2024-03-19T00:36:58.341835+0000","last_fullsized":"2024-03-19T00:36:58.341835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092790+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:36:58.092790+0000","last_peered":"2024-03-19T00:36:58.092790+0000","last_clean":"2024-03-19T00:36:58.092790+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:36:58.092790+0000","last_undegraded":"2024-03-19T00:36:58.092790+0000","last_fullsized":"2024-03-19T00:36:58.092790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943691+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:57.943691+0000","last_peered":"2024-03-19T00:36:57.943691+0000","last_clean":"2024-03-19T00:36:57.943691+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:57.943691+0000","last_undegraded":"2024-03-19T00:36:57.943691+0000","last_fullsized":"2024-03-19T00:36:57.943691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:37.389624+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:37:37.389624+0000","last_peered":"2024-03-19T00:37:37.389624+0000","last_clean":"2024-03-19T00:37:37.389624+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:37:37.389624+0000","last_undegraded":"2024-03-19T00:37:37.389624+0000","last_fullsized":"2024-03-19T00:37:37.389624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092507+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:36:58.092507+0000","last_peered":"2024-03-19T00:36:58.092507+0000","last_clean":"2024-03-19T00:36:58.092507+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:36:58.092507+0000","last_undegraded":"2024-03-19T00:36:58.092507+0000","last_fullsized":"2024-03-19T00:36:58.092507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018782+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:58.018782+0000","last_peered":"2024-03-19T00:36:58.018782+0000","last_clean":"2024-03-19T00:36:58.018782+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:58.018782+0000","last_undegraded":"2024-03-19T00:36:58.018782+0000","last_fullsized":"2024-03-19T00:36:58.018782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947646+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:57.947646+0000","last_peered":"2024-03-19T00:36:57.947646+0000","last_clean":"2024-03-19T00:36:57.947646+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:57.947646+0000","last_undegraded":"2024-03-19T00:36:57.947646+0000","last_fullsized":"2024-03-19T00:36:57.947646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341807+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:36:58.341807+0000","last_peered":"2024-03-19T00:36:58.341807+0000","last_clean":"2024-03-19T00:36:58.341807+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:36:58.341807+0000","last_undegraded":"2024-03-19T00:36:58.341807+0000","last_fullsized":"2024-03-19T00:36:58.341807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341693+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:58.341693+0000","last_peered":"2024-03-19T00:36:58.341693+0000","last_clean":"2024-03-19T00:36:58.341693+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:58.341693+0000","last_undegraded":"2024-03-19T00:36:58.341693+0000","last_fullsized":"2024-03-19T00:36:58.341693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948726+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:57.948726+0000","last_peered":"2024-03-19T00:36:57.948726+0000","last_clean":"2024-03-19T00:36:57.948726+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:57.948726+0000","last_undegraded":"2024-03-19T00:36:57.948726+0000","last_fullsized":"2024-03-19T00:36:57.948726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946252+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:36:57.946252+0000","last_peered":"2024-03-19T00:36:57.946252+0000","last_clean":"2024-03-19T00:36:57.946252+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:36:57.946252+0000","last_undegraded":"2024-03-19T00:36:57.946252+0000","last_fullsized":"2024-03-19T00:36:57.946252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947846+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:57.947846+0000","last_peered":"2024-03-19T00:36:57.947846+0000","last_clean":"2024-03-19T00:36:57.947846+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:57.947846+0000","last_undegraded":"2024-03-19T00:36:57.947846+0000","last_fullsized":"2024-03-19T00:36:57.947846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943503+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:57.943503+0000","last_peered":"2024-03-19T00:36:57.943503+0000","last_clean":"2024-03-19T00:36:57.943503+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:57.943503+0000","last_undegraded":"2024-03-19T00:36:57.943503+0000","last_fullsized":"2024-03-19T00:36:57.943503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297221+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:36:58.297221+0000","last_peered":"2024-03-19T00:36:58.297221+0000","last_clean":"2024-03-19T00:36:58.297221+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:36:58.297221+0000","last_undegraded":"2024-03-19T00:36:58.297221+0000","last_fullsized":"2024-03-19T00:36:58.297221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018617+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:58.018617+0000","last_peered":"2024-03-19T00:36:58.018617+0000","last_clean":"2024-03-19T00:36:58.018617+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:58.018617+0000","last_undegraded":"2024-03-19T00:36:58.018617+0000","last_fullsized":"2024-03-19T00:36:58.018617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092520+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:58.092520+0000","last_peered":"2024-03-19T00:36:58.092520+0000","last_clean":"2024-03-19T00:36:58.092520+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:58.092520+0000","last_undegraded":"2024-03-19T00:36:58.092520+0000","last_fullsized":"2024-03-19T00:36:58.092520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297222+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:36:58.297222+0000","last_peered":"2024-03-19T00:36:58.297222+0000","last_clean":"2024-03-19T00:36:58.297222+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:36:58.297222+0000","last_undegraded":"2024-03-19T00:36:58.297222+0000","last_fullsized":"2024-03-19T00:36:58.297222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018710+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:58.018710+0000","last_peered":"2024-03-19T00:36:58.018710+0000","last_clean":"2024-03-19T00:36:58.018710+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:58.018710+0000","last_undegraded":"2024-03-19T00:36:58.018710+0000","last_fullsized":"2024-03-19T00:36:58.018710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341417+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:36:58.341417+0000","last_peered":"2024-03-19T00:36:58.341417+0000","last_clean":"2024-03-19T00:36:58.341417+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:36:58.341417+0000","last_undegraded":"2024-03-19T00:36:58.341417+0000","last_fullsized":"2024-03-19T00:36:58.341417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:38.390275+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:37:38.390275+0000","last_peered":"2024-03-19T00:37:38.390275+0000","last_clean":"2024-03-19T00:37:38.390275+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:37:38.390275+0000","last_undegraded":"2024-03-19T00:37:38.390275+0000","last_fullsized":"2024-03-19T00:37:38.390275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092809+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:58.092809+0000","last_peered":"2024-03-19T00:36:58.092809+0000","last_clean":"2024-03-19T00:36:58.092809+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:58.092809+0000","last_undegraded":"2024-03-19T00:36:58.092809+0000","last_fullsized":"2024-03-19T00:36:58.092809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946398+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:36:57.946398+0000","last_peered":"2024-03-19T00:36:57.946398+0000","last_clean":"2024-03-19T00:36:57.946398+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:36:57.946398+0000","last_undegraded":"2024-03-19T00:36:57.946398+0000","last_fullsized":"2024-03-19T00:36:57.946398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943975+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:57.943975+0000","last_peered":"2024-03-19T00:36:57.943975+0000","last_clean":"2024-03-19T00:36:57.943975+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:57.943975+0000","last_undegraded":"2024-03-19T00:36:57.943975+0000","last_fullsized":"2024-03-19T00:36:57.943975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092415+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:58.092415+0000","last_peered":"2024-03-19T00:36:58.092415+0000","last_clean":"2024-03-19T00:36:58.092415+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:58.092415+0000","last_undegraded":"2024-03-19T00:36:58.092415+0000","last_fullsized":"2024-03-19T00:36:58.092415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341960+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:58.341960+0000","last_peered":"2024-03-19T00:36:58.341960+0000","last_clean":"2024-03-19T00:36:58.341960+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:58.341960+0000","last_undegraded":"2024-03-19T00:36:58.341960+0000","last_fullsized":"2024-03-19T00:36:58.341960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018094+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:36:58.018094+0000","last_peered":"2024-03-19T00:36:58.018094+0000","last_clean":"2024-03-19T00:36:58.018094+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:36:58.018094+0000","last_undegraded":"2024-03-19T00:36:58.018094+0000","last_fullsized":"2024-03-19T00:36:58.018094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092437+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:58.092437+0000","last_peered":"2024-03-19T00:36:58.092437+0000","last_clean":"2024-03-19T00:36:58.092437+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:58.092437+0000","last_undegraded":"2024-03-19T00:36:58.092437+0000","last_fullsized":"2024-03-19T00:36:58.092437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945763+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:57.945763+0000","last_peered":"2024-03-19T00:36:57.945763+0000","last_clean":"2024-03-19T00:36:57.945763+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:57.945763+0000","last_undegraded":"2024-03-19T00:36:57.945763+0000","last_fullsized":"2024-03-19T00:36:57.945763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947645+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:57.947645+0000","last_peered":"2024-03-19T00:36:57.947645+0000","last_clean":"2024-03-19T00:36:57.947645+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:57.947645+0000","last_undegraded":"2024-03-19T00:36:57.947645+0000","last_fullsized":"2024-03-19T00:36:57.947645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018762+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:36:58.018762+0000","last_peered":"2024-03-19T00:36:58.018762+0000","last_clean":"2024-03-19T00:36:58.018762+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:36:58.018762+0000","last_undegraded":"2024-03-19T00:36:58.018762+0000","last_fullsized":"2024-03-19T00:36:58.018762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946042+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:57.946042+0000","last_peered":"2024-03-19T00:36:57.946042+0000","last_clean":"2024-03-19T00:36:57.946042+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:57.946042+0000","last_undegraded":"2024-03-19T00:36:57.946042+0000","last_fullsized":"2024-03-19T00:36:57.946042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.954788+0000","last_change":"2024-03-19T00:36:57.954788+0000","last_active":"2024-03-19T00:36:57.954788+0000","last_peered":"2024-03-19T00:36:57.954788+0000","last_clean":"2024-03-19T00:36:57.954788+0000","last_became_active":"2024-03-19T00:36:57.949916+0000","last_became_peered":"2024-03-19T00:36:57.949916+0000","last_unstale":"2024-03-19T00:36:57.954788+0000","last_undegraded":"2024-03-19T00:36:57.954788+0000","last_fullsized":"2024-03-19T00:36:57.954788+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:09.959915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946940+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:36:57.946940+0000","last_peered":"2024-03-19T00:36:57.946940+0000","last_clean":"2024-03-19T00:36:57.946940+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:36:57.946940+0000","last_undegraded":"2024-03-19T00:36:57.946940+0000","last_fullsized":"2024-03-19T00:36:57.946940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944199+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:57.944199+0000","last_peered":"2024-03-19T00:36:57.944199+0000","last_clean":"2024-03-19T00:36:57.944199+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:57.944199+0000","last_undegraded":"2024-03-19T00:36:57.944199+0000","last_fullsized":"2024-03-19T00:36:57.944199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945111+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:57.945111+0000","last_peered":"2024-03-19T00:36:57.945111+0000","last_clean":"2024-03-19T00:36:57.945111+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:57.945111+0000","last_undegraded":"2024-03-19T00:36:57.945111+0000","last_fullsized":"2024-03-19T00:36:57.945111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947426+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:36:57.947426+0000","last_peered":"2024-03-19T00:36:57.947426+0000","last_clean":"2024-03-19T00:36:57.947426+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:36:57.947426+0000","last_undegraded":"2024-03-19T00:36:57.947426+0000","last_fullsized":"2024-03-19T00:36:57.947426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342763+0000","last_change":"2024-03-19T00:36:58.342763+0000","last_active":"2024-03-19T00:36:58.342763+0000","last_peered":"2024-03-19T00:36:58.342763+0000","last_clean":"2024-03-19T00:36:58.342763+0000","last_became_active":"2024-03-19T00:36:58.342305+0000","last_became_peered":"2024-03-19T00:36:58.342305+0000","last_unstale":"2024-03-19T00:36:58.342763+0000","last_undegraded":"2024-03-19T00:36:58.342763+0000","last_fullsized":"2024-03-19T00:36:58.342763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:44.312347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.952182+0000","last_change":"2024-03-19T00:36:57.952182+0000","last_active":"2024-03-19T00:36:57.952182+0000","last_peered":"2024-03-19T00:36:57.952182+0000","last_clean":"2024-03-19T00:36:57.952182+0000","last_became_active":"2024-03-19T00:36:57.948822+0000","last_became_peered":"2024-03-19T00:36:57.948822+0000","last_unstale":"2024-03-19T00:36:57.952182+0000","last_undegraded":"2024-03-19T00:36:57.952182+0000","last_fullsized":"2024-03-19T00:36:57.952182+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.527748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947418+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:36:57.947418+0000","last_peered":"2024-03-19T00:36:57.947418+0000","last_clean":"2024-03-19T00:36:57.947418+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:36:57.947418+0000","last_undegraded":"2024-03-19T00:36:57.947418+0000","last_fullsized":"2024-03-19T00:36:57.947418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943795+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:57.943795+0000","last_peered":"2024-03-19T00:36:57.943795+0000","last_clean":"2024-03-19T00:36:57.943795+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:57.943795+0000","last_undegraded":"2024-03-19T00:36:57.943795+0000","last_fullsized":"2024-03-19T00:36:57.943795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946295+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:57.946295+0000","last_peered":"2024-03-19T00:36:57.946295+0000","last_clean":"2024-03-19T00:36:57.946295+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:57.946295+0000","last_undegraded":"2024-03-19T00:36:57.946295+0000","last_fullsized":"2024-03-19T00:36:57.946295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945488+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:36:57.945488+0000","last_peered":"2024-03-19T00:36:57.945488+0000","last_clean":"2024-03-19T00:36:57.945488+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:36:57.945488+0000","last_undegraded":"2024-03-19T00:36:57.945488+0000","last_fullsized":"2024-03-19T00:36:57.945488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944664+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:57.944664+0000","last_peered":"2024-03-19T00:36:57.944664+0000","last_clean":"2024-03-19T00:36:57.944664+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:57.944664+0000","last_undegraded":"2024-03-19T00:36:57.944664+0000","last_fullsized":"2024-03-19T00:36:57.944664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944531+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:36:57.944531+0000","last_peered":"2024-03-19T00:36:57.944531+0000","last_clean":"2024-03-19T00:36:57.944531+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:36:57.944531+0000","last_undegraded":"2024-03-19T00:36:57.944531+0000","last_fullsized":"2024-03-19T00:36:57.944531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946473+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:36:57.946473+0000","last_peered":"2024-03-19T00:36:57.946473+0000","last_clean":"2024-03-19T00:36:57.946473+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:36:57.946473+0000","last_undegraded":"2024-03-19T00:36:57.946473+0000","last_fullsized":"2024-03-19T00:36:57.946473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944016+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:57.944016+0000","last_peered":"2024-03-19T00:36:57.944016+0000","last_clean":"2024-03-19T00:36:57.944016+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:57.944016+0000","last_undegraded":"2024-03-19T00:36:57.944016+0000","last_fullsized":"2024-03-19T00:36:57.944016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947670+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:57.947670+0000","last_peered":"2024-03-19T00:36:57.947670+0000","last_clean":"2024-03-19T00:36:57.947670+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:57.947670+0000","last_undegraded":"2024-03-19T00:36:57.947670+0000","last_fullsized":"2024-03-19T00:36:57.947670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092661+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:36:58.092661+0000","last_peered":"2024-03-19T00:36:58.092661+0000","last_clean":"2024-03-19T00:36:58.092661+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:36:58.092661+0000","last_undegraded":"2024-03-19T00:36:58.092661+0000","last_fullsized":"2024-03-19T00:36:58.092661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946161+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:57.946161+0000","last_peered":"2024-03-19T00:36:57.946161+0000","last_clean":"2024-03-19T00:36:57.946161+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:57.946161+0000","last_undegraded":"2024-03-19T00:36:57.946161+0000","last_fullsized":"2024-03-19T00:36:57.946161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944946+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:57.944946+0000","last_peered":"2024-03-19T00:36:57.944946+0000","last_clean":"2024-03-19T00:36:57.944946+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:57.944946+0000","last_undegraded":"2024-03-19T00:36:57.944946+0000","last_fullsized":"2024-03-19T00:36:57.944946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.093141+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:36:58.093141+0000","last_peered":"2024-03-19T00:36:58.093141+0000","last_clean":"2024-03-19T00:36:58.093141+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:36:58.093141+0000","last_undegraded":"2024-03-19T00:36:58.093141+0000","last_fullsized":"2024-03-19T00:36:58.093141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947166+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:57.947166+0000","last_peered":"2024-03-19T00:36:57.947166+0000","last_clean":"2024-03-19T00:36:57.947166+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:57.947166+0000","last_undegraded":"2024-03-19T00:36:57.947166+0000","last_fullsized":"2024-03-19T00:36:57.947166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341940+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:58.341940+0000","last_peered":"2024-03-19T00:36:58.341940+0000","last_clean":"2024-03-19T00:36:58.341940+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:58.341940+0000","last_undegraded":"2024-03-19T00:36:58.341940+0000","last_fullsized":"2024-03-19T00:36:58.341940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092594+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:36:58.092594+0000","last_peered":"2024-03-19T00:36:58.092594+0000","last_clean":"2024-03-19T00:36:58.092594+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:36:58.092594+0000","last_undegraded":"2024-03-19T00:36:58.092594+0000","last_fullsized":"2024-03-19T00:36:58.092594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341207+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:58.341207+0000","last_peered":"2024-03-19T00:36:58.341207+0000","last_clean":"2024-03-19T00:36:58.341207+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:58.341207+0000","last_undegraded":"2024-03-19T00:36:58.341207+0000","last_fullsized":"2024-03-19T00:36:58.341207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018745+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:58.018745+0000","last_peered":"2024-03-19T00:36:58.018745+0000","last_clean":"2024-03-19T00:36:58.018745+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:58.018745+0000","last_undegraded":"2024-03-19T00:36:58.018745+0000","last_fullsized":"2024-03-19T00:36:58.018745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092947+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:36:58.092947+0000","last_peered":"2024-03-19T00:36:58.092947+0000","last_clean":"2024-03-19T00:36:58.092947+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:36:58.092947+0000","last_undegraded":"2024-03-19T00:36:58.092947+0000","last_fullsized":"2024-03-19T00:36:58.092947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945517+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:36:57.945517+0000","last_peered":"2024-03-19T00:36:57.945517+0000","last_clean":"2024-03-19T00:36:57.945517+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:36:57.945517+0000","last_undegraded":"2024-03-19T00:36:57.945517+0000","last_fullsized":"2024-03-19T00:36:57.945517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944702+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:36:57.944702+0000","last_peered":"2024-03-19T00:36:57.944702+0000","last_clean":"2024-03-19T00:36:57.944702+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:36:57.944702+0000","last_undegraded":"2024-03-19T00:36:57.944702+0000","last_fullsized":"2024-03-19T00:36:57.944702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944435+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:36:57.944435+0000","last_peered":"2024-03-19T00:36:57.944435+0000","last_clean":"2024-03-19T00:36:57.944435+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:36:57.944435+0000","last_undegraded":"2024-03-19T00:36:57.944435+0000","last_fullsized":"2024-03-19T00:36:57.944435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.019535+0000","last_change":"2024-03-19T00:36:58.019535+0000","last_active":"2024-03-19T00:36:58.019535+0000","last_peered":"2024-03-19T00:36:58.019535+0000","last_clean":"2024-03-19T00:36:58.019535+0000","last_became_active":"2024-03-19T00:36:58.019220+0000","last_became_peered":"2024-03-19T00:36:58.019220+0000","last_unstale":"2024-03-19T00:36:58.019535+0000","last_undegraded":"2024-03-19T00:36:58.019535+0000","last_fullsized":"2024-03-19T00:36:58.019535+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:04:51.797603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018726+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:36:58.018726+0000","last_peered":"2024-03-19T00:36:58.018726+0000","last_clean":"2024-03-19T00:36:58.018726+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:36:58.018726+0000","last_undegraded":"2024-03-19T00:36:58.018726+0000","last_fullsized":"2024-03-19T00:36:58.018726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945683+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:36:57.945683+0000","last_peered":"2024-03-19T00:36:57.945683+0000","last_clean":"2024-03-19T00:36:57.945683+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:36:57.945683+0000","last_undegraded":"2024-03-19T00:36:57.945683+0000","last_fullsized":"2024-03-19T00:36:57.945683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944912+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:36:57.944912+0000","last_peered":"2024-03-19T00:36:57.944912+0000","last_clean":"2024-03-19T00:36:57.944912+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:36:57.944912+0000","last_undegraded":"2024-03-19T00:36:57.944912+0000","last_fullsized":"2024-03-19T00:36:57.944912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947092+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:36:57.947092+0000","last_peered":"2024-03-19T00:36:57.947092+0000","last_clean":"2024-03-19T00:36:57.947092+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:36:57.947092+0000","last_undegraded":"2024-03-19T00:36:57.947092+0000","last_fullsized":"2024-03-19T00:36:57.947092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341314+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:36:58.341314+0000","last_peered":"2024-03-19T00:36:58.341314+0000","last_clean":"2024-03-19T00:36:58.341314+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:36:58.341314+0000","last_undegraded":"2024-03-19T00:36:58.341314+0000","last_fullsized":"2024-03-19T00:36:58.341314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946198+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:36:57.946198+0000","last_peered":"2024-03-19T00:36:57.946198+0000","last_clean":"2024-03-19T00:36:57.946198+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:36:57.946198+0000","last_undegraded":"2024-03-19T00:36:57.946198+0000","last_fullsized":"2024-03-19T00:36:57.946198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946433+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:36:57.946433+0000","last_peered":"2024-03-19T00:36:57.946433+0000","last_clean":"2024-03-19T00:36:57.946433+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:36:57.946433+0000","last_undegraded":"2024-03-19T00:36:57.946433+0000","last_fullsized":"2024-03-19T00:36:57.946433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018580+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:36:58.018580+0000","last_peered":"2024-03-19T00:36:58.018580+0000","last_clean":"2024-03-19T00:36:58.018580+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:36:58.018580+0000","last_undegraded":"2024-03-19T00:36:58.018580+0000","last_fullsized":"2024-03-19T00:36:58.018580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.951256+0000","last_change":"2024-03-19T00:36:57.951256+0000","last_active":"2024-03-19T00:36:57.951256+0000","last_peered":"2024-03-19T00:36:57.951256+0000","last_clean":"2024-03-19T00:36:57.951256+0000","last_became_active":"2024-03-19T00:36:57.950879+0000","last_became_peered":"2024-03-19T00:36:57.950879+0000","last_unstale":"2024-03-19T00:36:57.951256+0000","last_undegraded":"2024-03-19T00:36:57.951256+0000","last_fullsized":"2024-03-19T00:36:57.951256+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:26.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67749,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":467,"num_read_kb":306,"num_write":162,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":203247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1151831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":500971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1634304,"data_stored":1097271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":559494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":561235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":496421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":501268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1642496,"data_stored":1092468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:37:41.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-19T00:37:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:41 smithi044 ceph-mon[66366]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:42.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:41 smithi007 ceph-mon[62854]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:41 smithi204 ceph-mon[66572]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:42.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:42 smithi007 ceph-mon[62854]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:37:43.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:42 smithi044 ceph-mon[66366]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:37:43.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:42 smithi204 ceph-mon[66572]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:37:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 -- 172.21.15.7:0/4240554703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8530107680 msgr2=0x7f853010f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 --2- 172.21.15.7:0/4240554703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8530107680 0x7f853010f200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f852800b0d0 tx=0x7f852800b3e0 comp rx=0 tx=0).stop 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 -- 172.21.15.7:0/4240554703 shutdown_connections 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 --2- 172.21.15.7:0/4240554703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8530107680 0x7f853010f200 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 --2- 172.21.15.7:0/4240554703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853006ac10 0x7f853006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 --2- 172.21.15.7:0/4240554703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006b5c0 0x7f8530106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 -- 172.21.15.7:0/4240554703 >> 172.21.15.7:0/4240554703 conn(0x7f8530078180 msgr2=0x7f8530078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 -- 172.21.15.7:0/4240554703 shutdown_connections 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.948+0000 7f8538413700 1 -- 172.21.15.7:0/4240554703 wait complete. 2024-03-19T00:37:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 Processor -- start 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 -- start start 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006ac10 0x7f85301012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853006b5c0 0x7f8530101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8530107680 0x7f8530101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85301124f0 con 0x7f853006b5c0 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8530112370 con 0x7f853006ac10 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f8538413700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8530112670 con 0x7f8530107680 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f85359ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853006b5c0 0x7f8530101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f85359ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853006b5c0 0x7f8530101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35176/0 (socket says 172.21.15.7:35176) 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f85361af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006ac10 0x7f85301012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.949+0000 7f85359ae700 1 -- 172.21.15.7:0/2903509838 learned_addr learned my addr 172.21.15.7:0/2903509838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85369b0700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8530107680 0x7f8530101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85361af700 1 -- 172.21.15.7:0/2903509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8530107680 msgr2=0x7f8530101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85361af700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8530107680 0x7f8530101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85361af700 1 -- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853006b5c0 msgr2=0x7f8530101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85361af700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853006b5c0 0x7f8530101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85361af700 1 -- 172.21.15.7:0/2903509838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85301a6ef0 con 0x7f853006ac10 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85361af700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006ac10 0x7f85301012d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f852c00e9f0 tx=0x7f852c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c004220 con 0x7f853006ac10 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85301a71e0 con 0x7f853006ac10 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f852c0043c0 con 0x7f853006ac10 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c005020 con 0x7f853006ac10 2024-03-19T00:37:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.950+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85301a76d0 con 0x7f853006ac10 2024-03-19T00:37:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.951+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853010a830 con 0x7f853006ac10 2024-03-19T00:37:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.952+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f852c004750 con 0x7f853006ac10 2024-03-19T00:37:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.953+0000 7f85277fe700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f851c0a2920 0x7f851c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.953+0000 7f85359ae700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f851c0a2920 0x7f851c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.953+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f852c0ceef0 con 0x7f853006ac10 2024-03-19T00:37:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.953+0000 7f85359ae700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f851c0a2920 0x7f851c0a4de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8520009680 tx=0x7f8520009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:43.961+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f852c09ba70 con 0x7f853006ac10 2024-03-19T00:37:43.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:43 smithi007 ceph-mon[62854]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:43 smithi044 ceph-mon[66366]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:44.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.160+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8530062750 con 0x7f851c0a2920 2024-03-19T00:37:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.169+0000 7f85277fe700 1 -- 172.21.15.7:0/2903509838 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340360 (secure 0 0 0) 0x7f8530062750 con 0x7f851c0a2920 2024-03-19T00:37:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f851c0a2920 msgr2=0x7f851c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f851c0a2920 0x7f851c0a4de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8520009680 tx=0x7f8520009260 comp rx=0 tx=0).stop 2024-03-19T00:37:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006ac10 msgr2=0x7f85301012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006ac10 0x7f85301012d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f852c00e9f0 tx=0x7f852c00ed00 comp rx=0 tx=0).stop 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 shutdown_connections 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f851c0a2920 0x7f851c0a4de0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8530107680 0x7f8530101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f853006ac10 0x7f85301012d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 --2- 172.21.15.7:0/2903509838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853006b5c0 0x7f8530101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 >> 172.21.15.7:0/2903509838 conn(0x7f8530078180 msgr2=0x7f85300ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 shutdown_connections 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:44.172+0000 7f8538413700 1 -- 172.21.15.7:0/2903509838 wait complete. 2024-03-19T00:37:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:37:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:43 smithi204 ceph-mon[66572]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:44.993 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-03-19T00:37:44.025154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658351,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":618,"num_read_kb":425,"num_write":345,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404912,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536912,"statfs":{"total":1151940427776,"available":1151525797888,"internally_reserved":0,"allocated":14434304,"data_stored":7960763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946530+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:36:57.946530+0000","last_peered":"2024-03-19T00:36:57.946530+0000","last_clean":"2024-03-19T00:36:57.946530+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:36:57.946530+0000","last_undegraded":"2024-03-19T00:36:57.946530+0000","last_fullsized":"2024-03-19T00:36:57.946530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947489+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:36:57.947489+0000","last_peered":"2024-03-19T00:36:57.947489+0000","last_clean":"2024-03-19T00:36:57.947489+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:36:57.947489+0000","last_undegraded":"2024-03-19T00:36:57.947489+0000","last_fullsized":"2024-03-19T00:36:57.947489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947223+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:36:57.947223+0000","last_peered":"2024-03-19T00:36:57.947223+0000","last_clean":"2024-03-19T00:36:57.947223+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:36:57.947223+0000","last_undegraded":"2024-03-19T00:36:57.947223+0000","last_fullsized":"2024-03-19T00:36:57.947223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944781+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:36:57.944781+0000","last_peered":"2024-03-19T00:36:57.944781+0000","last_clean":"2024-03-19T00:36:57.944781+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:36:57.944781+0000","last_undegraded":"2024-03-19T00:36:57.944781+0000","last_fullsized":"2024-03-19T00:36:57.944781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947569+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:36:57.947569+0000","last_peered":"2024-03-19T00:36:57.947569+0000","last_clean":"2024-03-19T00:36:57.947569+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:36:57.947569+0000","last_undegraded":"2024-03-19T00:36:57.947569+0000","last_fullsized":"2024-03-19T00:36:57.947569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297097+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:36:58.297097+0000","last_peered":"2024-03-19T00:36:58.297097+0000","last_clean":"2024-03-19T00:36:58.297097+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:36:58.297097+0000","last_undegraded":"2024-03-19T00:36:58.297097+0000","last_fullsized":"2024-03-19T00:36:58.297097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092426+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:36:58.092426+0000","last_peered":"2024-03-19T00:36:58.092426+0000","last_clean":"2024-03-19T00:36:58.092426+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:36:58.092426+0000","last_undegraded":"2024-03-19T00:36:58.092426+0000","last_fullsized":"2024-03-19T00:36:58.092426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946148+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:36:57.946148+0000","last_peered":"2024-03-19T00:36:57.946148+0000","last_clean":"2024-03-19T00:36:57.946148+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:36:57.946148+0000","last_undegraded":"2024-03-19T00:36:57.946148+0000","last_fullsized":"2024-03-19T00:36:57.946148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946720+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:36:57.946720+0000","last_peered":"2024-03-19T00:36:57.946720+0000","last_clean":"2024-03-19T00:36:57.946720+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:36:57.946720+0000","last_undegraded":"2024-03-19T00:36:57.946720+0000","last_fullsized":"2024-03-19T00:36:57.946720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297440+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:36:58.297440+0000","last_peered":"2024-03-19T00:36:58.297440+0000","last_clean":"2024-03-19T00:36:58.297440+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:36:58.297440+0000","last_undegraded":"2024-03-19T00:36:58.297440+0000","last_fullsized":"2024-03-19T00:36:58.297440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.093053+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:36:58.093053+0000","last_peered":"2024-03-19T00:36:58.093053+0000","last_clean":"2024-03-19T00:36:58.093053+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:36:58.093053+0000","last_undegraded":"2024-03-19T00:36:58.093053+0000","last_fullsized":"2024-03-19T00:36:58.093053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297533+0000","last_change":"2024-03-19T00:36:58.297533+0000","last_active":"2024-03-19T00:36:58.297533+0000","last_peered":"2024-03-19T00:36:58.297533+0000","last_clean":"2024-03-19T00:36:58.297533+0000","last_became_active":"2024-03-19T00:36:58.297218+0000","last_became_peered":"2024-03-19T00:36:58.297218+0000","last_unstale":"2024-03-19T00:36:58.297533+0000","last_undegraded":"2024-03-19T00:36:58.297533+0000","last_fullsized":"2024-03-19T00:36:58.297533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:06.256765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018519+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:36:58.018519+0000","last_peered":"2024-03-19T00:36:58.018519+0000","last_clean":"2024-03-19T00:36:58.018519+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:36:58.018519+0000","last_undegraded":"2024-03-19T00:36:58.018519+0000","last_fullsized":"2024-03-19T00:36:58.018519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946782+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:36:57.946782+0000","last_peered":"2024-03-19T00:36:57.946782+0000","last_clean":"2024-03-19T00:36:57.946782+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:36:57.946782+0000","last_undegraded":"2024-03-19T00:36:57.946782+0000","last_fullsized":"2024-03-19T00:36:57.946782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946540+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:36:57.946540+0000","last_peered":"2024-03-19T00:36:57.946540+0000","last_clean":"2024-03-19T00:36:57.946540+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:36:57.946540+0000","last_undegraded":"2024-03-19T00:36:57.946540+0000","last_fullsized":"2024-03-19T00:36:57.946540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946136+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:36:57.946136+0000","last_peered":"2024-03-19T00:36:57.946136+0000","last_clean":"2024-03-19T00:36:57.946136+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:36:57.946136+0000","last_undegraded":"2024-03-19T00:36:57.946136+0000","last_fullsized":"2024-03-19T00:36:57.946136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946442+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:36:57.946442+0000","last_peered":"2024-03-19T00:36:57.946442+0000","last_clean":"2024-03-19T00:36:57.946442+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:36:57.946442+0000","last_undegraded":"2024-03-19T00:36:57.946442+0000","last_fullsized":"2024-03-19T00:36:57.946442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944003+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:36:57.944003+0000","last_peered":"2024-03-19T00:36:57.944003+0000","last_clean":"2024-03-19T00:36:57.944003+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:36:57.944003+0000","last_undegraded":"2024-03-19T00:36:57.944003+0000","last_fullsized":"2024-03-19T00:36:57.944003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945235+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:36:57.945235+0000","last_peered":"2024-03-19T00:36:57.945235+0000","last_clean":"2024-03-19T00:36:57.945235+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:36:57.945235+0000","last_undegraded":"2024-03-19T00:36:57.945235+0000","last_fullsized":"2024-03-19T00:36:57.945235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944493+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:57.944493+0000","last_peered":"2024-03-19T00:36:57.944493+0000","last_clean":"2024-03-19T00:36:57.944493+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:57.944493+0000","last_undegraded":"2024-03-19T00:36:57.944493+0000","last_fullsized":"2024-03-19T00:36:57.944493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943824+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:57.943824+0000","last_peered":"2024-03-19T00:36:57.943824+0000","last_clean":"2024-03-19T00:36:57.943824+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:57.943824+0000","last_undegraded":"2024-03-19T00:36:57.943824+0000","last_fullsized":"2024-03-19T00:36:57.943824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947402+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:36:57.947402+0000","last_peered":"2024-03-19T00:36:57.947402+0000","last_clean":"2024-03-19T00:36:57.947402+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:36:57.947402+0000","last_undegraded":"2024-03-19T00:36:57.947402+0000","last_fullsized":"2024-03-19T00:36:57.947402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943659+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:57.943659+0000","last_peered":"2024-03-19T00:36:57.943659+0000","last_clean":"2024-03-19T00:36:57.943659+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:57.943659+0000","last_undegraded":"2024-03-19T00:36:57.943659+0000","last_fullsized":"2024-03-19T00:36:57.943659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947499+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:57.947499+0000","last_peered":"2024-03-19T00:36:57.947499+0000","last_clean":"2024-03-19T00:36:57.947499+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:57.947499+0000","last_undegraded":"2024-03-19T00:36:57.947499+0000","last_fullsized":"2024-03-19T00:36:57.947499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947563+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:36:57.947563+0000","last_peered":"2024-03-19T00:36:57.947563+0000","last_clean":"2024-03-19T00:36:57.947563+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:36:57.947563+0000","last_undegraded":"2024-03-19T00:36:57.947563+0000","last_fullsized":"2024-03-19T00:36:57.947563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943869+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:57.943869+0000","last_peered":"2024-03-19T00:36:57.943869+0000","last_clean":"2024-03-19T00:36:57.943869+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:57.943869+0000","last_undegraded":"2024-03-19T00:36:57.943869+0000","last_fullsized":"2024-03-19T00:36:57.943869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947966+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:57.947966+0000","last_peered":"2024-03-19T00:36:57.947966+0000","last_clean":"2024-03-19T00:36:57.947966+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:57.947966+0000","last_undegraded":"2024-03-19T00:36:57.947966+0000","last_fullsized":"2024-03-19T00:36:57.947966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946545+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:36:57.946545+0000","last_peered":"2024-03-19T00:36:57.946545+0000","last_clean":"2024-03-19T00:36:57.946545+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:36:57.946545+0000","last_undegraded":"2024-03-19T00:36:57.946545+0000","last_fullsized":"2024-03-19T00:36:57.946545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943744+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:57.943744+0000","last_peered":"2024-03-19T00:36:57.943744+0000","last_clean":"2024-03-19T00:36:57.943744+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:57.943744+0000","last_undegraded":"2024-03-19T00:36:57.943744+0000","last_fullsized":"2024-03-19T00:36:57.943744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948143+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:57.948143+0000","last_peered":"2024-03-19T00:36:57.948143+0000","last_clean":"2024-03-19T00:36:57.948143+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:57.948143+0000","last_undegraded":"2024-03-19T00:36:57.948143+0000","last_fullsized":"2024-03-19T00:36:57.948143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948938+0000","last_change":"2024-03-19T00:36:57.948938+0000","last_active":"2024-03-19T00:36:57.948938+0000","last_peered":"2024-03-19T00:36:57.948938+0000","last_clean":"2024-03-19T00:36:57.948938+0000","last_became_active":"2024-03-19T00:36:57.948519+0000","last_became_peered":"2024-03-19T00:36:57.948519+0000","last_unstale":"2024-03-19T00:36:57.948938+0000","last_undegraded":"2024-03-19T00:36:57.948938+0000","last_fullsized":"2024-03-19T00:36:57.948938+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:33:49.099555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.298764+0000","last_change":"2024-03-19T00:36:58.298764+0000","last_active":"2024-03-19T00:36:58.298764+0000","last_peered":"2024-03-19T00:36:58.298764+0000","last_clean":"2024-03-19T00:36:58.298764+0000","last_became_active":"2024-03-19T00:36:58.298265+0000","last_became_peered":"2024-03-19T00:36:58.298265+0000","last_unstale":"2024-03-19T00:36:58.298764+0000","last_undegraded":"2024-03-19T00:36:58.298764+0000","last_fullsized":"2024-03-19T00:36:58.298764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:45.983861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:42.385904+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:37:42.385904+0000","last_peered":"2024-03-19T00:37:42.385904+0000","last_clean":"2024-03-19T00:37:42.385904+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:37:42.385904+0000","last_undegraded":"2024-03-19T00:37:42.385904+0000","last_fullsized":"2024-03-19T00:37:42.385904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947887+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:36:57.947887+0000","last_peered":"2024-03-19T00:36:57.947887+0000","last_clean":"2024-03-19T00:36:57.947887+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:36:57.947887+0000","last_undegraded":"2024-03-19T00:36:57.947887+0000","last_fullsized":"2024-03-19T00:36:57.947887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342937+0000","last_change":"2024-03-19T00:36:58.342937+0000","last_active":"2024-03-19T00:36:58.342937+0000","last_peered":"2024-03-19T00:36:58.342937+0000","last_clean":"2024-03-19T00:36:58.342937+0000","last_became_active":"2024-03-19T00:36:58.342474+0000","last_became_peered":"2024-03-19T00:36:58.342474+0000","last_unstale":"2024-03-19T00:36:58.342937+0000","last_undegraded":"2024-03-19T00:36:58.342937+0000","last_fullsized":"2024-03-19T00:36:58.342937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:55.531546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297043+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:58.297043+0000","last_peered":"2024-03-19T00:36:58.297043+0000","last_clean":"2024-03-19T00:36:58.297043+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:58.297043+0000","last_undegraded":"2024-03-19T00:36:58.297043+0000","last_fullsized":"2024-03-19T00:36:58.297043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342182+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:36:58.342182+0000","last_peered":"2024-03-19T00:36:58.342182+0000","last_clean":"2024-03-19T00:36:58.342182+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:36:58.342182+0000","last_undegraded":"2024-03-19T00:36:58.342182+0000","last_fullsized":"2024-03-19T00:36:58.342182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.950687+0000","last_change":"2024-03-19T00:36:57.950687+0000","last_active":"2024-03-19T00:36:57.950687+0000","last_peered":"2024-03-19T00:36:57.950687+0000","last_clean":"2024-03-19T00:36:57.950687+0000","last_became_active":"2024-03-19T00:36:57.950151+0000","last_became_peered":"2024-03-19T00:36:57.950151+0000","last_unstale":"2024-03-19T00:36:57.950687+0000","last_undegraded":"2024-03-19T00:36:57.950687+0000","last_fullsized":"2024-03-19T00:36:57.950687+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:41.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018228+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:58.018228+0000","last_peered":"2024-03-19T00:36:58.018228+0000","last_clean":"2024-03-19T00:36:58.018228+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:58.018228+0000","last_undegraded":"2024-03-19T00:36:58.018228+0000","last_fullsized":"2024-03-19T00:36:58.018228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946096+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:36:57.946096+0000","last_peered":"2024-03-19T00:36:57.946096+0000","last_clean":"2024-03-19T00:36:57.946096+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:36:57.946096+0000","last_undegraded":"2024-03-19T00:36:57.946096+0000","last_fullsized":"2024-03-19T00:36:57.946096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947422+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:57.947422+0000","last_peered":"2024-03-19T00:36:57.947422+0000","last_clean":"2024-03-19T00:36:57.947422+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:57.947422+0000","last_undegraded":"2024-03-19T00:36:57.947422+0000","last_fullsized":"2024-03-19T00:36:57.947422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341344+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:58.341344+0000","last_peered":"2024-03-19T00:36:58.341344+0000","last_clean":"2024-03-19T00:36:58.341344+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:58.341344+0000","last_undegraded":"2024-03-19T00:36:58.341344+0000","last_fullsized":"2024-03-19T00:36:58.341344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943255+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:36:57.943255+0000","last_peered":"2024-03-19T00:36:57.943255+0000","last_clean":"2024-03-19T00:36:57.943255+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:36:57.943255+0000","last_undegraded":"2024-03-19T00:36:57.943255+0000","last_fullsized":"2024-03-19T00:36:57.943255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946477+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:57.946477+0000","last_peered":"2024-03-19T00:36:57.946477+0000","last_clean":"2024-03-19T00:36:57.946477+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:57.946477+0000","last_undegraded":"2024-03-19T00:36:57.946477+0000","last_fullsized":"2024-03-19T00:36:57.946477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:42.385486+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:37:42.385486+0000","last_peered":"2024-03-19T00:37:42.385486+0000","last_clean":"2024-03-19T00:37:42.385486+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:37:42.385486+0000","last_undegraded":"2024-03-19T00:37:42.385486+0000","last_fullsized":"2024-03-19T00:37:42.385486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946907+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:36:57.946907+0000","last_peered":"2024-03-19T00:36:57.946907+0000","last_clean":"2024-03-19T00:36:57.946907+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:36:57.946907+0000","last_undegraded":"2024-03-19T00:36:57.946907+0000","last_fullsized":"2024-03-19T00:36:57.946907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943152+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:57.943152+0000","last_peered":"2024-03-19T00:36:57.943152+0000","last_clean":"2024-03-19T00:36:57.943152+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:57.943152+0000","last_undegraded":"2024-03-19T00:36:57.943152+0000","last_fullsized":"2024-03-19T00:36:57.943152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092828+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:58.092828+0000","last_peered":"2024-03-19T00:36:58.092828+0000","last_clean":"2024-03-19T00:36:58.092828+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:58.092828+0000","last_undegraded":"2024-03-19T00:36:58.092828+0000","last_fullsized":"2024-03-19T00:36:58.092828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947451+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:36:57.947451+0000","last_peered":"2024-03-19T00:36:57.947451+0000","last_clean":"2024-03-19T00:36:57.947451+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:36:57.947451+0000","last_undegraded":"2024-03-19T00:36:57.947451+0000","last_fullsized":"2024-03-19T00:36:57.947451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944083+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:57.944083+0000","last_peered":"2024-03-19T00:36:57.944083+0000","last_clean":"2024-03-19T00:36:57.944083+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:57.944083+0000","last_undegraded":"2024-03-19T00:36:57.944083+0000","last_fullsized":"2024-03-19T00:36:57.944083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341822+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:58.341822+0000","last_peered":"2024-03-19T00:36:58.341822+0000","last_clean":"2024-03-19T00:36:58.341822+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:58.341822+0000","last_undegraded":"2024-03-19T00:36:58.341822+0000","last_fullsized":"2024-03-19T00:36:58.341822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946646+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:36:57.946646+0000","last_peered":"2024-03-19T00:36:57.946646+0000","last_clean":"2024-03-19T00:36:57.946646+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:36:57.946646+0000","last_undegraded":"2024-03-19T00:36:57.946646+0000","last_fullsized":"2024-03-19T00:36:57.946646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943258+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:57.943258+0000","last_peered":"2024-03-19T00:36:57.943258+0000","last_clean":"2024-03-19T00:36:57.943258+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:57.943258+0000","last_undegraded":"2024-03-19T00:36:57.943258+0000","last_fullsized":"2024-03-19T00:36:57.943258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":224,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:43.386543+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:37:43.386543+0000","last_peered":"2024-03-19T00:37:43.386543+0000","last_clean":"2024-03-19T00:37:43.386543+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:37:43.386543+0000","last_undegraded":"2024-03-19T00:37:43.386543+0000","last_fullsized":"2024-03-19T00:37:43.386543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63357,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947770+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:36:57.947770+0000","last_peered":"2024-03-19T00:36:57.947770+0000","last_clean":"2024-03-19T00:36:57.947770+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:36:57.947770+0000","last_undegraded":"2024-03-19T00:36:57.947770+0000","last_fullsized":"2024-03-19T00:36:57.947770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342057+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:58.342057+0000","last_peered":"2024-03-19T00:36:58.342057+0000","last_clean":"2024-03-19T00:36:58.342057+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:58.342057+0000","last_undegraded":"2024-03-19T00:36:58.342057+0000","last_fullsized":"2024-03-19T00:36:58.342057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945034+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:57.945034+0000","last_peered":"2024-03-19T00:36:57.945034+0000","last_clean":"2024-03-19T00:36:57.945034+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:57.945034+0000","last_undegraded":"2024-03-19T00:36:57.945034+0000","last_fullsized":"2024-03-19T00:36:57.945034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018317+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:36:58.018317+0000","last_peered":"2024-03-19T00:36:58.018317+0000","last_clean":"2024-03-19T00:36:58.018317+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:36:58.018317+0000","last_undegraded":"2024-03-19T00:36:58.018317+0000","last_fullsized":"2024-03-19T00:36:58.018317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945351+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:57.945351+0000","last_peered":"2024-03-19T00:36:57.945351+0000","last_clean":"2024-03-19T00:36:57.945351+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:57.945351+0000","last_undegraded":"2024-03-19T00:36:57.945351+0000","last_fullsized":"2024-03-19T00:36:57.945351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341835+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:58.341835+0000","last_peered":"2024-03-19T00:36:58.341835+0000","last_clean":"2024-03-19T00:36:58.341835+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:58.341835+0000","last_undegraded":"2024-03-19T00:36:58.341835+0000","last_fullsized":"2024-03-19T00:36:58.341835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092790+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:36:58.092790+0000","last_peered":"2024-03-19T00:36:58.092790+0000","last_clean":"2024-03-19T00:36:58.092790+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:36:58.092790+0000","last_undegraded":"2024-03-19T00:36:58.092790+0000","last_fullsized":"2024-03-19T00:36:58.092790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943691+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:57.943691+0000","last_peered":"2024-03-19T00:36:57.943691+0000","last_clean":"2024-03-19T00:36:57.943691+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:57.943691+0000","last_undegraded":"2024-03-19T00:36:57.943691+0000","last_fullsized":"2024-03-19T00:36:57.943691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:42.395840+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:37:42.395840+0000","last_peered":"2024-03-19T00:37:42.395840+0000","last_clean":"2024-03-19T00:37:42.395840+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:37:42.395840+0000","last_undegraded":"2024-03-19T00:37:42.395840+0000","last_fullsized":"2024-03-19T00:37:42.395840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092507+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:36:58.092507+0000","last_peered":"2024-03-19T00:36:58.092507+0000","last_clean":"2024-03-19T00:36:58.092507+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:36:58.092507+0000","last_undegraded":"2024-03-19T00:36:58.092507+0000","last_fullsized":"2024-03-19T00:36:58.092507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018782+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:58.018782+0000","last_peered":"2024-03-19T00:36:58.018782+0000","last_clean":"2024-03-19T00:36:58.018782+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:58.018782+0000","last_undegraded":"2024-03-19T00:36:58.018782+0000","last_fullsized":"2024-03-19T00:36:58.018782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947646+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:57.947646+0000","last_peered":"2024-03-19T00:36:57.947646+0000","last_clean":"2024-03-19T00:36:57.947646+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:57.947646+0000","last_undegraded":"2024-03-19T00:36:57.947646+0000","last_fullsized":"2024-03-19T00:36:57.947646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341807+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:36:58.341807+0000","last_peered":"2024-03-19T00:36:58.341807+0000","last_clean":"2024-03-19T00:36:58.341807+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:36:58.341807+0000","last_undegraded":"2024-03-19T00:36:58.341807+0000","last_fullsized":"2024-03-19T00:36:58.341807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341693+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:58.341693+0000","last_peered":"2024-03-19T00:36:58.341693+0000","last_clean":"2024-03-19T00:36:58.341693+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:58.341693+0000","last_undegraded":"2024-03-19T00:36:58.341693+0000","last_fullsized":"2024-03-19T00:36:58.341693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948726+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:57.948726+0000","last_peered":"2024-03-19T00:36:57.948726+0000","last_clean":"2024-03-19T00:36:57.948726+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:57.948726+0000","last_undegraded":"2024-03-19T00:36:57.948726+0000","last_fullsized":"2024-03-19T00:36:57.948726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946252+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:36:57.946252+0000","last_peered":"2024-03-19T00:36:57.946252+0000","last_clean":"2024-03-19T00:36:57.946252+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:36:57.946252+0000","last_undegraded":"2024-03-19T00:36:57.946252+0000","last_fullsized":"2024-03-19T00:36:57.946252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947846+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:57.947846+0000","last_peered":"2024-03-19T00:36:57.947846+0000","last_clean":"2024-03-19T00:36:57.947846+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:57.947846+0000","last_undegraded":"2024-03-19T00:36:57.947846+0000","last_fullsized":"2024-03-19T00:36:57.947846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943503+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:57.943503+0000","last_peered":"2024-03-19T00:36:57.943503+0000","last_clean":"2024-03-19T00:36:57.943503+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:57.943503+0000","last_undegraded":"2024-03-19T00:36:57.943503+0000","last_fullsized":"2024-03-19T00:36:57.943503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297221+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:36:58.297221+0000","last_peered":"2024-03-19T00:36:58.297221+0000","last_clean":"2024-03-19T00:36:58.297221+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:36:58.297221+0000","last_undegraded":"2024-03-19T00:36:58.297221+0000","last_fullsized":"2024-03-19T00:36:58.297221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018617+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:58.018617+0000","last_peered":"2024-03-19T00:36:58.018617+0000","last_clean":"2024-03-19T00:36:58.018617+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:58.018617+0000","last_undegraded":"2024-03-19T00:36:58.018617+0000","last_fullsized":"2024-03-19T00:36:58.018617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092520+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:58.092520+0000","last_peered":"2024-03-19T00:36:58.092520+0000","last_clean":"2024-03-19T00:36:58.092520+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:58.092520+0000","last_undegraded":"2024-03-19T00:36:58.092520+0000","last_fullsized":"2024-03-19T00:36:58.092520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297222+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:36:58.297222+0000","last_peered":"2024-03-19T00:36:58.297222+0000","last_clean":"2024-03-19T00:36:58.297222+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:36:58.297222+0000","last_undegraded":"2024-03-19T00:36:58.297222+0000","last_fullsized":"2024-03-19T00:36:58.297222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018710+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:58.018710+0000","last_peered":"2024-03-19T00:36:58.018710+0000","last_clean":"2024-03-19T00:36:58.018710+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:58.018710+0000","last_undegraded":"2024-03-19T00:36:58.018710+0000","last_fullsized":"2024-03-19T00:36:58.018710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341417+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:36:58.341417+0000","last_peered":"2024-03-19T00:36:58.341417+0000","last_clean":"2024-03-19T00:36:58.341417+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:36:58.341417+0000","last_undegraded":"2024-03-19T00:36:58.341417+0000","last_fullsized":"2024-03-19T00:36:58.341417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:38.390275+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:37:38.390275+0000","last_peered":"2024-03-19T00:37:38.390275+0000","last_clean":"2024-03-19T00:37:38.390275+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:37:38.390275+0000","last_undegraded":"2024-03-19T00:37:38.390275+0000","last_fullsized":"2024-03-19T00:37:38.390275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092809+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:58.092809+0000","last_peered":"2024-03-19T00:36:58.092809+0000","last_clean":"2024-03-19T00:36:58.092809+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:58.092809+0000","last_undegraded":"2024-03-19T00:36:58.092809+0000","last_fullsized":"2024-03-19T00:36:58.092809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946398+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:36:57.946398+0000","last_peered":"2024-03-19T00:36:57.946398+0000","last_clean":"2024-03-19T00:36:57.946398+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:36:57.946398+0000","last_undegraded":"2024-03-19T00:36:57.946398+0000","last_fullsized":"2024-03-19T00:36:57.946398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943975+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:57.943975+0000","last_peered":"2024-03-19T00:36:57.943975+0000","last_clean":"2024-03-19T00:36:57.943975+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:57.943975+0000","last_undegraded":"2024-03-19T00:36:57.943975+0000","last_fullsized":"2024-03-19T00:36:57.943975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092415+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:58.092415+0000","last_peered":"2024-03-19T00:36:58.092415+0000","last_clean":"2024-03-19T00:36:58.092415+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:58.092415+0000","last_undegraded":"2024-03-19T00:36:58.092415+0000","last_fullsized":"2024-03-19T00:36:58.092415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341960+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:58.341960+0000","last_peered":"2024-03-19T00:36:58.341960+0000","last_clean":"2024-03-19T00:36:58.341960+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:58.341960+0000","last_undegraded":"2024-03-19T00:36:58.341960+0000","last_fullsized":"2024-03-19T00:36:58.341960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018094+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:36:58.018094+0000","last_peered":"2024-03-19T00:36:58.018094+0000","last_clean":"2024-03-19T00:36:58.018094+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:36:58.018094+0000","last_undegraded":"2024-03-19T00:36:58.018094+0000","last_fullsized":"2024-03-19T00:36:58.018094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092437+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:58.092437+0000","last_peered":"2024-03-19T00:36:58.092437+0000","last_clean":"2024-03-19T00:36:58.092437+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:58.092437+0000","last_undegraded":"2024-03-19T00:36:58.092437+0000","last_fullsized":"2024-03-19T00:36:58.092437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945763+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:57.945763+0000","last_peered":"2024-03-19T00:36:57.945763+0000","last_clean":"2024-03-19T00:36:57.945763+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:57.945763+0000","last_undegraded":"2024-03-19T00:36:57.945763+0000","last_fullsized":"2024-03-19T00:36:57.945763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947645+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:57.947645+0000","last_peered":"2024-03-19T00:36:57.947645+0000","last_clean":"2024-03-19T00:36:57.947645+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:57.947645+0000","last_undegraded":"2024-03-19T00:36:57.947645+0000","last_fullsized":"2024-03-19T00:36:57.947645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018762+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:36:58.018762+0000","last_peered":"2024-03-19T00:36:58.018762+0000","last_clean":"2024-03-19T00:36:58.018762+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:36:58.018762+0000","last_undegraded":"2024-03-19T00:36:58.018762+0000","last_fullsized":"2024-03-19T00:36:58.018762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946042+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:57.946042+0000","last_peered":"2024-03-19T00:36:57.946042+0000","last_clean":"2024-03-19T00:36:57.946042+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:57.946042+0000","last_undegraded":"2024-03-19T00:36:57.946042+0000","last_fullsized":"2024-03-19T00:36:57.946042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.954788+0000","last_change":"2024-03-19T00:36:57.954788+0000","last_active":"2024-03-19T00:36:57.954788+0000","last_peered":"2024-03-19T00:36:57.954788+0000","last_clean":"2024-03-19T00:36:57.954788+0000","last_became_active":"2024-03-19T00:36:57.949916+0000","last_became_peered":"2024-03-19T00:36:57.949916+0000","last_unstale":"2024-03-19T00:36:57.954788+0000","last_undegraded":"2024-03-19T00:36:57.954788+0000","last_fullsized":"2024-03-19T00:36:57.954788+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:09.959915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946940+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:36:57.946940+0000","last_peered":"2024-03-19T00:36:57.946940+0000","last_clean":"2024-03-19T00:36:57.946940+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:36:57.946940+0000","last_undegraded":"2024-03-19T00:36:57.946940+0000","last_fullsized":"2024-03-19T00:36:57.946940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944199+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:57.944199+0000","last_peered":"2024-03-19T00:36:57.944199+0000","last_clean":"2024-03-19T00:36:57.944199+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:57.944199+0000","last_undegraded":"2024-03-19T00:36:57.944199+0000","last_fullsized":"2024-03-19T00:36:57.944199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945111+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:57.945111+0000","last_peered":"2024-03-19T00:36:57.945111+0000","last_clean":"2024-03-19T00:36:57.945111+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:57.945111+0000","last_undegraded":"2024-03-19T00:36:57.945111+0000","last_fullsized":"2024-03-19T00:36:57.945111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947426+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:36:57.947426+0000","last_peered":"2024-03-19T00:36:57.947426+0000","last_clean":"2024-03-19T00:36:57.947426+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:36:57.947426+0000","last_undegraded":"2024-03-19T00:36:57.947426+0000","last_fullsized":"2024-03-19T00:36:57.947426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342763+0000","last_change":"2024-03-19T00:36:58.342763+0000","last_active":"2024-03-19T00:36:58.342763+0000","last_peered":"2024-03-19T00:36:58.342763+0000","last_clean":"2024-03-19T00:36:58.342763+0000","last_became_active":"2024-03-19T00:36:58.342305+0000","last_became_peered":"2024-03-19T00:36:58.342305+0000","last_unstale":"2024-03-19T00:36:58.342763+0000","last_undegraded":"2024-03-19T00:36:58.342763+0000","last_fullsized":"2024-03-19T00:36:58.342763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:44.312347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.952182+0000","last_change":"2024-03-19T00:36:57.952182+0000","last_active":"2024-03-19T00:36:57.952182+0000","last_peered":"2024-03-19T00:36:57.952182+0000","last_clean":"2024-03-19T00:36:57.952182+0000","last_became_active":"2024-03-19T00:36:57.948822+0000","last_became_peered":"2024-03-19T00:36:57.948822+0000","last_unstale":"2024-03-19T00:36:57.952182+0000","last_undegraded":"2024-03-19T00:36:57.952182+0000","last_fullsized":"2024-03-19T00:36:57.952182+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.527748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947418+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:36:57.947418+0000","last_peered":"2024-03-19T00:36:57.947418+0000","last_clean":"2024-03-19T00:36:57.947418+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:36:57.947418+0000","last_undegraded":"2024-03-19T00:36:57.947418+0000","last_fullsized":"2024-03-19T00:36:57.947418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943795+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:57.943795+0000","last_peered":"2024-03-19T00:36:57.943795+0000","last_clean":"2024-03-19T00:36:57.943795+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:57.943795+0000","last_undegraded":"2024-03-19T00:36:57.943795+0000","last_fullsized":"2024-03-19T00:36:57.943795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946295+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:57.946295+0000","last_peered":"2024-03-19T00:36:57.946295+0000","last_clean":"2024-03-19T00:36:57.946295+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:57.946295+0000","last_undegraded":"2024-03-19T00:36:57.946295+0000","last_fullsized":"2024-03-19T00:36:57.946295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945488+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:36:57.945488+0000","last_peered":"2024-03-19T00:36:57.945488+0000","last_clean":"2024-03-19T00:36:57.945488+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:36:57.945488+0000","last_undegraded":"2024-03-19T00:36:57.945488+0000","last_fullsized":"2024-03-19T00:36:57.945488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944664+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:57.944664+0000","last_peered":"2024-03-19T00:36:57.944664+0000","last_clean":"2024-03-19T00:36:57.944664+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:57.944664+0000","last_undegraded":"2024-03-19T00:36:57.944664+0000","last_fullsized":"2024-03-19T00:36:57.944664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944531+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:36:57.944531+0000","last_peered":"2024-03-19T00:36:57.944531+0000","last_clean":"2024-03-19T00:36:57.944531+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:36:57.944531+0000","last_undegraded":"2024-03-19T00:36:57.944531+0000","last_fullsized":"2024-03-19T00:36:57.944531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946473+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:36:57.946473+0000","last_peered":"2024-03-19T00:36:57.946473+0000","last_clean":"2024-03-19T00:36:57.946473+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:36:57.946473+0000","last_undegraded":"2024-03-19T00:36:57.946473+0000","last_fullsized":"2024-03-19T00:36:57.946473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944016+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:57.944016+0000","last_peered":"2024-03-19T00:36:57.944016+0000","last_clean":"2024-03-19T00:36:57.944016+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:57.944016+0000","last_undegraded":"2024-03-19T00:36:57.944016+0000","last_fullsized":"2024-03-19T00:36:57.944016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947670+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:57.947670+0000","last_peered":"2024-03-19T00:36:57.947670+0000","last_clean":"2024-03-19T00:36:57.947670+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:57.947670+0000","last_undegraded":"2024-03-19T00:36:57.947670+0000","last_fullsized":"2024-03-19T00:36:57.947670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092661+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:36:58.092661+0000","last_peered":"2024-03-19T00:36:58.092661+0000","last_clean":"2024-03-19T00:36:58.092661+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:36:58.092661+0000","last_undegraded":"2024-03-19T00:36:58.092661+0000","last_fullsized":"2024-03-19T00:36:58.092661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946161+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:57.946161+0000","last_peered":"2024-03-19T00:36:57.946161+0000","last_clean":"2024-03-19T00:36:57.946161+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:57.946161+0000","last_undegraded":"2024-03-19T00:36:57.946161+0000","last_fullsized":"2024-03-19T00:36:57.946161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944946+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:57.944946+0000","last_peered":"2024-03-19T00:36:57.944946+0000","last_clean":"2024-03-19T00:36:57.944946+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:57.944946+0000","last_undegraded":"2024-03-19T00:36:57.944946+0000","last_fullsized":"2024-03-19T00:36:57.944946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.093141+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:36:58.093141+0000","last_peered":"2024-03-19T00:36:58.093141+0000","last_clean":"2024-03-19T00:36:58.093141+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:36:58.093141+0000","last_undegraded":"2024-03-19T00:36:58.093141+0000","last_fullsized":"2024-03-19T00:36:58.093141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947166+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:57.947166+0000","last_peered":"2024-03-19T00:36:57.947166+0000","last_clean":"2024-03-19T00:36:57.947166+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:57.947166+0000","last_undegraded":"2024-03-19T00:36:57.947166+0000","last_fullsized":"2024-03-19T00:36:57.947166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341940+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:58.341940+0000","last_peered":"2024-03-19T00:36:58.341940+0000","last_clean":"2024-03-19T00:36:58.341940+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:58.341940+0000","last_undegraded":"2024-03-19T00:36:58.341940+0000","last_fullsized":"2024-03-19T00:36:58.341940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092594+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:36:58.092594+0000","last_peered":"2024-03-19T00:36:58.092594+0000","last_clean":"2024-03-19T00:36:58.092594+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:36:58.092594+0000","last_undegraded":"2024-03-19T00:36:58.092594+0000","last_fullsized":"2024-03-19T00:36:58.092594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341207+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:58.341207+0000","last_peered":"2024-03-19T00:36:58.341207+0000","last_clean":"2024-03-19T00:36:58.341207+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:58.341207+0000","last_undegraded":"2024-03-19T00:36:58.341207+0000","last_fullsized":"2024-03-19T00:36:58.341207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018745+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:58.018745+0000","last_peered":"2024-03-19T00:36:58.018745+0000","last_clean":"2024-03-19T00:36:58.018745+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:58.018745+0000","last_undegraded":"2024-03-19T00:36:58.018745+0000","last_fullsized":"2024-03-19T00:36:58.018745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092947+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:36:58.092947+0000","last_peered":"2024-03-19T00:36:58.092947+0000","last_clean":"2024-03-19T00:36:58.092947+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:36:58.092947+0000","last_undegraded":"2024-03-19T00:36:58.092947+0000","last_fullsized":"2024-03-19T00:36:58.092947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945517+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:36:57.945517+0000","last_peered":"2024-03-19T00:36:57.945517+0000","last_clean":"2024-03-19T00:36:57.945517+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:36:57.945517+0000","last_undegraded":"2024-03-19T00:36:57.945517+0000","last_fullsized":"2024-03-19T00:36:57.945517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944702+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:36:57.944702+0000","last_peered":"2024-03-19T00:36:57.944702+0000","last_clean":"2024-03-19T00:36:57.944702+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:36:57.944702+0000","last_undegraded":"2024-03-19T00:36:57.944702+0000","last_fullsized":"2024-03-19T00:36:57.944702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944435+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:36:57.944435+0000","last_peered":"2024-03-19T00:36:57.944435+0000","last_clean":"2024-03-19T00:36:57.944435+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:36:57.944435+0000","last_undegraded":"2024-03-19T00:36:57.944435+0000","last_fullsized":"2024-03-19T00:36:57.944435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.019535+0000","last_change":"2024-03-19T00:36:58.019535+0000","last_active":"2024-03-19T00:36:58.019535+0000","last_peered":"2024-03-19T00:36:58.019535+0000","last_clean":"2024-03-19T00:36:58.019535+0000","last_became_active":"2024-03-19T00:36:58.019220+0000","last_became_peered":"2024-03-19T00:36:58.019220+0000","last_unstale":"2024-03-19T00:36:58.019535+0000","last_undegraded":"2024-03-19T00:36:58.019535+0000","last_fullsized":"2024-03-19T00:36:58.019535+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:04:51.797603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018726+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:36:58.018726+0000","last_peered":"2024-03-19T00:36:58.018726+0000","last_clean":"2024-03-19T00:36:58.018726+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:36:58.018726+0000","last_undegraded":"2024-03-19T00:36:58.018726+0000","last_fullsized":"2024-03-19T00:36:58.018726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945683+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:36:57.945683+0000","last_peered":"2024-03-19T00:36:57.945683+0000","last_clean":"2024-03-19T00:36:57.945683+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:36:57.945683+0000","last_undegraded":"2024-03-19T00:36:57.945683+0000","last_fullsized":"2024-03-19T00:36:57.945683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944912+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:36:57.944912+0000","last_peered":"2024-03-19T00:36:57.944912+0000","last_clean":"2024-03-19T00:36:57.944912+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:36:57.944912+0000","last_undegraded":"2024-03-19T00:36:57.944912+0000","last_fullsized":"2024-03-19T00:36:57.944912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947092+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:36:57.947092+0000","last_peered":"2024-03-19T00:36:57.947092+0000","last_clean":"2024-03-19T00:36:57.947092+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:36:57.947092+0000","last_undegraded":"2024-03-19T00:36:57.947092+0000","last_fullsized":"2024-03-19T00:36:57.947092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341314+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:36:58.341314+0000","last_peered":"2024-03-19T00:36:58.341314+0000","last_clean":"2024-03-19T00:36:58.341314+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:36:58.341314+0000","last_undegraded":"2024-03-19T00:36:58.341314+0000","last_fullsized":"2024-03-19T00:36:58.341314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946198+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:36:57.946198+0000","last_peered":"2024-03-19T00:36:57.946198+0000","last_clean":"2024-03-19T00:36:57.946198+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:36:57.946198+0000","last_undegraded":"2024-03-19T00:36:57.946198+0000","last_fullsized":"2024-03-19T00:36:57.946198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946433+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:36:57.946433+0000","last_peered":"2024-03-19T00:36:57.946433+0000","last_clean":"2024-03-19T00:36:57.946433+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:36:57.946433+0000","last_undegraded":"2024-03-19T00:36:57.946433+0000","last_fullsized":"2024-03-19T00:36:57.946433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018580+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:36:58.018580+0000","last_peered":"2024-03-19T00:36:58.018580+0000","last_clean":"2024-03-19T00:36:58.018580+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:36:58.018580+0000","last_undegraded":"2024-03-19T00:36:58.018580+0000","last_fullsized":"2024-03-19T00:36:58.018580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.951256+0000","last_change":"2024-03-19T00:36:57.951256+0000","last_active":"2024-03-19T00:36:57.951256+0000","last_peered":"2024-03-19T00:36:57.951256+0000","last_clean":"2024-03-19T00:36:57.951256+0000","last_became_active":"2024-03-19T00:36:57.950879+0000","last_became_peered":"2024-03-19T00:36:57.950879+0000","last_unstale":"2024-03-19T00:36:57.951256+0000","last_undegraded":"2024-03-19T00:36:57.951256+0000","last_fullsized":"2024-03-19T00:36:57.951256+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:26.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67749,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":492,"num_read_kb":316,"num_write":162,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":203247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1151831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":500971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1634304,"data_stored":1097271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":559494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":561235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":496421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":501268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1642496,"data_stored":1092468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:37:44.999 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-19T00:37:44.999 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-19T00:37:44.999 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-19T00:37:45.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-19T00:37:45.252 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:45 smithi007 ceph-mon[62854]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:45 smithi007 ceph-mon[62854]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:37:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:45 smithi044 ceph-mon[66366]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:45 smithi044 ceph-mon[66366]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:37:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:45 smithi204 ceph-mon[66572]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:45 smithi204 ceph-mon[66572]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 -- 172.21.15.7:0/699899599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc102d20 msgr2=0x7efbfc10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 --2- 172.21.15.7:0/699899599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc102d20 0x7efbfc10f160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b0d0 tx=0x7efbf400b3e0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 -- 172.21.15.7:0/699899599 shutdown_connections 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 --2- 172.21.15.7:0/699899599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc102d20 0x7efbfc10f160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 --2- 172.21.15.7:0/699899599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc108340 0x7efbfc108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 --2- 172.21.15.7:0/699899599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbfc102360 0x7efbfc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 -- 172.21.15.7:0/699899599 >> 172.21.15.7:0/699899599 conn(0x7efbfc079440 msgr2=0x7efbfc079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 -- 172.21.15.7:0/699899599 shutdown_connections 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.002+0000 7efc01965700 1 -- 172.21.15.7:0/699899599 wait complete. 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 Processor -- start 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 -- start start 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc102360 0x7efbfc074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbfc102d20 0x7efbfc074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 0x7efbfc075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc112470 con 0x7efbfc108340 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1122f0 con 0x7efbfc102360 2024-03-19T00:37:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efc01965700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1125f0 con 0x7efbfc102d20 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efbfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 0x7efbfc075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efbfaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc102360 0x7efbfc074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efbfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 0x7efbfc075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50282/0 (socket says 172.21.15.7:50282) 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efbfb7fe700 1 -- 172.21.15.7:0/3638514682 learned_addr learned my addr 172.21.15.7:0/3638514682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.003+0000 7efbfa7fc700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbfc102d20 0x7efbfc074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfaffd700 1 -- 172.21.15.7:0/3638514682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbfc102d20 msgr2=0x7efbfc074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfaffd700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbfc102d20 0x7efbfc074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfaffd700 1 -- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 msgr2=0x7efbfc075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfaffd700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 0x7efbfc075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfaffd700 1 -- 172.21.15.7:0/3638514682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc1a6e90 con 0x7efbfc102360 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfb7fe700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 0x7efbfc075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efbfaffd700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc102360 0x7efbfc074610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efbec00e9f0 tx=0x7efbec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec00c9c0 con 0x7efbfc102360 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbec00cb60 con 0x7efbfc102360 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc1a7180 con 0x7efbfc102360 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec018680 con 0x7efbfc102360 2024-03-19T00:37:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.004+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1a7710 con 0x7efbfc102360 2024-03-19T00:37:47.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.005+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc068fd0 con 0x7efbfc102360 2024-03-19T00:37:47.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.006+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7efbec010430 con 0x7efbfc102360 2024-03-19T00:37:47.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.006+0000 7efc00963700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efbe40a29a0 0x7efbe40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:47.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.007+0000 7efbfa7fc700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efbe40a29a0 0x7efbe40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:47.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.007+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efbec0cefd0 con 0x7efbfc102360 2024-03-19T00:37:47.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.007+0000 7efbfa7fc700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efbe40a29a0 0x7efbe40a4e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efbfc079b00 tx=0x7efbf00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:47.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.013+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efbec09bb50 con 0x7efbfc102360 2024-03-19T00:37:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.265+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efbfc02dbb0 con 0x7efbfc102360 2024-03-19T00:37:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.266+0000 7efc00963700 1 -- 172.21.15.7:0/3638514682 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efbec09b450 con 0x7efbfc102360 2024-03-19T00:37:47.267 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:47.267 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efbe40a29a0 msgr2=0x7efbe40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efbe40a29a0 0x7efbe40a4e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efbfc079b00 tx=0x7efbf00073d0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc102360 msgr2=0x7efbfc074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc102360 0x7efbfc074610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efbec00e9f0 tx=0x7efbec00ed00 comp rx=0 tx=0).stop 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 shutdown_connections 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efbe40a29a0 0x7efbe40a4e60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc108340 0x7efbfc075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbfc102360 0x7efbfc074610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 --2- 172.21.15.7:0/3638514682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efbfc102d20 0x7efbfc074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 >> 172.21.15.7:0/3638514682 conn(0x7efbfc079440 msgr2=0x7efbfc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 shutdown_connections 2024-03-19T00:37:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:47.268+0000 7efc01965700 1 -- 172.21.15.7:0/3638514682 wait complete. 2024-03-19T00:37:47.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:47 smithi007 ceph-mon[62854]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:47.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:47 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3638514682' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T00:37:48.060 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-19T00:37:48.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:37:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:47 smithi044 ceph-mon[66366]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:47 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3638514682' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 -- 172.21.15.7:0/3306255858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 msgr2=0x7f2144100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 --2- 172.21.15.7:0/3306255858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f2144100b70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f213c009900 tx=0x7f213c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 -- 172.21.15.7:0/3306255858 shutdown_connections 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 --2- 172.21.15.7:0/3306255858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2144102300 0x7f2144108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 --2- 172.21.15.7:0/3306255858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101940 0x7f2144101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 --2- 172.21.15.7:0/3306255858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f2144100b70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 -- 172.21.15.7:0/3306255858 >> 172.21.15.7:0/3306255858 conn(0x7f21440fbcc0 msgr2=0x7f21440fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 -- 172.21.15.7:0/3306255858 shutdown_connections 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 -- 172.21.15.7:0/3306255858 wait complete. 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 Processor -- start 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.169+0000 7f214c17e640 1 -- start start 2024-03-19T00:37:48.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f21441969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101940 0x7f2144196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2144102300 0x7f214419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f214410c270 con 0x7f2144100750 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f214410c0f0 con 0x7f2144102300 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f214410c3f0 con 0x7f2144101940 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f21441969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f21441969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50302/0 (socket says 172.21.15.7:50302) 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 -- 172.21.15.7:0/3337337275 learned_addr learned my addr 172.21.15.7:0/3337337275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149719640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101940 0x7f2144196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214a71b640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2144102300 0x7f214419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 -- 172.21.15.7:0/3337337275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101940 msgr2=0x7f2144196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101940 0x7f2144196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 -- 172.21.15.7:0/3337337275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2144102300 msgr2=0x7f214419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2144102300 0x7f214419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 -- 172.21.15.7:0/3337337275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214419e5c0 con 0x7f2144100750 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2149f1a640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f21441969a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c004e00 con 0x7f2144100750 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f213c005450 con 0x7f2144100750 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c047480 con 0x7f2144100750 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2144068af0 con 0x7f2144100750 2024-03-19T00:37:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.170+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2144069040 con 0x7f2144100750 2024-03-19T00:37:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.171+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21440686a0 con 0x7f2144100750 2024-03-19T00:37:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.173+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f213c003710 con 0x7f2144100750 2024-03-19T00:37:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.173+0000 7f2142ffd640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f212c0a2890 0x7f212c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.173+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f213c0f3390 con 0x7f2144100750 2024-03-19T00:37:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.173+0000 7f2149719640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f212c0a2890 0x7f212c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.174+0000 7f2149719640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f212c0a2890 0x7f212c0a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2144197eb0 tx=0x7f2138005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.177+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f213c0c0080 con 0x7f2144100750 2024-03-19T00:37:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:47 smithi204 ceph-mon[66572]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:47 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3638514682' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T00:37:48.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.366+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21441075d0 con 0x7f2144100750 2024-03-19T00:37:48.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.368+0000 7f2142ffd640 1 -- 172.21.15.7:0/3337337275 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f213c0bf980 con 0x7f2144100750 2024-03-19T00:37:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:48.369 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f212c0a2890 msgr2=0x7f212c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f212c0a2890 0x7f212c0a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2144197eb0 tx=0x7f2138005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 msgr2=0x7f21441969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f21441969a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 shutdown_connections 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f212c0a2890 0x7f212c0a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2144102300 0x7f214419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2144101940 0x7f2144196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 --2- 172.21.15.7:0/3337337275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2144100750 0x7f21441969a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 >> 172.21.15.7:0/3337337275 conn(0x7f21440fbcc0 msgr2=0x7f2144104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:48.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 shutdown_connections 2024-03-19T00:37:48.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.370+0000 7f214c17e640 1 -- 172.21.15.7:0/3337337275 wait complete. 2024-03-19T00:37:48.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:37:48.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:37:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 -- 172.21.15.7:0/2810825139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050103420 msgr2=0x7fa050109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 --2- 172.21.15.7:0/2810825139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050103420 0x7fa050109cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa040009900 tx=0x7fa04002bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 -- 172.21.15.7:0/2810825139 shutdown_connections 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 --2- 172.21.15.7:0/2810825139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050103420 0x7fa050109cb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 --2- 172.21.15.7:0/2810825139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050102a60 0x7fa050102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 --2- 172.21.15.7:0/2810825139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa050101870 0x7fa050101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.487+0000 7fa0572dc640 1 -- 172.21.15.7:0/2810825139 >> 172.21.15.7:0/2810825139 conn(0x7fa0500fcde0 msgr2=0x7fa0500ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 -- 172.21.15.7:0/2810825139 shutdown_connections 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 -- 172.21.15.7:0/2810825139 wait complete. 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 Processor -- start 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 -- start start 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050101870 0x7fa050197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa050102a60 0x7fa050197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 0x7fa05019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010d3a0 con 0x7fa050103420 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010d220 con 0x7fa050101870 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa0572dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010d520 con 0x7fa050102a60 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa055879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 0x7fa05019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa055879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 0x7fa05019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50322/0 (socket says 172.21.15.7:50322) 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa055879640 1 -- 172.21.15.7:0/3499175908 learned_addr learned my addr 172.21.15.7:0/3499175908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa055078640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050101870 0x7fa050197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.488+0000 7fa054877640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa050102a60 0x7fa050197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055879640 1 -- 172.21.15.7:0/3499175908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa050102a60 msgr2=0x7fa050197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055879640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa050102a60 0x7fa050197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055879640 1 -- 172.21.15.7:0/3499175908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050101870 msgr2=0x7fa050197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055879640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050101870 0x7fa050197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055879640 1 -- 172.21.15.7:0/3499175908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05019f660 con 0x7fa050103420 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055078640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050101870 0x7fa050197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa055879640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 0x7fa05019eff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa040009230 tx=0x7fa040005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040043020 con 0x7fa050103420 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05019f8f0 con 0x7fa050103420 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0400043d0 con 0x7fa050103420 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040003b40 con 0x7fa050103420 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.489+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05019fe90 con 0x7fa050103420 2024-03-19T00:37:48.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.490+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa050108b90 con 0x7fa050103420 2024-03-19T00:37:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.491+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fa040046020 con 0x7fa050103420 2024-03-19T00:37:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.492+0000 7fa04e7fc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0380a2890 0x7fa0380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.492+0000 7fa055078640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0380a2890 0x7fa0380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.492+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa0400f1440 con 0x7fa050103420 2024-03-19T00:37:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.492+0000 7fa055078640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0380a2890 0x7fa0380a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0501028c0 tx=0x7fa048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.495+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa04004d020 con 0x7fa050103420 2024-03-19T00:37:48.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.681+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa05002d1f0 con 0x7fa050103420 2024-03-19T00:37:48.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.682+0000 7fa04e7fc640 1 -- 172.21.15.7:0/3499175908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa0400be130 con 0x7fa050103420 2024-03-19T00:37:48.683 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0380a2890 msgr2=0x7fa0380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0380a2890 0x7fa0380a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0501028c0 tx=0x7fa048005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 msgr2=0x7fa05019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 0x7fa05019eff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa040009230 tx=0x7fa040005e40 comp rx=0 tx=0).stop 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 shutdown_connections 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0380a2890 0x7fa0380a4d50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa050103420 0x7fa05019eff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa050102a60 0x7fa050197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 --2- 172.21.15.7:0/3499175908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa050101870 0x7fa050197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 >> 172.21.15.7:0/3499175908 conn(0x7fa0500fcde0 msgr2=0x7fa050105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:48.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 shutdown_connections 2024-03-19T00:37:48.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.684+0000 7fa0572dc640 1 -- 172.21.15.7:0/3499175908 wait complete. 2024-03-19T00:37:48.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/3179947051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b41096e0 msgr2=0x7ff0b410bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/3179947051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b41096e0 0x7ff0b410bb70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009ca0 tx=0x7ff0ac02c180 comp rx=0 tx=0).stop 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/3179947051 shutdown_connections 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/3179947051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b41096e0 0x7ff0b410bb70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/3179947051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b4074200 0x7ff0b4074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/3179947051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0b40757e0 0x7ff0b4073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/3179947051 >> 172.21.15.7:0/3179947051 conn(0x7ff0b40fcda0 msgr2=0x7ff0b40ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/3179947051 shutdown_connections 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.801+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/3179947051 wait complete. 2024-03-19T00:37:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 Processor -- start 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 -- start start 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 0x7ff0b4199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0b40757e0 0x7ff0b419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b41096e0 0x7ff0b41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410f550 con 0x7ff0b4074200 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410f3d0 con 0x7ff0b40757e0 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0ba0ea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410f6d0 con 0x7ff0b41096e0 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 0x7ff0b4199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 0x7ff0b4199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50340/0 (socket says 172.21.15.7:50340) 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 -- 172.21.15.7:0/323350940 learned_addr learned my addr 172.21.15.7:0/323350940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b2ffd640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0b40757e0 0x7ff0b419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b3fff640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b41096e0 0x7ff0b41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 -- 172.21.15.7:0/323350940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b41096e0 msgr2=0x7ff0b41a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b41096e0 0x7ff0b41a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 -- 172.21.15.7:0/323350940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0b40757e0 msgr2=0x7ff0b419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0b40757e0 0x7ff0b419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.802+0000 7ff0b37fe640 1 -- 172.21.15.7:0/323350940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b41a1800 con 0x7ff0b4074200 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.803+0000 7ff0b37fe640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 0x7ff0b4199be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9f0 tx=0x7ff0a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.803+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400cd80 con 0x7ff0b4074200 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.803+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b41a1af0 con 0x7ff0b4074200 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.803+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0a40108e0 con 0x7ff0b4074200 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.803+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4018920 con 0x7ff0b4074200 2024-03-19T00:37:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.803+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41a2090 con 0x7ff0b4074200 2024-03-19T00:37:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.804+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b41084a0 con 0x7ff0b4074200 2024-03-19T00:37:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.805+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7ff0a4010a90 con 0x7ff0b4074200 2024-03-19T00:37:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.805+0000 7ff0b0ff9640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff09c0a2890 0x7ff09c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.805+0000 7ff0b2ffd640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff09c0a2890 0x7ff09c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.805+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff0a40cf260 con 0x7ff0b4074200 2024-03-19T00:37:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.806+0000 7ff0b2ffd640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff09c0a2890 0x7ff09c0a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0b419b0f0 tx=0x7ff0a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.810+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff0a409b7f0 con 0x7ff0b4074200 2024-03-19T00:37:48.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:48 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3337337275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:48.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:48 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3499175908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:37:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.996+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff0b402d430 con 0x7ff0b4074200 2024-03-19T00:37:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.997+0000 7ff0b0ff9640 1 -- 172.21.15.7:0/323350940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ff0a409f890 con 0x7ff0b4074200 2024-03-19T00:37:48.997 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff09c0a2890 msgr2=0x7ff09c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff09c0a2890 0x7ff09c0a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0b419b0f0 tx=0x7ff0a8005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 msgr2=0x7ff0b4199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 0x7ff0b4199be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9f0 tx=0x7ff0a400eeb0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 shutdown_connections 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff09c0a2890 0x7ff09c0a4d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b41096e0 0x7ff0b41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0b40757e0 0x7ff0b419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 --2- 172.21.15.7:0/323350940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b4074200 0x7ff0b4199be0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 >> 172.21.15.7:0/323350940 conn(0x7ff0b40fcda0 msgr2=0x7ff0b4105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 shutdown_connections 2024-03-19T00:37:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:48.999+0000 7ff0ba0ea640 1 -- 172.21.15.7:0/323350940 wait complete. 2024-03-19T00:37:49.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:37:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3337337275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3499175908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 -- 172.21.15.7:0/3337704115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac1022f0 msgr2=0x7fe1ac108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 --2- 172.21.15.7:0/3337704115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac1022f0 0x7fe1ac108b80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009ca0 tx=0x7fe19c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 -- 172.21.15.7:0/3337704115 shutdown_connections 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 --2- 172.21.15.7:0/3337704115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac1022f0 0x7fe1ac108b80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 --2- 172.21.15.7:0/3337704115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac101930 0x7fe1ac101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 --2- 172.21.15.7:0/3337704115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1ac100740 0x7fe1ac100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 -- 172.21.15.7:0/3337704115 >> 172.21.15.7:0/3337704115 conn(0x7fe1ac0fbcd0 msgr2=0x7fe1ac0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 -- 172.21.15.7:0/3337704115 shutdown_connections 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 -- 172.21.15.7:0/3337704115 wait complete. 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 Processor -- start 2024-03-19T00:37:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.116+0000 7fe1b4661640 1 -- start start 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac100740 0x7fe1ac06aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 0x7fe1ac19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1ac1022f0 0x7fe1ac19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10c250 con 0x7fe1ac101930 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10c0d0 con 0x7fe1ac100740 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10c3d0 con 0x7fe1ac1022f0 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 0x7fe1ac19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 0x7fe1ac19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50356/0 (socket says 172.21.15.7:50356) 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 -- 172.21.15.7:0/1969298301 learned_addr learned my addr 172.21.15.7:0/1969298301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b23fd640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac100740 0x7fe1ac06aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b2bfe640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1ac1022f0 0x7fe1ac19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 -- 172.21.15.7:0/1969298301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1ac1022f0 msgr2=0x7fe1ac19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1ac1022f0 0x7fe1ac19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 -- 172.21.15.7:0/1969298301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac100740 msgr2=0x7fe1ac06aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac100740 0x7fe1ac06aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 -- 172.21.15.7:0/1969298301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac19f620 con 0x7fe1ac101930 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b23fd640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac100740 0x7fe1ac06aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b1bfc640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 0x7fe1ac19c720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000ea60 tx=0x7fe1a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000cde0 con 0x7fe1ac101930 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a00035f0 con 0x7fe1ac101930 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac19f910 con 0x7fe1ac101930 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0010690 con 0x7fe1ac101930 2024-03-19T00:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.117+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac19feb0 con 0x7fe1ac101930 2024-03-19T00:37:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.119+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac107ab0 con 0x7fe1ac101930 2024-03-19T00:37:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.120+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fe1a00037a0 con 0x7fe1ac101930 2024-03-19T00:37:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.120+0000 7fe1ab7fe640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe1940a2910 0x7fe1940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.120+0000 7fe1b23fd640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe1940a2910 0x7fe1940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.120+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe1a00cf130 con 0x7fe1ac101930 2024-03-19T00:37:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.121+0000 7fe1b23fd640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe1940a2910 0x7fe1940a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac101790 tx=0x7fe1a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.125+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe1a009bd10 con 0x7fe1ac101930 2024-03-19T00:37:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:48 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3337337275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:48 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3499175908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:37:49.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.309+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1ac02d3f0 con 0x7fe1ac101930 2024-03-19T00:37:49.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.309+0000 7fe1ab7fe640 1 -- 172.21.15.7:0/1969298301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fe1a009b610 con 0x7fe1ac101930 2024-03-19T00:37:49.310 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.311+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe1940a2910 msgr2=0x7fe1940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.311+0000 7fe1b4661640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe1940a2910 0x7fe1940a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac101790 tx=0x7fe1a4005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.311+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 msgr2=0x7fe1ac19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.311+0000 7fe1b4661640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 0x7fe1ac19c720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000ea60 tx=0x7fe1a000ef20 comp rx=0 tx=0).stop 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 shutdown_connections 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe1940a2910 0x7fe1940a4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1ac1022f0 0x7fe1ac19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac101930 0x7fe1ac19c720 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 --2- 172.21.15.7:0/1969298301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1ac100740 0x7fe1ac06aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 >> 172.21.15.7:0/1969298301 conn(0x7fe1ac0fbcd0 msgr2=0x7fe1ac104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 shutdown_connections 2024-03-19T00:37:49.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.312+0000 7fe1b4661640 1 -- 172.21.15.7:0/1969298301 wait complete. 2024-03-19T00:37:49.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:37:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 -- 172.21.15.7:0/1139976564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198100740 msgr2=0x7f2198100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 --2- 172.21.15.7:0/1139976564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198100740 0x7f2198100b60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2188009900 tx=0x7f218802bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 -- 172.21.15.7:0/1139976564 shutdown_connections 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 --2- 172.21.15.7:0/1139976564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f2198108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 --2- 172.21.15.7:0/1139976564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2198101930 0x7f2198101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 --2- 172.21.15.7:0/1139976564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198100740 0x7f2198100b60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 -- 172.21.15.7:0/1139976564 >> 172.21.15.7:0/1139976564 conn(0x7f21980fbcb0 msgr2=0x7f21980fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 -- 172.21.15.7:0/1139976564 shutdown_connections 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 -- 172.21.15.7:0/1139976564 wait complete. 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 Processor -- start 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.431+0000 7f219e827640 1 -- start start 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2198100740 0x7f21981969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198101930 0x7f2198196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f219819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f219810c240 con 0x7f21981022f0 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f219810c0c0 con 0x7f2198100740 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f219810c3c0 con 0x7f2198101930 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f219819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f219819df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50370/0 (socket says 172.21.15.7:50370) 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 -- 172.21.15.7:0/864093697 learned_addr learned my addr 172.21.15.7:0/864093697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f21977fe640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198101930 0x7f2198196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f2197fff640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2198100740 0x7f21981969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 -- 172.21.15.7:0/864093697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198101930 msgr2=0x7f2198196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198101930 0x7f2198196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 -- 172.21.15.7:0/864093697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2198100740 msgr2=0x7f21981969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2198100740 0x7f21981969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 -- 172.21.15.7:0/864093697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219819e5f0 con 0x7f21981022f0 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219cdc4640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f219819df80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f219000eb10 tx=0x7f219000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190004b20 con 0x7f21981022f0 2024-03-19T00:37:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.432+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2198068b50 con 0x7f21981022f0 2024-03-19T00:37:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.433+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2190004cc0 con 0x7f21981022f0 2024-03-19T00:37:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.433+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190010690 con 0x7f21981022f0 2024-03-19T00:37:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.433+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21980690a0 con 0x7f21981022f0 2024-03-19T00:37:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.434+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21980686a0 con 0x7f21981022f0 2024-03-19T00:37:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.435+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f2190005020 con 0x7f21981022f0 2024-03-19T00:37:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.435+0000 7f21957fa640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f21800a2890 0x7f21800a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.435+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f21900cf540 con 0x7f21981022f0 2024-03-19T00:37:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.435+0000 7f2197fff640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f21800a2890 0x7f21800a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.436+0000 7f2197fff640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f21800a2890 0x7f21800a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2188005ab0 tx=0x7f2188005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.440+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f219009c0c0 con 0x7f21981022f0 2024-03-19T00:37:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.627+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f219819e7c0 con 0x7f21981022f0 2024-03-19T00:37:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.628+0000 7f21957fa640 1 -- 172.21.15.7:0/864093697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f219009b9c0 con 0x7f21981022f0 2024-03-19T00:37:49.629 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:37:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f21800a2890 msgr2=0x7f21800a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f21800a2890 0x7f21800a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2188005ab0 tx=0x7f2188005fd0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 msgr2=0x7f219819df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f219819df80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f219000eb10 tx=0x7f219000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 shutdown_connections 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f21800a2890 0x7f21800a4d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21981022f0 0x7f219819df80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2198101930 0x7f2198196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 --2- 172.21.15.7:0/864093697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2198100740 0x7f21981969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 >> 172.21.15.7:0/864093697 conn(0x7f21980fbcb0 msgr2=0x7f2198104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 shutdown_connections 2024-03-19T00:37:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.630+0000 7f219e827640 1 -- 172.21.15.7:0/864093697 wait complete. 2024-03-19T00:37:49.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.746+0000 7fe613e3d640 1 -- 172.21.15.7:0/2520608700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c102360 msgr2=0x7fe60c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.746+0000 7fe613e3d640 1 --2- 172.21.15.7:0/2520608700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c102360 0x7fe60c104750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe608009860 tx=0x7fe60802bb70 comp rx=0 tx=0).stop 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.746+0000 7fe613e3d640 1 -- 172.21.15.7:0/2520608700 shutdown_connections 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 --2- 172.21.15.7:0/2520608700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe60c1075c0 0x7fe60c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 --2- 172.21.15.7:0/2520608700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 --2- 172.21.15.7:0/2520608700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c102360 0x7fe60c104750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- 172.21.15.7:0/2520608700 >> 172.21.15.7:0/2520608700 conn(0x7fe60c0fbcb0 msgr2=0x7fe60c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- 172.21.15.7:0/2520608700 shutdown_connections 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- 172.21.15.7:0/2520608700 wait complete. 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 Processor -- start 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- start start 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe60c102360 0x7fe60c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c1075c0 0x7fe60c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c10d380 con 0x7fe60c104c90 2024-03-19T00:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c10d200 con 0x7fe60c1075c0 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe613e3d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c10d500 con 0x7fe60c102360 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe6113d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe6113d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50396/0 (socket says 172.21.15.7:50396) 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.747+0000 7fe6113d8640 1 -- 172.21.15.7:0/719515749 learned_addr learned my addr 172.21.15.7:0/719515749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6123da640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c1075c0 0x7fe60c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe611bd9640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe60c102360 0x7fe60c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6113d8640 1 -- 172.21.15.7:0/719515749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe60c102360 msgr2=0x7fe60c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6113d8640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe60c102360 0x7fe60c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6113d8640 1 -- 172.21.15.7:0/719515749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c1075c0 msgr2=0x7fe60c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6113d8640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c1075c0 0x7fe60c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6113d8640 1 -- 172.21.15.7:0/719515749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c19f670 con 0x7fe60c104c90 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe6113d8640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c197f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00e9a0 tx=0x7fe5fc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00cd80 con 0x7fe60c104c90 2024-03-19T00:37:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c19f960 con 0x7fe60c104c90 2024-03-19T00:37:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5fc0035f0 con 0x7fe60c104c90 2024-03-19T00:37:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc010690 con 0x7fe60c104c90 2024-03-19T00:37:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.748+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c19feb0 con 0x7fe60c104c90 2024-03-19T00:37:49.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.749+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c0fd7a0 con 0x7fe60c104c90 2024-03-19T00:37:49.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.750+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fe5fc0037a0 con 0x7fe60c104c90 2024-03-19T00:37:49.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.751+0000 7fe602ffd640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe5f80a2890 0x7fe5f80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:49.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.751+0000 7fe611bd9640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe5f80a2890 0x7fe5f80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:49.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.751+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe5fc0cf280 con 0x7fe60c104c90 2024-03-19T00:37:49.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.752+0000 7fe611bd9640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe5f80a2890 0x7fe5f80a4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe608000c00 tx=0x7fe6080349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.755+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe5fc09bef0 con 0x7fe60c104c90 2024-03-19T00:37:49.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:49 smithi007 ceph-mon[62854]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:49 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/323350940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:37:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:49 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1969298301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:37:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:49 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/864093697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:37:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.979+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe60c02d440 con 0x7fe60c104c90 2024-03-19T00:37:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.980+0000 7fe602ffd640 1 -- 172.21.15.7:0/719515749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7fe5fc09b7f0 con 0x7fe60c104c90 2024-03-19T00:37:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:49.981 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:26.295874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:37:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe5f80a2890 msgr2=0x7fe5f80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe5f80a2890 0x7fe5f80a4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe608000c00 tx=0x7fe6080349a0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 msgr2=0x7fe60c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c197f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00e9a0 tx=0x7fe5fc00ee60 comp rx=0 tx=0).stop 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 shutdown_connections 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe5f80a2890 0x7fe5f80a4d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe60c1075c0 0x7fe60c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe60c104c90 0x7fe60c197f90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 --2- 172.21.15.7:0/719515749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe60c102360 0x7fe60c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 >> 172.21.15.7:0/719515749 conn(0x7fe60c0fbcb0 msgr2=0x7fe60c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 shutdown_connections 2024-03-19T00:37:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:49.982+0000 7fe613e3d640 1 -- 172.21.15.7:0/719515749 wait complete. 2024-03-19T00:37:49.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 -- 172.21.15.7:0/3426036832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20102aa0 msgr2=0x7f1f20102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 --2- 172.21.15.7:0/3426036832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20102aa0 0x7f1f20102f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009860 tx=0x7f1f1002bb70 comp rx=0 tx=0).stop 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 -- 172.21.15.7:0/3426036832 shutdown_connections 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 --2- 172.21.15.7:0/3426036832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20103460 0x7f1f20109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 --2- 172.21.15.7:0/3426036832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20102aa0 0x7f1f20102f20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 --2- 172.21.15.7:0/3426036832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 -- 172.21.15.7:0/3426036832 >> 172.21.15.7:0/3426036832 conn(0x7f1f200fce40 msgr2=0x7f1f200ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 -- 172.21.15.7:0/3426036832 shutdown_connections 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 -- 172.21.15.7:0/3426036832 wait complete. 2024-03-19T00:37:50.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 Processor -- start 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.099+0000 7f1f259e6640 1 -- start start 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f259e6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f259e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20102aa0 0x7f1f20197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f259e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20103460 0x7f1f2019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f259e6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010d350 con 0x7f1f201018b0 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f259e6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010d1d0 con 0x7f1f20103460 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f259e6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010d4d0 con 0x7f1f20102aa0 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50420/0 (socket says 172.21.15.7:50420) 2024-03-19T00:37:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1effd640 1 -- 172.21.15.7:0/2685575075 learned_addr learned my addr 172.21.15.7:0/2685575075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20102aa0 0x7f1f20197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1f7fe640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20103460 0x7f1f2019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 -- 172.21.15.7:0/2685575075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20103460 msgr2=0x7f1f2019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20103460 0x7f1f2019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 -- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 msgr2=0x7f1f20197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 -- 172.21.15.7:0/2685575075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2019f640 con 0x7f1f20102aa0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1f7fe640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20103460 0x7f1f2019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1effd640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.100+0000 7f1f1e7fc640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20102aa0 0x7f1f20197f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f10037040 tx=0x7f1f100040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.101+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10041020 con 0x7f1f20102aa0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.101+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2019f8d0 con 0x7f1f20102aa0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.101+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2019fe10 con 0x7f1f20102aa0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.101+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f100044d0 con 0x7f1f20102aa0 2024-03-19T00:37:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.101+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10039040 con 0x7f1f20102aa0 2024-03-19T00:37:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.102+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c0052c0 con 0x7f1f20102aa0 2024-03-19T00:37:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.103+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f1f10044020 con 0x7f1f20102aa0 2024-03-19T00:37:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.103+0000 7f1f249e4640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f080a2890 0x7f1f080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.103+0000 7f1f1effd640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f080a2890 0x7f1f080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.103+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1f100f09b0 con 0x7f1f20102aa0 2024-03-19T00:37:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.104+0000 7f1f1effd640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f080a2890 0x7f1f080a4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1f140096c0 tx=0x7f1f14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:49 smithi044 ceph-mon[66366]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:49 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/323350940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:49 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1969298301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:49 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/864093697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.111+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f1004b020 con 0x7f1f20102aa0 2024-03-19T00:37:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:49 smithi204 ceph-mon[66572]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:49 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/323350940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:49 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1969298301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:49 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/864093697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.298+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f0c005880 con 0x7f1f20102aa0 2024-03-19T00:37:50.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.299+0000 7f1f249e4640 1 -- 172.21.15.7:0/2685575075 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f1f100bd680 con 0x7f1f20102aa0 2024-03-19T00:37:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:50.301 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:37:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f080a2890 msgr2=0x7f1f080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f080a2890 0x7f1f080a4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1f140096c0 tx=0x7f1f14009260 comp rx=0 tx=0).stop 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20102aa0 msgr2=0x7f1f20197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20102aa0 0x7f1f20197f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f10037040 tx=0x7f1f100040e0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 shutdown_connections 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f080a2890 0x7f1f080a4d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20103460 0x7f1f2019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f20102aa0 0x7f1f20197f60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.301+0000 7f1f259e6640 1 --2- 172.21.15.7:0/2685575075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f201018b0 0x7f1f20197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.302+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 >> 172.21.15.7:0/2685575075 conn(0x7f1f200fce40 msgr2=0x7f1f20105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.302+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 shutdown_connections 2024-03-19T00:37:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.302+0000 7f1f259e6640 1 -- 172.21.15.7:0/2685575075 wait complete. 2024-03-19T00:37:50.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:37:50.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:37:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.420+0000 7f76cfff3640 1 -- 172.21.15.7:0/611067539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c81022f0 msgr2=0x7f76c8108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.420+0000 7f76cfff3640 1 --2- 172.21.15.7:0/611067539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c81022f0 0x7f76c8108b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009ca0 tx=0x7f76c002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- 172.21.15.7:0/611067539 shutdown_connections 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 --2- 172.21.15.7:0/611067539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c81022f0 0x7f76c8108b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 --2- 172.21.15.7:0/611067539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8101930 0x7f76c8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 --2- 172.21.15.7:0/611067539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c8100740 0x7f76c8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- 172.21.15.7:0/611067539 >> 172.21.15.7:0/611067539 conn(0x7f76c80fbcb0 msgr2=0x7f76c80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- 172.21.15.7:0/611067539 shutdown_connections 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- 172.21.15.7:0/611067539 wait complete. 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 Processor -- start 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- start start 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 0x7f76c8104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8101930 0x7f76c8102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 0x7f76c8103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810c250 con 0x7f76c81022f0 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810c0d0 con 0x7f76c8100740 2024-03-19T00:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cfff3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810c3d0 con 0x7f76c8101930 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76ce590640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 0x7f76c8103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cdd8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 0x7f76c8104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76ce590640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 0x7f76c8103250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50434/0 (socket says 172.21.15.7:50434) 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76ce590640 1 -- 172.21.15.7:0/4055679934 learned_addr learned my addr 172.21.15.7:0/4055679934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.421+0000 7f76cdd8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 0x7f76c8104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43050/0 (socket says 172.21.15.7:43050) 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76cd58e640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8101930 0x7f76c8102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76ce590640 1 -- 172.21.15.7:0/4055679934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8101930 msgr2=0x7f76c8102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76ce590640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8101930 0x7f76c8102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76ce590640 1 -- 172.21.15.7:0/4055679934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 msgr2=0x7f76c8104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76ce590640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 0x7f76c8104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76ce590640 1 -- 172.21.15.7:0/4055679934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8103b10 con 0x7f76c81022f0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76cdd8f640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 0x7f76c8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76ce590640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 0x7f76c8103250 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76c002c700 tx=0x7f76c00328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0041020 con 0x7f76c81022f0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76c0003e20 con 0x7f76c81022f0 2024-03-19T00:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c81a1b30 con 0x7f76c81022f0 2024-03-19T00:37:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0039040 con 0x7f76c81022f0 2024-03-19T00:37:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.422+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81a1ff0 con 0x7f76c81022f0 2024-03-19T00:37:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.423+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c81a21c0 con 0x7f76c81022f0 2024-03-19T00:37:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.424+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f76c0044020 con 0x7f76c81022f0 2024-03-19T00:37:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.425+0000 7f76baffd640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76b40a2890 0x7f76b40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.425+0000 7f76cdd8f640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76b40a2890 0x7f76b40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.425+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f76c00f13b0 con 0x7f76c81022f0 2024-03-19T00:37:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.426+0000 7f76cdd8f640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76b40a2890 0x7f76b40a4d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76c8101790 tx=0x7f76bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:50.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.429+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76c004b020 con 0x7f76c81022f0 2024-03-19T00:37:50.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.626+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76c81a2550 con 0x7f76c81022f0 2024-03-19T00:37:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.627+0000 7f76baffd640 1 -- 172.21.15.7:0/4055679934 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f76c00be0e0 con 0x7f76c81022f0 2024-03-19T00:37:50.628 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76b40a2890 msgr2=0x7f76b40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76b40a2890 0x7f76b40a4d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76c8101790 tx=0x7f76bc005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 msgr2=0x7f76c8103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 0x7f76c8103250 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76c002c700 tx=0x7f76c00328a0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 shutdown_connections 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76b40a2890 0x7f76b40a4d50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81022f0 0x7f76c8103250 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76c8101930 0x7f76c8102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 --2- 172.21.15.7:0/4055679934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8100740 0x7f76c8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 >> 172.21.15.7:0/4055679934 conn(0x7f76c80fbcb0 msgr2=0x7f76c80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 shutdown_connections 2024-03-19T00:37:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.629+0000 7f76cfff3640 1 -- 172.21.15.7:0/4055679934 wait complete. 2024-03-19T00:37:50.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:37:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 -- 172.21.15.7:0/1711266558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d41022e0 msgr2=0x7fb3d4108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 --2- 172.21.15.7:0/1711266558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d41022e0 0x7fb3d4108b70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009be0 tx=0x7fb3cc02bef0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 -- 172.21.15.7:0/1711266558 shutdown_connections 2024-03-19T00:37:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 --2- 172.21.15.7:0/1711266558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d41022e0 0x7fb3d4108b70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 --2- 172.21.15.7:0/1711266558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4101920 0x7fb3d4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 --2- 172.21.15.7:0/1711266558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d4100730 0x7fb3d4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 -- 172.21.15.7:0/1711266558 >> 172.21.15.7:0/1711266558 conn(0x7fb3d40fbc80 msgr2=0x7fb3d40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 -- 172.21.15.7:0/1711266558 shutdown_connections 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.745+0000 7fb3da9df640 1 -- 172.21.15.7:0/1711266558 wait complete. 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 Processor -- start 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 -- start start 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 0x7fb3d4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d4101920 0x7fb3d4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d41022e0 0x7fb3d419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d410d380 con 0x7fb3d4100730 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d410d200 con 0x7fb3d4101920 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3da9df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d410d500 con 0x7fb3d41022e0 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 0x7fb3d4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 0x7fb3d4197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50438/0 (socket says 172.21.15.7:50438) 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 -- 172.21.15.7:0/3548415121 learned_addr learned my addr 172.21.15.7:0/3548415121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d37fe640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d4101920 0x7fb3d4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d8f7c640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d41022e0 0x7fb3d419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 -- 172.21.15.7:0/3548415121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d41022e0 msgr2=0x7fb3d419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d41022e0 0x7fb3d419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 -- 172.21.15.7:0/3548415121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d4101920 msgr2=0x7fb3d4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d4101920 0x7fb3d4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.746+0000 7fb3d3fff640 1 -- 172.21.15.7:0/3548415121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d419f620 con 0x7fb3d4100730 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.747+0000 7fb3d3fff640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 0x7fb3d4197a00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9c0 tx=0x7fb3c400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.747+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400ccc0 con 0x7fb3d4100730 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.747+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d419f910 con 0x7fb3d4100730 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.747+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c40035f0 con 0x7fb3d4100730 2024-03-19T00:37:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.747+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400ce60 con 0x7fb3d4100730 2024-03-19T00:37:50.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.747+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d419fe60 con 0x7fb3d4100730 2024-03-19T00:37:50.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.748+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d4107ad0 con 0x7fb3d4100730 2024-03-19T00:37:50.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.749+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fb3c40037a0 con 0x7fb3d4100730 2024-03-19T00:37:50.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.749+0000 7fb3d17fa640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3bc0a2890 0x7fb3bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:50.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.749+0000 7fb3d37fe640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3bc0a2890 0x7fb3bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:50.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.750+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb3c40cf480 con 0x7fb3d4100730 2024-03-19T00:37:50.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.750+0000 7fb3d37fe640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3bc0a2890 0x7fb3bc0a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4198f10 tx=0x7fb3c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.754+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb3c409c000 con 0x7fb3d4100730 2024-03-19T00:37:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.937+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3d4062420 con 0x7fb3d4100730 2024-03-19T00:37:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.937+0000 7fb3d17fa640 1 -- 172.21.15.7:0/3548415121 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fb3c409b900 con 0x7fb3d4100730 2024-03-19T00:37:50.938 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:37:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3bc0a2890 msgr2=0x7fb3bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3bc0a2890 0x7fb3bc0a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4198f10 tx=0x7fb3c8009260 comp rx=0 tx=0).stop 2024-03-19T00:37:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 msgr2=0x7fb3d4197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 0x7fb3d4197a00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9c0 tx=0x7fb3c400edc0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 shutdown_connections 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3bc0a2890 0x7fb3bc0a4d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d41022e0 0x7fb3d419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d4101920 0x7fb3d4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 --2- 172.21.15.7:0/3548415121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3d4100730 0x7fb3d4197a00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 >> 172.21.15.7:0/3548415121 conn(0x7fb3d40fbc80 msgr2=0x7fb3d4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 shutdown_connections 2024-03-19T00:37:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:50.939+0000 7fb3da9df640 1 -- 172.21.15.7:0/3548415121 wait complete. 2024-03-19T00:37:50.950 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:50 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/719515749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:50.950 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:50 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2685575075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:50.950 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:50 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4055679934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:37:50.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:37:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.059+0000 7f516fb1b640 1 -- 172.21.15.7:0/4040321176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 msgr2=0x7f5168101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.059+0000 7f516fb1b640 1 --2- 172.21.15.7:0/4040321176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168101db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5158009900 tx=0x7f515802bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- 172.21.15.7:0/4040321176 shutdown_connections 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 --2- 172.21.15.7:0/4040321176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 0x7f5168108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 --2- 172.21.15.7:0/4040321176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168101db0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 --2- 172.21.15.7:0/4040321176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- 172.21.15.7:0/4040321176 >> 172.21.15.7:0/4040321176 conn(0x7f51680fbcb0 msgr2=0x7f51680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- 172.21.15.7:0/4040321176 shutdown_connections 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- 172.21.15.7:0/4040321176 wait complete. 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 Processor -- start 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- start start 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 0x7f516819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f516810d390 con 0x7f5168101930 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f516810d210 con 0x7f51681022f0 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516fb1b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f516810d510 con 0x7f5168100740 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.060+0000 7f516d0b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d8b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d8b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:60518/0 (socket says 172.21.15.7:60518) 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516e0b8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 0x7f516819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50462/0 (socket says 172.21.15.7:50462) 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d8b7640 1 -- 172.21.15.7:0/1971503751 learned_addr learned my addr 172.21.15.7:0/1971503751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 -- 172.21.15.7:0/1971503751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 msgr2=0x7f5168197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 -- 172.21.15.7:0/1971503751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 msgr2=0x7f516819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 0x7f516819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 -- 172.21.15.7:0/1971503751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516819f6a0 con 0x7f5168101930 2024-03-19T00:37:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516e0b8640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 0x7f516819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d8b7640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516d0b6640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168198010 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5158000c00 tx=0x7f5158005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158043020 con 0x7f5168101930 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5158004350 con 0x7f5168101930 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516819f930 con 0x7f5168101930 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158003b50 con 0x7f5168101930 2024-03-19T00:37:51.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.061+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516819fe80 con 0x7f5168101930 2024-03-19T00:37:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.062+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5168107ba0 con 0x7f5168101930 2024-03-19T00:37:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.063+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f5158046020 con 0x7f5168101930 2024-03-19T00:37:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.064+0000 7f515effd640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51540a2890 0x7f51540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.064+0000 7f516d8b7640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51540a2890 0x7f51540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.064+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f51580f0f10 con 0x7f5168101930 2024-03-19T00:37:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.064+0000 7f516d8b7640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51540a2890 0x7f51540a4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f51600058c0 tx=0x7f5160005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.068+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f515804d020 con 0x7f5168101930 2024-03-19T00:37:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:50 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/719515749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:50 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2685575075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:50 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4055679934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:37:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.252+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f516802d440 con 0x7f5168101930 2024-03-19T00:37:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.252+0000 7f515effd640 1 -- 172.21.15.7:0/1971503751 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f51580bdba0 con 0x7f5168101930 2024-03-19T00:37:51.253 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:37:51.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51540a2890 msgr2=0x7f51540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51540a2890 0x7f51540a4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f51600058c0 tx=0x7f5160005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 msgr2=0x7f5168198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168198010 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5158000c00 tx=0x7f5158005dc0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 shutdown_connections 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51540a2890 0x7f51540a4d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51681022f0 0x7f516819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5168101930 0x7f5168198010 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 --2- 172.21.15.7:0/1971503751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5168100740 0x7f5168197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 >> 172.21.15.7:0/1971503751 conn(0x7f51680fbcb0 msgr2=0x7f5168104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 shutdown_connections 2024-03-19T00:37:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.254+0000 7f516fb1b640 1 -- 172.21.15.7:0/1971503751 wait complete. 2024-03-19T00:37:51.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:50 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/719515749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:51.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:50 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2685575075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:51.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:50 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4055679934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:37:51.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:37:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.370+0000 7f5c28b69640 1 -- 172.21.15.7:0/3419182051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c24101930 msgr2=0x7f5c24101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.370+0000 7f5c28b69640 1 --2- 172.21.15.7:0/3419182051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c24101930 0x7f5c24101db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009900 tx=0x7f5c1402bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 -- 172.21.15.7:0/3419182051 shutdown_connections 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 --2- 172.21.15.7:0/3419182051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c241022f0 0x7f5c24108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 --2- 172.21.15.7:0/3419182051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c24101930 0x7f5c24101db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 --2- 172.21.15.7:0/3419182051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c24100740 0x7f5c24100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 -- 172.21.15.7:0/3419182051 >> 172.21.15.7:0/3419182051 conn(0x7f5c240fbcb0 msgr2=0x7f5c240fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 -- 172.21.15.7:0/3419182051 shutdown_connections 2024-03-19T00:37:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 -- 172.21.15.7:0/3419182051 wait complete. 2024-03-19T00:37:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 Processor -- start 2024-03-19T00:37:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 -- start start 2024-03-19T00:37:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c24100740 0x7f5c24197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.371+0000 7f5c28b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c24101930 0x7f5c24197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c28b69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 0x7f5c2419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c28b69640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410d390 con 0x7f5c241022f0 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c28b69640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410d210 con 0x7f5c24101930 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c28b69640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410d510 con 0x7f5c24100740 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 0x7f5c2419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 0x7f5c2419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50486/0 (socket says 172.21.15.7:50486) 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 -- 172.21.15.7:0/4198866762 learned_addr learned my addr 172.21.15.7:0/4198866762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c2259c640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c24100740 0x7f5c24197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c21d9b640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c24101930 0x7f5c24197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 -- 172.21.15.7:0/4198866762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c24100740 msgr2=0x7f5c24197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c24100740 0x7f5c24197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 -- 172.21.15.7:0/4198866762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c24101930 msgr2=0x7f5c24197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c24101930 0x7f5c24197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 -- 172.21.15.7:0/4198866762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2419f690 con 0x7f5c241022f0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c22d9d640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 0x7f5c2419f020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00eb10 tx=0x7f5c1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c004b20 con 0x7f5c241022f0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c1c004cc0 con 0x7f5c241022f0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2419f980 con 0x7f5c241022f0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c010690 con 0x7f5c241022f0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.372+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2419fe50 con 0x7f5c241022f0 2024-03-19T00:37:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.373+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c24107b80 con 0x7f5c241022f0 2024-03-19T00:37:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.374+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f5c1c005020 con 0x7f5c241022f0 2024-03-19T00:37:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.375+0000 7f5c1b7fe640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c100a2890 0x7f5c100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.375+0000 7f5c2259c640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c100a2890 0x7f5c100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.375+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5c1c0cf4e0 con 0x7f5c241022f0 2024-03-19T00:37:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.375+0000 7f5c2259c640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c100a2890 0x7f5c100a4d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0095f0 tx=0x7f5c0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.379+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5c1c09c0c0 con 0x7f5c241022f0 2024-03-19T00:37:51.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.562+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5c2402d1f0 con 0x7f5c241022f0 2024-03-19T00:37:51.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.562+0000 7f5c1b7fe640 1 -- 172.21.15.7:0/4198866762 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f5c1c09b9c0 con 0x7f5c241022f0 2024-03-19T00:37:51.566 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c100a2890 msgr2=0x7f5c100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c100a2890 0x7f5c100a4d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0095f0 tx=0x7f5c0c005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 msgr2=0x7f5c2419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 0x7f5c2419f020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00eb10 tx=0x7f5c1c00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 shutdown_connections 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c100a2890 0x7f5c100a4d50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c241022f0 0x7f5c2419f020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c24101930 0x7f5c24197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 --2- 172.21.15.7:0/4198866762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c24100740 0x7f5c24197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 >> 172.21.15.7:0/4198866762 conn(0x7f5c240fbcb0 msgr2=0x7f5c24104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 shutdown_connections 2024-03-19T00:37:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.564+0000 7f5c28b69640 1 -- 172.21.15.7:0/4198866762 wait complete. 2024-03-19T00:37:51.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 -- 172.21.15.7:0/1839467388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee50102ed0 msgr2=0x7fee501032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 --2- 172.21.15.7:0/1839467388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee50102ed0 0x7fee501032f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee48009900 tx=0x7fee4802bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 -- 172.21.15.7:0/1839467388 shutdown_connections 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 --2- 172.21.15.7:0/1839467388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee50100300 0x7fee50109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 --2- 172.21.15.7:0/1839467388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee500ff940 0x7fee500ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 --2- 172.21.15.7:0/1839467388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee50102ed0 0x7fee501032f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 -- 172.21.15.7:0/1839467388 >> 172.21.15.7:0/1839467388 conn(0x7fee500fb4a0 msgr2=0x7fee500fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 -- 172.21.15.7:0/1839467388 shutdown_connections 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 -- 172.21.15.7:0/1839467388 wait complete. 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 Processor -- start 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.677+0000 7fee57e12640 1 -- start start 2024-03-19T00:37:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 0x7fee50197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee50100300 0x7fee50197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee50102ed0 0x7fee5019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010d390 con 0x7fee500ff940 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010d210 con 0x7fee50100300 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010d510 con 0x7fee50102ed0 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee55bae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 0x7fee50197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee55bae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 0x7fee50197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50510/0 (socket says 172.21.15.7:50510) 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee55bae640 1 -- 172.21.15.7:0/1859260319 learned_addr learned my addr 172.21.15.7:0/1859260319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee50100300 0x7fee50197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee563af640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee50102ed0 0x7fee5019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 -- 172.21.15.7:0/1859260319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee50102ed0 msgr2=0x7fee5019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee50102ed0 0x7fee5019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 -- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 msgr2=0x7fee50197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 0x7fee50197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 -- 172.21.15.7:0/1859260319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5019f680 con 0x7fee50100300 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee55bae640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 0x7fee50197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee553ad640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee50100300 0x7fee50197fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e9f0 tx=0x7fee4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4400ccc0 con 0x7fee50100300 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5019f970 con 0x7fee50100300 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee440108e0 con 0x7fee50100300 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.678+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44018880 con 0x7fee50100300 2024-03-19T00:37:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.679+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5019fec0 con 0x7fee50100300 2024-03-19T00:37:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.679+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee50108d90 con 0x7fee50100300 2024-03-19T00:37:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.680+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fee44010a90 con 0x7fee50100300 2024-03-19T00:37:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.681+0000 7fee4effd640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee380a2890 0x7fee380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.681+0000 7fee55bae640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee380a2890 0x7fee380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.681+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fee440cf280 con 0x7fee50100300 2024-03-19T00:37:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.681+0000 7fee55bae640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee380a2890 0x7fee380a4d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee4802c840 tx=0x7fee480095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.685+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fee4409b7b0 con 0x7fee50100300 2024-03-19T00:37:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.908+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee5002d440 con 0x7fee50100300 2024-03-19T00:37:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.910+0000 7fee4effd640 1 -- 172.21.15.7:0/1859260319 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7fee4409f850 con 0x7fee50100300 2024-03-19T00:37:51.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:51.911 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:26.295874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:37:51.911 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:37:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee380a2890 msgr2=0x7fee380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee380a2890 0x7fee380a4d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee4802c840 tx=0x7fee480095e0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee50100300 msgr2=0x7fee50197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee50100300 0x7fee50197fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e9f0 tx=0x7fee4400eeb0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 shutdown_connections 2024-03-19T00:37:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee380a2890 0x7fee380a4d50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee50102ed0 0x7fee5019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee50100300 0x7fee50197fa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 --2- 172.21.15.7:0/1859260319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee500ff940 0x7fee50197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 >> 172.21.15.7:0/1859260319 conn(0x7fee500fb4a0 msgr2=0x7fee50105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 shutdown_connections 2024-03-19T00:37:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:51.912+0000 7fee57e12640 1 -- 172.21.15.7:0/1859260319 wait complete. 2024-03-19T00:37:51.922 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:51 smithi007 ceph-mon[62854]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:51.922 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:51 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3548415121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:37:51.922 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:51 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1971503751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:37:51.922 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:51 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4198866762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:37:51.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 -- 172.21.15.7:0/1316773329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728100750 msgr2=0x7fa728100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 --2- 172.21.15.7:0/1316773329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728100750 0x7fa728100b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa724009900 tx=0x7fa72402bc30 comp rx=0 tx=0).stop 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 -- 172.21.15.7:0/1316773329 shutdown_connections 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 --2- 172.21.15.7:0/1316773329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728102300 0x7fa728108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 --2- 172.21.15.7:0/1316773329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728101940 0x7fa728101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 --2- 172.21.15.7:0/1316773329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728100750 0x7fa728100b70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 -- 172.21.15.7:0/1316773329 >> 172.21.15.7:0/1316773329 conn(0x7fa7280fbcc0 msgr2=0x7fa7280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 -- 172.21.15.7:0/1316773329 shutdown_connections 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 -- 172.21.15.7:0/1316773329 wait complete. 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 Processor -- start 2024-03-19T00:37:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.026+0000 7fa72fe12640 1 -- start start 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 0x7fa72810b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728101940 0x7fa728109710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 0x7fa728109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810d3a0 con 0x7fa728102300 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810d220 con 0x7fa728100750 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810d520 con 0x7fa728101940 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72e3af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 0x7fa728109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72dbae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 0x7fa72810b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728101940 0x7fa728109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72dbae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 0x7fa72810b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43146/0 (socket says 172.21.15.7:43146) 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72e3af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 0x7fa728109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50526/0 (socket says 172.21.15.7:50526) 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72dbae640 1 -- 172.21.15.7:0/991456784 learned_addr learned my addr 172.21.15.7:0/991456784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 -- 172.21.15.7:0/991456784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 msgr2=0x7fa72810b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 0x7fa72810b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 -- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 msgr2=0x7fa728109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 0x7fa728109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 -- 172.21.15.7:0/991456784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72810a4e0 con 0x7fa728101940 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72dbae640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 0x7fa72810b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72e3af640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 0x7fa728109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72d3ad640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728101940 0x7fa728109710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa71800e9f0 tx=0x7fa71800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71800ccc0 con 0x7fa728101940 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.027+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7281a1b80 con 0x7fa728101940 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.028+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7180108e0 con 0x7fa728101940 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.028+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718018880 con 0x7fa728101940 2024-03-19T00:37:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.028+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7281a2120 con 0x7fa728101940 2024-03-19T00:37:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.028+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa728107b10 con 0x7fa728101940 2024-03-19T00:37:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.029+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fa718010a90 con 0x7fa728101940 2024-03-19T00:37:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.030+0000 7fa71effd640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa7140a2890 0x7fa7140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.030+0000 7fa72dbae640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa7140a2890 0x7fa7140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.030+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa7180cf2c0 con 0x7fa728101940 2024-03-19T00:37:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.030+0000 7fa72dbae640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa7140a2890 0x7fa7140a4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c840 tx=0x7fa7240095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.033+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa71809b7f0 con 0x7fa728101940 2024-03-19T00:37:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:51 smithi044 ceph-mon[66366]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:51 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3548415121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:37:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:51 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1971503751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:37:52.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:51 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4198866762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:37:52.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.218+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa72802d370 con 0x7fa728101940 2024-03-19T00:37:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.220+0000 7fa71effd640 1 -- 172.21.15.7:0/991456784 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7fa71809f890 con 0x7fa728101940 2024-03-19T00:37:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:52.221 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:37:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.222+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa7140a2890 msgr2=0x7fa7140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.222+0000 7fa72fe12640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa7140a2890 0x7fa7140a4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c840 tx=0x7fa7240095e0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.222+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728101940 msgr2=0x7fa728109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.222+0000 7fa72fe12640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728101940 0x7fa728109710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa71800e9f0 tx=0x7fa71800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 shutdown_connections 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa7140a2890 0x7fa7140a4d50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa728102300 0x7fa728109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728101940 0x7fa728109710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 --2- 172.21.15.7:0/991456784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa728100750 0x7fa72810b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 >> 172.21.15.7:0/991456784 conn(0x7fa7280fbcc0 msgr2=0x7fa728104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 shutdown_connections 2024-03-19T00:37:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.223+0000 7fa72fe12640 1 -- 172.21.15.7:0/991456784 wait complete. 2024-03-19T00:37:52.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:37:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:51 smithi204 ceph-mon[66572]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:37:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:51 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3548415121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:37:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:51 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1971503751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:37:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:51 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4198866762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:37:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 -- 172.21.15.7:0/4053693171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4102300 msgr2=0x7fdab4108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/4053693171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4102300 0x7fdab4108b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009ca0 tx=0x7fdaac02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 -- 172.21.15.7:0/4053693171 shutdown_connections 2024-03-19T00:37:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/4053693171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4102300 0x7fdab4108b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/4053693171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 0x7fdab4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/4053693171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab4100750 0x7fdab4100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 -- 172.21.15.7:0/4053693171 >> 172.21.15.7:0/4053693171 conn(0x7fdab40fbcc0 msgr2=0x7fdab40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 -- 172.21.15.7:0/4053693171 shutdown_connections 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.337+0000 7fdaba6e8640 1 -- 172.21.15.7:0/4053693171 wait complete. 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 Processor -- start 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 -- start start 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 0x7fdab4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 0x7fdab4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab4102300 0x7fdab419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410c280 con 0x7fdab4100750 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410c100 con 0x7fdab4101940 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdaba6e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410c400 con 0x7fdab4102300 2024-03-19T00:37:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.338+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 0x7fdab4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 0x7fdab4197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50546/0 (socket says 172.21.15.7:50546) 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 -- 172.21.15.7:0/3287453083 learned_addr learned my addr 172.21.15.7:0/3287453083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab37fe640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 0x7fdab4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab8c85640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab4102300 0x7fdab419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 -- 172.21.15.7:0/3287453083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab4102300 msgr2=0x7fdab419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab4102300 0x7fdab419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 -- 172.21.15.7:0/3287453083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 msgr2=0x7fdab4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 0x7fdab4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 -- 172.21.15.7:0/3287453083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab419f670 con 0x7fdab4100750 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab37fe640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 0x7fdab4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab3fff640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 0x7fdab4197a50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400e9c0 tx=0x7fdaa400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400cd80 con 0x7fdab4100750 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa40035f0 con 0x7fdab4100750 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4010690 con 0x7fdab4100750 2024-03-19T00:37:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab419f960 con 0x7fdab4100750 2024-03-19T00:37:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.339+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab419fe60 con 0x7fdab4100750 2024-03-19T00:37:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.340+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab41a00f0 con 0x7fdab4100750 2024-03-19T00:37:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.341+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7fdaa40037a0 con 0x7fdab4100750 2024-03-19T00:37:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.342+0000 7fdab17fa640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda9c0a2890 0x7fda9c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.342+0000 7fdab37fe640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda9c0a2890 0x7fda9c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.342+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdaa40cf500 con 0x7fdab4100750 2024-03-19T00:37:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.342+0000 7fdab37fe640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda9c0a2890 0x7fda9c0a4d50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdab4198f60 tx=0x7fdaa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.346+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdaa409c080 con 0x7fdab4100750 2024-03-19T00:37:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.566+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab41a02c0 con 0x7fdab4100750 2024-03-19T00:37:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.567+0000 7fdab17fa640 1 -- 172.21.15.7:0/3287453083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7fdaa409b980 con 0x7fdab4100750 2024-03-19T00:37:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:37:52.568 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:26.295874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:37:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda9c0a2890 msgr2=0x7fda9c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda9c0a2890 0x7fda9c0a4d50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdab4198f60 tx=0x7fdaa8005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 msgr2=0x7fdab4197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 0x7fdab4197a50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400e9c0 tx=0x7fdaa400ee80 comp rx=0 tx=0).stop 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 shutdown_connections 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fda9c0a2890 0x7fda9c0a4d50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab4102300 0x7fdab419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdab4101940 0x7fdab4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 --2- 172.21.15.7:0/3287453083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4100750 0x7fdab4197a50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 >> 172.21.15.7:0/3287453083 conn(0x7fdab40fbcc0 msgr2=0x7fdab40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 shutdown_connections 2024-03-19T00:37:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.569+0000 7fdaba6e8640 1 -- 172.21.15.7:0/3287453083 wait complete. 2024-03-19T00:37:52.581 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-19T00:35:59.576911+0000', 'modified': '2024-03-19T00:36:26.295874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24497, 'mds_2': 24490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/3301743833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3301743833}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3301743833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/1036080774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 1036080774}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1036080774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/2630641732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2630641732}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2630641732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6839/945498116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 945498116}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 945498116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.44:6837/3379136750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 3379136750}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 3379136750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/617428839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 617428839}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 617428839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24521, 24490, 24497]} 2024-03-19T00:37:52.581 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-19T00:37:52.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-19T00:37:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 -- 172.21.15.7:0/1686698887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c1022f0 msgr2=0x7f279c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 --2- 172.21.15.7:0/1686698887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c1022f0 0x7f279c108b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2794009ca0 tx=0x7f279402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 -- 172.21.15.7:0/1686698887 shutdown_connections 2024-03-19T00:37:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 --2- 172.21.15.7:0/1686698887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c1022f0 0x7f279c108b80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 --2- 172.21.15.7:0/1686698887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c101930 0x7f279c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 --2- 172.21.15.7:0/1686698887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c100740 0x7f279c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 -- 172.21.15.7:0/1686698887 >> 172.21.15.7:0/1686698887 conn(0x7f279c0fbcb0 msgr2=0x7f279c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 -- 172.21.15.7:0/1686698887 shutdown_connections 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.684+0000 7f27a293b640 1 -- 172.21.15.7:0/1686698887 wait complete. 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 Processor -- start 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 -- start start 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c100740 0x7f279c1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c101930 0x7f279c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 0x7f279c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10c250 con 0x7f279c1022f0 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10c0d0 con 0x7f279c101930 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a293b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10c3d0 con 0x7f279c100740 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 0x7f279c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c101930 0x7f279c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c100740 0x7f279c1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 0x7f279c19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50564/0 (socket says 172.21.15.7:50564) 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c101930 0x7f279c196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43184/0 (socket says 172.21.15.7:43184) 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a0ed8640 1 -- 172.21.15.7:0/814249427 learned_addr learned my addr 172.21.15.7:0/814249427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279bfff640 1 -- 172.21.15.7:0/814249427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c101930 msgr2=0x7f279c196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279bfff640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c101930 0x7f279c196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279bfff640 1 -- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 msgr2=0x7f279c19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279bfff640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 0x7f279c19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f279bfff640 1 -- 172.21.15.7:0/814249427 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c19e510 con 0x7f279c100740 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.685+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 0x7f279c19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.686+0000 7f279bfff640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c100740 0x7f279c1968f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e9c0 tx=0x7f278c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.686+0000 7f27997fa640 1 -- 172.21.15.7:0/814249427 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00cd80 con 0x7f279c100740 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.686+0000 7f27997fa640 1 -- 172.21.15.7:0/814249427 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f278c0108e0 con 0x7f279c100740 2024-03-19T00:37:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.686+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c19e800 con 0x7f279c100740 2024-03-19T00:37:52.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.686+0000 7f27997fa640 1 -- 172.21.15.7:0/814249427 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c018860 con 0x7f279c100740 2024-03-19T00:37:52.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.686+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c19ed50 con 0x7f279c100740 2024-03-19T00:37:52.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27997fa640 1 -- 172.21.15.7:0/814249427 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f278c01f020 con 0x7f279c100740 2024-03-19T00:37:52.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 Processor -- start 2024-03-19T00:37:52.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 -- start start 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e10411f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 0x55e10411fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10412e2f0 0x55e1041306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e104088de0 con 0x55e10411f600 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e104031c80 con 0x55e104120970 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.688+0000 7f27a5601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e104084e80 con 0x55e10412e2f0 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f279bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 0x55e10411fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e10411f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f279bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 0x55e10411fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50572/0 (socket says 172.21.15.7:50572) 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27997fa640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27840a2890 0x7f27840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f279bfff640 1 -- 172.21.15.7:0/4076467314 learned_addr learned my addr 172.21.15.7:0/4076467314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f279b7fe640 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10412e2f0 0x55e1041306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27840a2890 0x7f27840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 -- 172.21.15.7:0/4076467314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10412e2f0 msgr2=0x55e1041306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10412e2f0 0x55e1041306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 -- 172.21.15.7:0/4076467314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 msgr2=0x55e10411fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 0x55e10411fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 -- 172.21.15.7:0/4076467314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e104043fb0 con 0x55e104120970 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f279bfff640 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 0x55e10411fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27997fa640 1 -- 172.21.15.7:0/814249427 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f278c0cef70 con 0x7f279c100740 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e10411f0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2794032380 tx=0x7f27940328e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d0d2e8a64a21c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27840a2890 0x7f27840a4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2794047000 tx=0x7f2794048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27827fc640 1 -- 172.21.15.7:0/4076467314 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794050070 con 0x55e104120970 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27827fc640 1 -- 172.21.15.7:0/4076467314 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27940046f0 con 0x55e104120970 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.689+0000 7f27827fc640 1 -- 172.21.15.7:0/4076467314 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794039040 con 0x55e104120970 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 -- 172.21.15.7:0/4076467314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 msgr2=0x55e10411f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e10411f0c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2794032380 tx=0x7f27940328e0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 -- 172.21.15.7:0/4076467314 shutdown_connections 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10412e2f0 0x55e1041306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10411f600 0x55e10411fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 --2- 172.21.15.7:0/4076467314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e10411f0c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 -- 172.21.15.7:0/4076467314 >> 172.21.15.7:0/4076467314 conn(0x55e104121770 msgr2=0x55e104121b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 -- 172.21.15.7:0/4076467314 shutdown_connections 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 -- 172.21.15.7:0/4076467314 wait complete. 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.690+0000 7f27a5601740 1 -- start start 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 Processor -- start 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e1041328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 0x55e104132e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10412e2f0 0x55e104139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e104130ec0 con 0x55e104120970 2024-03-19T00:37:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e104044fc0 con 0x55e10412e2f0 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e104131040 con 0x55e10411f600 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a5601740 10 client.0 _ll_register_callbacks cb 0x55e104080a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 0x55e104132e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 0x55e104132e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50580/0 (socket says 172.21.15.7:50580) 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10412e2f0 0x55e104139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 -- 172.21.15.7:0/3050170171 learned_addr learned my addr 172.21.15.7:0/3050170171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279bfff640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e1041328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 -- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 msgr2=0x55e1041328f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e1041328f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 -- 172.21.15.7:0/3050170171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10412e2f0 msgr2=0x55e104139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10412e2f0 0x55e104139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 -- 172.21.15.7:0/3050170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e103d9d2d0 con 0x55e104120970 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10412e2f0 0x55e104139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f279b7fe640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 0x55e104132e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f27900126e0 tx=0x7f2790012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707c4d5a1f8525a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.691+0000 7f277b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f277b7fe640 1 -- 172.21.15.7:0/3050170171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000f610 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f277b7fe640 1 -- 172.21.15.7:0/3050170171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f279000f7b0 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27940046f0 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f277b7fe640 1 -- 172.21.15.7:0/3050170171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f279000f9b0 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e103e676b0 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f277b7fe640 1 -- 172.21.15.7:0/3050170171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27940046f0 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f277b7fe640 1 -- 172.21.15.7:0/3050170171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2790010b90 con 0x55e104120970 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 20 client.15240 populate_metadata read hostname 'smithi007' 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e10413f830 0x55e104140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 4 client.15240 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e104096320 con 0x55e10413f830 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.692+0000 7f279bfff640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e10413f830 0x55e104140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.693+0000 7f279bfff640 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e10413f830 0x55e104140ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31ec7531ee532cf6 server_cookie=bcef377406893285 in_seq=0 out_seq=0 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.693+0000 7f277b7fe640 10 client.15240 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.693+0000 7f277b7fe640 1 -- 172.21.15.7:0/3050170171 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e104096320 con 0x55e10413f830 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.693+0000 7f277b7fe640 10 client.15240 handle_command_reply: tid=0 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.693+0000 7f27a5601740 1 client.15240 shutdown 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 msgr2=0x55e104132e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 0x55e104132e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f27900126e0 tx=0x7f2790012ae0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 shutdown_connections 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e10413f830 0x55e104140ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10412e2f0 0x55e104139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e104120970 0x55e104132e30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f277b7fe640 0 client.15240 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:37:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 --2- 172.21.15.7:0/3050170171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e1041328f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 >> 172.21.15.7:0/3050170171 conn(0x55e104121770 msgr2=0x55e10411e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 shutdown_connections 2024-03-19T00:37:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 1 -- 172.21.15.7:0/3050170171 wait complete. 2024-03-19T00:37:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.694+0000 7f27a5601740 20 client.15240 trim_cache size 0 max 16384 2024-03-19T00:37:52.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 Processor -- start 2024-03-19T00:37:52.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 -- start start 2024-03-19T00:37:52.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10411f600 0x55e10413a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e104120970 0x55e104139e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e104131f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e10413efe0 con 0x55e10412e2f0 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e10413f6b0 con 0x55e10411f600 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a5601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e10413fd80 con 0x55e104120970 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e104131f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e104131f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50586/0 (socket says 172.21.15.7:50586) 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f27a0ed8640 1 -- 172.21.15.7:0/1018024860 learned_addr learned my addr 172.21.15.7:0/1018024860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f279bfff640 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10411f600 0x55e10413a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.712+0000 7f279b7fe640 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e104120970 0x55e104139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a0ed8640 1 -- 172.21.15.7:0/1018024860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e104120970 msgr2=0x55e104139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e104120970 0x55e104139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a0ed8640 1 -- 172.21.15.7:0/1018024860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10411f600 msgr2=0x55e10413a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10411f600 0x55e10413a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a0ed8640 1 -- 172.21.15.7:0/1018024860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1041224e0 con 0x55e10412e2f0 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f279bfff640 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10411f600 0x55e10413a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e104131f10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2794032380 tx=0x7f2794043600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8825ca4ad5c16 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f2781ffb640 1 -- 172.21.15.7:0/1018024860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794058070 con 0x55e10412e2f0 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f2781ffb640 1 -- 172.21.15.7:0/1018024860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2794043670 con 0x55e10412e2f0 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f2781ffb640 1 -- 172.21.15.7:0/1018024860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794039050 con 0x55e10412e2f0 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 -- 172.21.15.7:0/1018024860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 msgr2=0x55e104131f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e104131f10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2794032380 tx=0x7f2794043600 comp rx=0 tx=0).stop 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 -- 172.21.15.7:0/1018024860 shutdown_connections 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e104131f10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e104120970 0x55e104139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 --2- 172.21.15.7:0/1018024860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e10411f600 0x55e10413a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.713+0000 7f27a5601740 1 -- 172.21.15.7:0/1018024860 >> 172.21.15.7:0/1018024860 conn(0x55e104121770 msgr2=0x55e10412a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 -- 172.21.15.7:0/1018024860 shutdown_connections 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 -- 172.21.15.7:0/1018024860 wait complete. 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 -- start start 2024-03-19T00:37:52.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 Processor -- start 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e104134d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e1041352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e1041a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e104140b20 con 0x55e10412e2f0 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e104140450 con 0x55e104120970 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e1041411f0 con 0x55e10411f600 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a5601740 10 client.0 _ll_register_callbacks cb 0x7f2790009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e1041a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f279bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e104134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a0ed8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e1041a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50590/0 (socket says 172.21.15.7:50590) 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f27a0ed8640 1 -- 172.21.15.7:0/3264141670 learned_addr learned my addr 172.21.15.7:0/3264141670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.714+0000 7f279b7fe640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e1041352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a0ed8640 1 -- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 msgr2=0x55e104134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e104134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a0ed8640 1 -- 172.21.15.7:0/3264141670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 msgr2=0x55e1041352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e1041352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a0ed8640 1 -- 172.21.15.7:0/3264141670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1041a3ad0 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f279b7fe640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e1041352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a0ed8640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e1041a3590 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2794043c30 tx=0x7f2794043c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7decab648220455 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f277b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f277b7fe640 1 -- 172.21.15.7:0/3264141670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794058040 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f277b7fe640 1 -- 172.21.15.7:0/3264141670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2794004760 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2794043670 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f277b7fe640 1 -- 172.21.15.7:0/3264141670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2794039020 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a5601740 10 client.15246 fetch_fsmap learned FSMap version 24 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e1041a4d80 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f277b7fe640 1 -- 172.21.15.7:0/3264141670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2794043670 con 0x55e10412e2f0 2024-03-19T00:37:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.715+0000 7f277b7fe640 1 -- 172.21.15.7:0/3264141670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f279403f070 con 0x55e10412e2f0 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 10 client.15246 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 20 client.15246 populate_metadata read hostname 'smithi007' 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e1041a93f0 0x55e1041a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 4 client.15246 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "status"}) -- 0x55e104090670 con 0x55e1041a93f0 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f279bfff640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e1041a93f0 0x55e1041a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f277b7fe640 10 client.15246 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f279bfff640 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e1041a93f0 0x55e1041a8100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f28333463d2db0c6 server_cookie=15952b75925c0841 in_seq=0 out_seq=0 2024-03-19T00:37:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f277b7fe640 1 -- 172.21.15.7:0/3264141670 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e104090670 con 0x55e1041a93f0 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f277b7fe640 10 client.15246 handle_command_reply: tid=0 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 1 client.15246 shutdown 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 msgr2=0x55e1041a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.716+0000 7f27a5601740 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e1041a3590 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2794043c30 tx=0x7f2794043c60 comp rx=0 tx=0).stop 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 shutdown_connections 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e1041a93f0 0x55e1041a8100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e10412e2f0 0x55e1041a3590 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e104120970 0x55e1041352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f277b7fe640 0 client.15246 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 --2- 172.21.15.7:0/3264141670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10411f600 0x55e104134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 >> 172.21.15.7:0/3264141670 conn(0x55e104121770 msgr2=0x55e1041249d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 shutdown_connections 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 1 -- 172.21.15.7:0/3264141670 wait complete. 2024-03-19T00:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a5601740 20 client.15246 trim_cache size 0 max 16384 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24490, 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 87.431154375000006, 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 91, 2024-03-19T00:37:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:37:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 125.361461191 2024-03-19T00:37:52.726 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:37:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27840a2890 msgr2=0x7f27840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27840a2890 0x7f27840a4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2794047000 tx=0x7f2794048000 comp rx=0 tx=0).stop 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c100740 msgr2=0x7f279c1968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c100740 0x7f279c1968f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e9c0 tx=0x7f278c00ee80 comp rx=0 tx=0).stop 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 shutdown_connections 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27840a2890 0x7f27840a4d50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f279c1022f0 0x7f279c19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f279c101930 0x7f279c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 --2- 172.21.15.7:0/814249427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f279c100740 0x7f279c1968f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 >> 172.21.15.7:0/814249427 conn(0x7f279c0fbcb0 msgr2=0x7f279c104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 shutdown_connections 2024-03-19T00:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.717+0000 7f27a293b640 1 -- 172.21.15.7:0/814249427 wait complete. 2024-03-19T00:37:52.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-19T00:37:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 -- 172.21.15.7:0/3983679207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 msgr2=0x7f3f50108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 --2- 172.21.15.7:0/3983679207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f50108b80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009ca0 tx=0x7f3f4802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 -- 172.21.15.7:0/3983679207 shutdown_connections 2024-03-19T00:37:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 --2- 172.21.15.7:0/3983679207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f50108b80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 --2- 172.21.15.7:0/3983679207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f50101930 0x7f3f50101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 --2- 172.21.15.7:0/3983679207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f50100740 0x7f3f50100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 -- 172.21.15.7:0/3983679207 >> 172.21.15.7:0/3983679207 conn(0x7f3f500fbcb0 msgr2=0x7f3f500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 -- 172.21.15.7:0/3983679207 shutdown_connections 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 -- 172.21.15.7:0/3983679207 wait complete. 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 Processor -- start 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 -- start start 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f50100740 0x7f3f50197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.833+0000 7f3f58485640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f50101930 0x7f3f50197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f58485640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f5019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f58485640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010c270 con 0x7f3f501022f0 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f58485640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010c0f0 con 0x7f3f50101930 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f58485640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010c3f0 con 0x7f3f50100740 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f5019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f5019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50610/0 (socket says 172.21.15.7:50610) 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 -- 172.21.15.7:0/843552273 learned_addr learned my addr 172.21.15.7:0/843552273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f55a20640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f50101930 0x7f3f50197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56221640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f50100740 0x7f3f50197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 -- 172.21.15.7:0/843552273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f50100740 msgr2=0x7f3f50197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f50100740 0x7f3f50197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 -- 172.21.15.7:0/843552273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f50101930 msgr2=0x7f3f50197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f50101930 0x7f3f50197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 -- 172.21.15.7:0/843552273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5019f670 con 0x7f3f501022f0 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f56a22640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f5019f000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f4802c700 tx=0x7f3f480328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f477fe640 1 -- 172.21.15.7:0/843552273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48041020 con 0x7f3f501022f0 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f477fe640 1 -- 172.21.15.7:0/843552273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f48003e20 con 0x7f3f501022f0 2024-03-19T00:37:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5019f900 con 0x7f3f501022f0 2024-03-19T00:37:52.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f477fe640 1 -- 172.21.15.7:0/843552273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48039040 con 0x7f3f501022f0 2024-03-19T00:37:52.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.834+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5019fe40 con 0x7f3f501022f0 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.836+0000 7f3f477fe640 1 -- 172.21.15.7:0/843552273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f3f48044020 con 0x7f3f501022f0 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 Processor -- start 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 -- start start 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c68410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c689e0 0x557a31c6d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f477fe640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3f3c0a2910 0x7f3f3c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c6db50 0x557a31c6ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557a31bccde0 con 0x557a31c67ff0 2024-03-19T00:37:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a31b75c80 con 0x557a31c6db50 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f5b078740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a31bc8e80 con 0x557a31c689e0 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f55a20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f56a22640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c6db50 0x557a31c6ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f55a20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c68410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50618/0 (socket says 172.21.15.7:50618) 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f55a20640 1 -- 172.21.15.7:0/725922424 learned_addr learned my addr 172.21.15.7:0/725922424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f56221640 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c689e0 0x557a31c6d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f55a20640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3f3c0a2910 0x7f3f3c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.837+0000 7f3f477fe640 1 -- 172.21.15.7:0/843552273 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3f480f0f60 con 0x7f3f501022f0 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f56a22640 1 -- 172.21.15.7:0/725922424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c689e0 msgr2=0x557a31c6d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f56a22640 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c689e0 0x557a31c6d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f56a22640 1 -- 172.21.15.7:0/725922424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 msgr2=0x557a31c68410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f56a22640 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c68410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f56a22640 1 -- 172.21.15.7:0/725922424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a31b87fb0 con 0x557a31c6db50 2024-03-19T00:37:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f55a20640 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c68410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f56a22640 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c6db50 0x557a31c6ffe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f480ee6c0 tx=0x7f3f48039bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63532972c44140b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f55a20640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3f3c0a2910 0x7f3f3c0a4dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x557a31c69240 tx=0x7f3f40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f44bfe640 1 -- 172.21.15.7:0/725922424 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f480c2bc0 con 0x557a31c6db50 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f44bfe640 1 -- 172.21.15.7:0/725922424 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f48034700 con 0x557a31c6db50 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.838+0000 7f3f44bfe640 1 -- 172.21.15.7:0/725922424 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f480dd7d0 con 0x557a31c6db50 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- 172.21.15.7:0/725922424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c6db50 msgr2=0x557a31c6ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c6db50 0x557a31c6ffe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f480ee6c0 tx=0x7f3f48039bb0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- 172.21.15.7:0/725922424 shutdown_connections 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c6db50 0x557a31c6ffe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c689e0 0x557a31c6d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- 172.21.15.7:0/725922424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c68410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- 172.21.15.7:0/725922424 >> 172.21.15.7:0/725922424 conn(0x557a31c63560 msgr2=0x557a31c63960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- 172.21.15.7:0/725922424 shutdown_connections 2024-03-19T00:37:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- 172.21.15.7:0/725922424 wait complete. 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- start start 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 Processor -- start 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c72250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c72790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31c797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557a31c708c0 con 0x557a31c67ff0 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a31b88fc0 con 0x557a31c689e0 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a31c70a40 con 0x557a31c6db50 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.839+0000 7f3f5b078740 10 client.0 _ll_register_callbacks cb 0x557a31bc4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c72250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c72250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50628/0 (socket says 172.21.15.7:50628) 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 -- 172.21.15.7:0/1699786696 learned_addr learned my addr 172.21.15.7:0/1699786696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c72790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56a22640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31c797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 -- 172.21.15.7:0/1699786696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 msgr2=0x557a31c797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31c797d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 -- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 msgr2=0x557a31c72790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c72790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 -- 172.21.15.7:0/1699786696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a318e12d0 con 0x557a31c67ff0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c72790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f56221640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c72250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0126e0 tx=0x7f3f4c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59a01878d503342 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1699786696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c004050 con 0x557a31c67ff0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1699786696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3f4c0041f0 con 0x557a31c67ff0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a31bda320 con 0x557a31c67ff0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1699786696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c0043f0 con 0x557a31c67ff0 2024-03-19T00:37:52.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f5b078740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557a319ab6b0 con 0x557a31c67ff0 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.840+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1699786696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557a31bda320 con 0x557a31c67ff0 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1699786696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3f4c003b50 con 0x557a31c67ff0 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f5b078740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f5b078740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f5b078740 20 client.15258 populate_metadata read hostname 'smithi007' 2024-03-19T00:37:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31c7f150 0x557a31c805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f5b078740 4 client.15258 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a31bd4670 con 0x557a31c7f150 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31c7f150 0x557a31c805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f39ffb640 10 client.15258 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.841+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31c7f150 0x557a31c805c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a83cabe12d808bcc server_cookie=b557f304ecca95a6 in_seq=0 out_seq=0 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1699786696 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557a31bd4670 con 0x557a31c7f150 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f39ffb640 10 client.15258 handle_command_reply: tid=0 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 client.15258 shutdown 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 msgr2=0x557a31c72250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c72250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0126e0 tx=0x7f3f4c012ba0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 shutdown_connections 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31c7f150 0x557a31c805c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31c797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c72790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f39ffb640 0 client.15258 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:37:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1699786696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c67ff0 0x557a31c72250 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 >> 172.21.15.7:0/1699786696 conn(0x557a31c63560 msgr2=0x557a31c627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 shutdown_connections 2024-03-19T00:37:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 1 -- 172.21.15.7:0/1699786696 wait complete. 2024-03-19T00:37:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.842+0000 7f3f5b078740 20 client.15258 trim_cache size 0 max 16384 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.860+0000 7f3f5b078740 1 Processor -- start 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.860+0000 7f3f5b078740 1 -- start start 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c67ff0 0x557a31c69ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 0x557a31c796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f5b078740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557a31c7e900 con 0x557a31c6db50 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f5b078740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a31c7efd0 con 0x557a31c689e0 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f5b078740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a31c7f6a0 con 0x557a31c67ff0 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 0x557a31c796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 0x557a31c796f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50638/0 (socket says 172.21.15.7:50638) 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 -- 172.21.15.7:0/3827358009 learned_addr learned my addr 172.21.15.7:0/3827358009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56221640 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c67ff0 0x557a31c69ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f55a20640 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 -- 172.21.15.7:0/3827358009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c67ff0 msgr2=0x557a31c69ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c67ff0 0x557a31c69ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 -- 172.21.15.7:0/3827358009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 msgr2=0x557a31c6a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c6a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 -- 172.21.15.7:0/3827358009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a31c6d860 con 0x557a31c6db50 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f56a22640 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 0x557a31c796f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f480ee740 tx=0x7f3f4802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee7e9023b48212d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f3bfff640 1 -- 172.21.15.7:0/3827358009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f480bd550 con 0x557a31c6db50 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f3bfff640 1 -- 172.21.15.7:0/3827358009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f480bdc50 con 0x557a31c6db50 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.861+0000 7f3f3bfff640 1 -- 172.21.15.7:0/3827358009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f480ee900 con 0x557a31c6db50 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- 172.21.15.7:0/3827358009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 msgr2=0x557a31c796f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 0x557a31c796f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f480ee740 tx=0x7f3f4802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- 172.21.15.7:0/3827358009 shutdown_connections 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c6db50 0x557a31c796f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c689e0 0x557a31c6a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- 172.21.15.7:0/3827358009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c67ff0 0x557a31c69ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- 172.21.15.7:0/3827358009 >> 172.21.15.7:0/3827358009 conn(0x557a31c63560 msgr2=0x557a31c63940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- 172.21.15.7:0/3827358009 shutdown_connections 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- 172.21.15.7:0/3827358009 wait complete. 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- start start 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 Processor -- start 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c67ff0 0x557a31c74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 0x557a31c74b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31ce2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557a31c80440 con 0x557a31c689e0 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557a31c7fd70 con 0x557a31c67ff0 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557a31c80b10 con 0x557a31c6db50 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f5b078740 10 client.0 _ll_register_callbacks cb 0x7f3f4c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:37:52.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.862+0000 7f3f55a20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 0x557a31c74b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 0x557a31c74b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50648/0 (socket says 172.21.15.7:50648) 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 -- 172.21.15.7:0/1028477202 learned_addr learned my addr 172.21.15.7:0/1028477202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f56221640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c67ff0 0x557a31c74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f56a22640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31ce2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 -- 172.21.15.7:0/1028477202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 msgr2=0x557a31ce2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31ce2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 -- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c67ff0 msgr2=0x557a31c74630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c67ff0 0x557a31c74630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 -- 172.21.15.7:0/1028477202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a31ce33f0 con 0x557a31c689e0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f56a22640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31ce2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f55a20640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 0x557a31c74b70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009b50 tx=0x7f3f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25175aa52c234c0a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1028477202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4002f020 con 0x557a31c689e0 2024-03-19T00:37:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1028477202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3f40011050 con 0x557a31c689e0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f480bdc50 con 0x557a31c689e0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1028477202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f4003a050 con 0x557a31c689e0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f5b078740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557a31ce4740 con 0x557a31c689e0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.863+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1028477202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f480bdc50 con 0x557a31c689e0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1028477202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3f4001d040 con 0x557a31c689e0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 20 client.15264 populate_metadata read hostname 'smithi007' 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31ce9370 0x557a31ce8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 4 client.15264 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "status"}) -- 0x557a31b52a00 con 0x557a31ce9370 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f56221640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31ce9370 0x557a31ce8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f39ffb640 10 client.15264 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f56221640 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31ce9370 0x557a31ce8080 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2de13f19869c118 server_cookie=7ebb0252d440ef3b in_seq=0 out_seq=0 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f39ffb640 1 -- 172.21.15.7:0/1028477202 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557a31b52a00 con 0x557a31ce9370 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f39ffb640 10 client.15264 handle_command_reply: tid=0 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 client.15264 shutdown 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 msgr2=0x557a31c74b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 0x557a31c74b70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009b50 tx=0x7f3f400046c0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 shutdown_connections 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.864+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x557a31ce9370 0x557a31ce8080 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a31c6db50 0x557a31ce2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a31c689e0 0x557a31c74b70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f39ffb640 0 client.15264 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 1 --2- 172.21.15.7:0/1028477202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a31c67ff0 0x557a31c74630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 >> 172.21.15.7:0/1028477202 conn(0x557a31c63560 msgr2=0x557a31c7df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 shutdown_connections 2024-03-19T00:37:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 1 -- 172.21.15.7:0/1028477202 wait complete. 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f5b078740 20 client.15264 trim_cache size 0 max 16384 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24497, 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 88.586320263999994, 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-03-19T00:37:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 91, 2024-03-19T00:37:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:37:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 128.255332469 2024-03-19T00:37:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:37:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3f3c0a2910 msgr2=0x7f3f3c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3f3c0a2910 0x7f3f3c0a4dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x557a31c69240 tx=0x7f3f40005f90 comp rx=0 tx=0).stop 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 msgr2=0x7f3f5019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f5019f000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f4802c700 tx=0x7f3f480328a0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 shutdown_connections 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3f3c0a2910 0x7f3f3c0a4dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f501022f0 0x7f3f5019f000 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f50101930 0x7f3f50197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 --2- 172.21.15.7:0/843552273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f50100740 0x7f3f50197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 >> 172.21.15.7:0/843552273 conn(0x7f3f500fbcb0 msgr2=0x7f3f500fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 shutdown_connections 2024-03-19T00:37:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.865+0000 7f3f58485640 1 -- 172.21.15.7:0/843552273 wait complete. 2024-03-19T00:37:52.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:52 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1859260319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:52.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:52 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/991456784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:52.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:52 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3287453083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:52.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 -- 172.21.15.7:0/4026314655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c1096e0 msgr2=0x7f278c10bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 --2- 172.21.15.7:0/4026314655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c1096e0 0x7f278c10bb70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f277c009ca0 tx=0x7f277c02c180 comp rx=0 tx=0).stop 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 -- 172.21.15.7:0/4026314655 shutdown_connections 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 --2- 172.21.15.7:0/4026314655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c1096e0 0x7f278c10bb70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 --2- 172.21.15.7:0/4026314655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f278c074200 0x7f278c074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 --2- 172.21.15.7:0/4026314655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f278c0757e0 0x7f278c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.979+0000 7f2790bb3640 1 -- 172.21.15.7:0/4026314655 >> 172.21.15.7:0/4026314655 conn(0x7f278c0fcda0 msgr2=0x7f278c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 -- 172.21.15.7:0/4026314655 shutdown_connections 2024-03-19T00:37:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 -- 172.21.15.7:0/4026314655 wait complete. 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 Processor -- start 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 -- start start 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 0x7f278c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f278c0757e0 0x7f278c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f278c1096e0 0x7f278c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f278c10f570 con 0x7f278c074200 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f278c10f3f0 con 0x7f278c1096e0 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f2790bb3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f278c10f6f0 con 0x7f278c0757e0 2024-03-19T00:37:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.980+0000 7f278a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 0x7f278c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 0x7f278c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50664/0 (socket says 172.21.15.7:50664) 2024-03-19T00:37:52.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 -- 172.21.15.7:0/225521168 learned_addr learned my addr 172.21.15.7:0/225521168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:52.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278ad9d640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f278c1096e0 0x7f278c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f2789d9b640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f278c0757e0 0x7f278c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 -- 172.21.15.7:0/225521168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f278c0757e0 msgr2=0x7f278c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f278c0757e0 0x7f278c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 -- 172.21.15.7:0/225521168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f278c1096e0 msgr2=0x7f278c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f278c1096e0 0x7f278c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 -- 172.21.15.7:0/225521168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1a1840 con 0x7f278c074200 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f278a59c640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 0x7f278c199c30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f278400ea60 tx=0x7f278400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f27837fe640 1 -- 172.21.15.7:0/225521168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278400cde0 con 0x7f278c074200 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f27837fe640 1 -- 172.21.15.7:0/225521168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27840035f0 con 0x7f278c074200 2024-03-19T00:37:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c1a1b30 con 0x7f278c074200 2024-03-19T00:37:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f27837fe640 1 -- 172.21.15.7:0/225521168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784010690 con 0x7f278c074200 2024-03-19T00:37:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.981+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c1a2080 con 0x7f278c074200 2024-03-19T00:37:52.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.983+0000 7f27837fe640 1 -- 172.21.15.7:0/225521168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f27840037a0 con 0x7f278c074200 2024-03-19T00:37:52.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 Processor -- start 2024-03-19T00:37:52.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f27837fe640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27780a2890 0x7f27780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f278ad9d640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27780a2890 0x7f27780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f27837fe640 1 -- 172.21.15.7:0/225521168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f27840cf270 con 0x7f278c074200 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 -- start start 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55860ad6ede0 con 0x55860ae0fb50 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55860ad17c80 con 0x55860ae0a9e0 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2793799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55860ad6ae80 con 0x55860ae09ff0 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2789d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f278a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:52.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f278a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae0a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:60738/0 (socket says 172.21.15.7:60738) 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f2789d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae0f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43324/0 (socket says 172.21.15.7:43324) 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f278a59c640 1 -- 172.21.15.7:0/1134170965 learned_addr learned my addr 172.21.15.7:0/1134170965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f278ad9d640 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.984+0000 7f278ad9d640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27780a2890 0x7f27780a4d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f277c005e30 tx=0x7f277c009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2789d9b640 1 -- 172.21.15.7:0/1134170965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 msgr2=0x55860ae0a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2789d9b640 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae0a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2789d9b640 1 -- 172.21.15.7:0/1134170965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 msgr2=0x55860ae11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2789d9b640 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2789d9b640 1 -- 172.21.15.7:0/1134170965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860ad29fb0 con 0x55860ae0a9e0 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f278ad9d640 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2789d9b640 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae0f4e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2774009c00 tx=0x7f2774011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea2963ecde26ec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:53.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2780ff9640 1 -- 172.21.15.7:0/1134170965 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774004230 con 0x55860ae0a9e0 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2780ff9640 1 -- 172.21.15.7:0/1134170965 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27740043d0 con 0x55860ae0a9e0 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.985+0000 7f2780ff9640 1 -- 172.21.15.7:0/1134170965 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774004c40 con 0x55860ae0a9e0 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- 172.21.15.7:0/1134170965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 msgr2=0x55860ae0f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae0f4e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2774009c00 tx=0x7f2774011810 comp rx=0 tx=0).stop 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- 172.21.15.7:0/1134170965 shutdown_connections 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae0f4e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- 172.21.15.7:0/1134170965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae0a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- 172.21.15.7:0/1134170965 >> 172.21.15.7:0/1134170965 conn(0x55860ae05560 msgr2=0x55860ae05960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- 172.21.15.7:0/1134170965 shutdown_connections 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- 172.21.15.7:0/1134170965 wait complete. 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- start start 2024-03-19T00:37:53.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 Processor -- start 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae144e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 0x55860ae14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0fb50 0x55860ae1ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55860ae12b00 con 0x55860ae0a9e0 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55860ad2afc0 con 0x55860ae09ff0 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55860ae12c80 con 0x55860ae0fb50 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2793799740 10 client.0 _ll_register_callbacks cb 0x55860ad66a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2789d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 0x55860ae14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2789d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 0x55860ae14a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50688/0 (socket says 172.21.15.7:50688) 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f2789d9b640 1 -- 172.21.15.7:0/3554193132 learned_addr learned my addr 172.21.15.7:0/3554193132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f278ad9d640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0fb50 0x55860ae1ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.986+0000 7f278a59c640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2789d9b640 1 -- 172.21.15.7:0/3554193132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0fb50 msgr2=0x55860ae1ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2789d9b640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0fb50 0x55860ae1ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2789d9b640 1 -- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 msgr2=0x55860ae144e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2789d9b640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae144e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2789d9b640 1 -- 172.21.15.7:0/3554193132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55860aa832d0 con 0x55860ae0a9e0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2789d9b640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 0x55860ae14a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2774009bd0 tx=0x7f2774019ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62ecd849cb60475 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2771ffb640 1 -- 172.21.15.7:0/3554193132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774028020 con 0x55860ae0a9e0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2771ffb640 1 -- 172.21.15.7:0/3554193132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2774024620 con 0x55860ae0a9e0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27740043d0 con 0x55860ae0a9e0 2024-03-19T00:37:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2771ffb640 1 -- 172.21.15.7:0/3554193132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2774033050 con 0x55860ae0a9e0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2793799740 10 client.15276 fetch_fsmap learned FSMap version 24 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55860ab4d6b0 con 0x55860ae0a9e0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.987+0000 7f2771ffb640 1 -- 172.21.15.7:0/3554193132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27740043d0 con 0x55860ae0a9e0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2771ffb640 1 -- 172.21.15.7:0/3554193132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2774018070 con 0x55860ae0a9e0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2793799740 10 client.15276 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2793799740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2793799740 20 client.15276 populate_metadata read hostname 'smithi007' 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2793799740 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae210c0 0x55860ae22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2793799740 4 client.15276 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55860ad7c320 con 0x55860ae210c0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f278a59c640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae210c0 0x55860ae22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f2771ffb640 10 client.15276 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.988+0000 7f278a59c640 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae210c0 0x55860ae22530 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7168641fb36acc2d server_cookie=329a6abd254248d6 in_seq=0 out_seq=0 2024-03-19T00:37:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2771ffb640 1 -- 172.21.15.7:0/3554193132 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55860ad7c320 con 0x55860ae210c0 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2771ffb640 10 client.15276 handle_command_reply: tid=0 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 client.15276 shutdown 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 msgr2=0x55860ae14a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 0x55860ae14a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2774009bd0 tx=0x7f2774019ad0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 shutdown_connections 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0fb50 0x55860ae1ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae210c0 0x55860ae22530 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0a9e0 0x55860ae14a20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 --2- 172.21.15.7:0/3554193132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae144e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2771ffb640 0 client.15276 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:37:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 >> 172.21.15.7:0/3554193132 conn(0x55860ae05560 msgr2=0x55860ae047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 shutdown_connections 2024-03-19T00:37:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.989+0000 7f2793799740 1 -- 172.21.15.7:0/3554193132 wait complete. 2024-03-19T00:37:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:52.990+0000 7f2793799740 20 client.15276 trim_cache size 0 max 16384 2024-03-19T00:37:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 Processor -- start 2024-03-19T00:37:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 -- start start 2024-03-19T00:37:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0a9e0 0x55860ae1ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55860ae20870 con 0x55860ae0fb50 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55860ae20f40 con 0x55860ae09ff0 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2793799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55860ae21610 con 0x55860ae0a9e0 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae14630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50692/0 (socket says 172.21.15.7:50692) 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 -- 172.21.15.7:0/1199475077 learned_addr learned my addr 172.21.15.7:0/1199475077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f2789d9b640 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0a9e0 0x55860ae1ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 -- 172.21.15.7:0/1199475077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0a9e0 msgr2=0x55860ae1ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0a9e0 0x55860ae1ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 -- 172.21.15.7:0/1199475077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 msgr2=0x55860ae140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 -- 172.21.15.7:0/1199475077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860ae062d0 con 0x55860ae0fb50 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278a59c640 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.008+0000 7f278ad9d640 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae14630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f277c032520 tx=0x7f277c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3088a92ff7b51f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2773fff640 1 -- 172.21.15.7:0/1199475077 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c039040 con 0x55860ae0fb50 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2773fff640 1 -- 172.21.15.7:0/1199475077 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277c047070 con 0x55860ae0fb50 2024-03-19T00:37:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2773fff640 1 -- 172.21.15.7:0/1199475077 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c04c480 con 0x55860ae0fb50 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 -- 172.21.15.7:0/1199475077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 msgr2=0x55860ae14630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae14630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f277c032520 tx=0x7f277c040000 comp rx=0 tx=0).stop 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 -- 172.21.15.7:0/1199475077 shutdown_connections 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae14630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae0a9e0 0x55860ae1ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 --2- 172.21.15.7:0/1199475077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae09ff0 0x55860ae140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 -- 172.21.15.7:0/1199475077 >> 172.21.15.7:0/1199475077 conn(0x55860ae05560 msgr2=0x55860ae20000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 -- 172.21.15.7:0/1199475077 shutdown_connections 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 -- 172.21.15.7:0/1199475077 wait complete. 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.009+0000 7f2793799740 1 -- start start 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 Processor -- start 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae1b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae18340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae18880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55860ae223b0 con 0x55860ae0fb50 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55860ae21ce0 con 0x55860ae0a9e0 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55860ae22a80 con 0x55860ae09ff0 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2793799740 10 client.0 _ll_register_callbacks cb 0x7f277c045030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae18880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae18880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50700/0 (socket says 172.21.15.7:50700) 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 -- 172.21.15.7:0/3720000111 learned_addr learned my addr 172.21.15.7:0/3720000111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2789d9b640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae18340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278a59c640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae1b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 -- 172.21.15.7:0/3720000111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 msgr2=0x55860ae1b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae1b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 -- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 msgr2=0x55860ae18340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae18340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 -- 172.21.15.7:0/3720000111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55860ae15610 con 0x55860ae0fb50 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f278ad9d640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae18880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f277c037040 tx=0x7f277c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7872557b52ccabfe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.010+0000 7f2771ffb640 1 -- 172.21.15.7:0/3720000111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c05c070 con 0x55860ae0fb50 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2771ffb640 1 -- 172.21.15.7:0/3720000111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f277c032a90 con 0x55860ae0fb50 2024-03-19T00:37:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c047070 con 0x55860ae0fb50 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2771ffb640 1 -- 172.21.15.7:0/3720000111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f277c039030 con 0x55860ae0fb50 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 10 client.15282 fetch_fsmap learned FSMap version 24 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55860ae126f0 con 0x55860ae0fb50 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2771ffb640 1 -- 172.21.15.7:0/3720000111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277c047070 con 0x55860ae0fb50 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2771ffb640 1 -- 172.21.15.7:0/3720000111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f277c03f040 con 0x55860ae0fb50 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 10 client.15282 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 20 client.15282 populate_metadata read hostname 'smithi007' 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae8c4c0 0x55860ae8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 4 client.15282 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "status"}) -- 0x55860ad76670 con 0x55860ae8c4c0 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.011+0000 7f278a59c640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae8c4c0 0x55860ae8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f278a59c640 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae8c4c0 0x55860ae8b1d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d99fa4559bbc68 server_cookie=18aa9035e919ac90 in_seq=0 out_seq=0 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2771ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2771ffb640 1 -- 172.21.15.7:0/3720000111 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55860ad76670 con 0x55860ae8c4c0 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2771ffb640 10 client.15282 handle_command_reply: tid=0 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 client.15282 shutdown 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 msgr2=0x55860ae18880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae18880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f277c037040 tx=0x7f277c009540 comp rx=0 tx=0).stop 2024-03-19T00:37:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 shutdown_connections 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55860ae8c4c0 0x55860ae8b1d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55860ae0fb50 0x55860ae18880 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2771ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55860ae0a9e0 0x55860ae18340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 --2- 172.21.15.7:0/3720000111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860ae09ff0 0x55860ae1b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 >> 172.21.15.7:0/3720000111 conn(0x55860ae05560 msgr2=0x55860ae140f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 shutdown_connections 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 1 -- 172.21.15.7:0/3720000111 wait complete. 2024-03-19T00:37:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2793799740 20 client.15282 trim_cache size 0 max 16384 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24521, 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:37:53.026 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 112.809148124, 2024-03-19T00:37:53.027 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-03-19T00:37:53.027 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 91, 2024-03-19T00:37:53.027 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:37:53.027 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 116.979944174 2024-03-19T00:37:53.027 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:37:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27780a2890 msgr2=0x7f27780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2790bb3640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27780a2890 0x7f27780a4d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f277c005e30 tx=0x7f277c009800 comp rx=0 tx=0).stop 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 msgr2=0x7f278c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.012+0000 7f2790bb3640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 0x7f278c199c30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f278400ea60 tx=0x7f278400ef20 comp rx=0 tx=0).stop 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 shutdown_connections 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27780a2890 0x7f27780a4d50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f278c1096e0 0x7f278c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f278c0757e0 0x7f278c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 --2- 172.21.15.7:0/225521168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f278c074200 0x7f278c199c30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 >> 172.21.15.7:0/225521168 conn(0x7f278c0fcda0 msgr2=0x7f278c105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 shutdown_connections 2024-03-19T00:37:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:53.013+0000 7f2790bb3640 1 -- 172.21.15.7:0/225521168 wait complete. 2024-03-19T00:37:53.030 INFO:tasks.cephadm:Setup complete, yielding 2024-03-19T00:37:53.031 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-19T00:37:53.041 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-19T00:37:53.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-19T00:37:53.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:52 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1859260319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:53.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:52 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/991456784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:53.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:52 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3287453083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:53.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:52 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1859260319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:53.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:52 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/991456784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:37:53.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:52 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3287453083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:37:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:53.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:53 smithi007 ceph-mon[62854]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:53 smithi044 ceph-mon[66366]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:54.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:53 smithi204 ceph-mon[66572]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:37:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 -- 172.21.15.7:0/1058517205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102440 msgr2=0x7f971c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 --2- 172.21.15.7:0/1058517205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102440 0x7f971c1028a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f97100099c0 tx=0x7f9710009cd0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 -- 172.21.15.7:0/1058517205 shutdown_connections 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 --2- 172.21.15.7:0/1058517205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f971c102de0 0x7f971c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 --2- 172.21.15.7:0/1058517205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 0x7f971c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 --2- 172.21.15.7:0/1058517205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102440 0x7f971c1028a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 -- 172.21.15.7:0/1058517205 >> 172.21.15.7:0/1058517205 conn(0x7f971c0fc7e0 msgr2=0x7f971c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 -- 172.21.15.7:0/1058517205 shutdown_connections 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 -- 172.21.15.7:0/1058517205 wait complete. 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.064+0000 7f9721cbf700 1 Processor -- start 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 -- start start 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 0x7f971c1051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f971c102440 0x7f971c103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 0x7f971c103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f971c10ccc0 con 0x7f971c102de0 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f971c10cb40 con 0x7f971c102440 2024-03-19T00:37:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f9721cbf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f971c10ce40 con 0x7f971c101240 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 0x7f971c103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 0x7f971c1051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f971c102440 0x7f971c103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 0x7f971c103d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44706/0 (socket says 172.21.15.7:44706) 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971bfff700 1 -- 172.21.15.7:0/2479348320 learned_addr learned my addr 172.21.15.7:0/2479348320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 -- 172.21.15.7:0/2479348320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f971c102440 msgr2=0x7f971c103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f971c102440 0x7f971c103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 -- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 msgr2=0x7f971c103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 0x7f971c103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 -- 172.21.15.7:0/2479348320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c104610 con 0x7f971c101240 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971bfff700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 0x7f971c103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.065+0000 7f971b7fe700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 0x7f971c1051d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e9c0 tx=0x7f970c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.066+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c004220 con 0x7f971c101240 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.066+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c1a2600 con 0x7f971c101240 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.066+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970c0043c0 con 0x7f971c101240 2024-03-19T00:37:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.066+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c003c40 con 0x7f971c101240 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.066+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1a2b10 con 0x7f971c101240 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.066+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c068fd0 con 0x7f971c101240 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.068+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137340+0+0 (secure 0 0 0) 0x7f970c00c7c0 con 0x7f971c101240 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.068+0000 7f9718ff9700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f97040a2920 0x7f97040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.068+0000 7f971affd700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f97040a2920 0x7f97040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.068+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f970c0cf200 con 0x7f971c101240 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.069+0000 7f971affd700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f97040a2920 0x7f97040a4de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c00 tx=0x7f9710005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.072+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f970c09bd80 con 0x7f971c101240 2024-03-19T00:37:55.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.295+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f971c1a2dd0 con 0x7f97040a2920 2024-03-19T00:37:55.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.296+0000 7f9718ff9700 1 -- 172.21.15.7:0/2479348320 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f971c1a2dd0 con 0x7f97040a2920 2024-03-19T00:37:55.297 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-03-19T00:37:55.297 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-03-19T00:37:55.297 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f97040a2920 msgr2=0x7f97040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f97040a2920 0x7f97040a4de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c00 tx=0x7f9710005fb0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 msgr2=0x7f971c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 0x7f971c1051d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e9c0 tx=0x7f970c00ecd0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 shutdown_connections 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f97040a2920 0x7f97040a4de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f971c102de0 0x7f971c103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f971c101240 0x7f971c1051d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 --2- 172.21.15.7:0/2479348320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f971c102440 0x7f971c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:55.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 >> 172.21.15.7:0/2479348320 conn(0x7f971c0fc7e0 msgr2=0x7f971c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:55.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 shutdown_connections 2024-03-19T00:37:55.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:55.298+0000 7f9721cbf700 1 -- 172.21.15.7:0/2479348320 wait complete. 2024-03-19T00:37:55.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:55 smithi007 ceph-mon[62854]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:55 smithi044 ceph-mon[66366]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:56.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-19T00:37:56.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:55 smithi204 ceph-mon[66572]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:37:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:37:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:56 smithi044 ceph-mon[66366]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:56 smithi007 ceph-mon[62854]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:37:57.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:56 smithi204 ceph-mon[66572]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:37:58.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:57 smithi007 ceph-mon[62854]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:57 smithi044 ceph-mon[66366]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:57 smithi204 ceph-mon[66572]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 -- 172.21.15.7:0/4113590979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c105700 msgr2=0x7efe0c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 --2- 172.21.15.7:0/4113590979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c105700 0x7efe0c107af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe00009990 tx=0x7efe00009ca0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 -- 172.21.15.7:0/4113590979 shutdown_connections 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 --2- 172.21.15.7:0/4113590979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c108030 0x7efe0c10a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 --2- 172.21.15.7:0/4113590979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0c102dd0 0x7efe0c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 --2- 172.21.15.7:0/4113590979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c105700 0x7efe0c107af0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 -- 172.21.15.7:0/4113590979 >> 172.21.15.7:0/4113590979 conn(0x7efe0c0fc7e0 msgr2=0x7efe0c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 -- 172.21.15.7:0/4113590979 shutdown_connections 2024-03-19T00:37:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 -- 172.21.15.7:0/4113590979 wait complete. 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.311+0000 7efe137e6700 1 Processor -- start 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 -- start start 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0c102dd0 0x7efe0c102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 0x7efe0c100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 0x7efe0c1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c10de00 con 0x7efe0c108030 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c10dc80 con 0x7efe0c102dd0 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe137e6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c10df80 con 0x7efe0c105700 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe11d83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 0x7efe0c1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 0x7efe0c100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe11d83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 0x7efe0c1012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44732/0 (socket says 172.21.15.7:44732) 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe11d83700 1 -- 172.21.15.7:0/2240804434 learned_addr learned my addr 172.21.15.7:0/2240804434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 0x7efe0c100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:34692/0 (socket says 172.21.15.7:34692) 2024-03-19T00:37:58.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe11582700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0c102dd0 0x7efe0c102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 -- 172.21.15.7:0/2240804434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0c102dd0 msgr2=0x7efe0c102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0c102dd0 0x7efe0c102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 -- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 msgr2=0x7efe0c1012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 0x7efe0c1012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 -- 172.21.15.7:0/2240804434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c101b70 con 0x7efe0c105700 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe11d83700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 0x7efe0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:37:58.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.312+0000 7efe10d81700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 0x7efe0c100d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe00009990 tx=0x7efe000047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.313+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00027070 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.313+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c1a2610 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.313+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe00022470 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.313+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00017670 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.313+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c1a2a60 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.313+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0c068fd0 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.315+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7efe0002a070 con 0x7efe0c105700 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.315+0000 7efe0a7fc700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efdf40a2890 0x7efdf40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:37:58.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.315+0000 7efe11582700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efdf40a2890 0x7efdf40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:37:58.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.315+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efe000cfab0 con 0x7efe0c105700 2024-03-19T00:37:58.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.316+0000 7efe11582700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efdf40a2890 0x7efdf40a4d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe0400f540 tx=0x7efe0400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:37:58.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.319+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe00013070 con 0x7efe0c105700 2024-03-19T00:37:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.537+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7efe0c062810 con 0x7efdf40a2890 2024-03-19T00:37:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.550+0000 7efe0a7fc700 1 -- 172.21.15.7:0/2240804434 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7efe0c062810 con 0x7efdf40a2890 2024-03-19T00:37:58.551 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-19T00:37:58.551 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (2m) 20s ago 2m 16.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 0b271c0ad50e 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi044 running (2m) 56s ago 2m 26.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 0e16aa255c9a 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi204 running (2m) 56s ago 2m 19.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 8af9b6feaac4 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (2m) 20s ago 2m 16.4M - 19.0.0-2206-gb6ae2491 2312c8e42d00 0912a57a2241 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi044 running (2m) 56s ago 2m 16.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 ebebdf52cadc 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi204 running (2m) 56s ago 2m 15.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 4bf907b49498 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (2m) 20s ago 2m 12.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 3e4143e76ce2 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi044 running (2m) 56s ago 2m 19.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 e3640b3b6b0d 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi204 running (2m) 56s ago 2m 17.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 3a6f3fc144cc 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (2m) 20s ago 2m 13.8M - 19.0.0-2206-gb6ae2491 2312c8e42d00 c1ad7694215d 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi044 running (2m) 56s ago 2m 16.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 8d088867846d 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi204 running (2m) 56s ago 2m 15.4M - 19.0.0-2206-gb6ae2491 2312c8e42d00 939651d69de5 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (9m) 20s ago 9m 516M - 19.0.0-2206-gb6ae2491 2312c8e42d00 d6d7ca8e28f6 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi044 *:8443,8765 running (7m) 56s ago 7m 455M - 19.0.0-2206-gb6ae2491 2312c8e42d00 bad536000e72 2024-03-19T00:37:58.552 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi204 *:8443,8765 running (7m) 56s ago 7m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 8ac1793993d0 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (9m) 20s ago 9m 49.9M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 ffb69919e0d5 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi044 running (8m) 56s ago 8m 43.3M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 cbc1839b7815 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi204 running (8m) 56s ago 8m 39.3M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 adb740de161f 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 20s ago 7m 45.5M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 ddd238c9500f 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi044 running (7m) 56s ago 7m 53.7M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 5d26cf42c6db 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi204 running (6m) 56s ago 6m 54.2M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 4c813985425b 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (6m) 20s ago 6m 53.0M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 7b1e09cedb53 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi044 running (5m) 56s ago 5m 50.5M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 d20d6a76da49 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi204 running (5m) 56s ago 5m 52.8M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 da4399545a8d 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (5m) 20s ago 5m 50.0M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 94df4489e41e 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi044 running (4m) 56s ago 4m 55.7M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 71a1957287b6 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi204 running (4m) 56s ago 4m 54.7M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 3b85e986c1be 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (3m) 20s ago 3m 42.6M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 926356c8b5cc 2024-03-19T00:37:58.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi044 running (3m) 56s ago 3m 43.3M 4273M 19.0.0-2206-gb6ae2491 2312c8e42d00 fd889d526491 2024-03-19T00:37:58.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi204 running (2m) 56s ago 2m 51.5M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 175da5bba740 2024-03-19T00:37:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efdf40a2890 msgr2=0x7efdf40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efdf40a2890 0x7efdf40a4d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe0400f540 tx=0x7efe0400b040 comp rx=0 tx=0).stop 2024-03-19T00:37:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 msgr2=0x7efe0c100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:37:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 0x7efe0c100d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe00009990 tx=0x7efe000047b0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 shutdown_connections 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efdf40a2890 0x7efdf40a4d50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe0c108030 0x7efe0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0c102dd0 0x7efe0c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 --2- 172.21.15.7:0/2240804434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe0c105700 0x7efe0c100d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 >> 172.21.15.7:0/2240804434 conn(0x7efe0c0fc7e0 msgr2=0x7efe0c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 shutdown_connections 2024-03-19T00:37:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:37:58.552+0000 7efe137e6700 1 -- 172.21.15.7:0/2240804434 wait complete. 2024-03-19T00:37:59.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:58 smithi044 ceph-mon[66366]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-19T00:37:59.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:58 smithi007 ceph-mon[62854]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-19T00:37:59.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:58 smithi204 ceph-mon[66572]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-19T00:37:59.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-19T00:37:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:38:00.053 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:59 smithi007 ceph-mon[62854]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-19T00:38:00.053 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:37:59 smithi007 ceph-mon[62854]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:59 smithi044 ceph-mon[66366]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-19T00:38:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:37:59 smithi044 ceph-mon[66366]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:59 smithi204 ceph-mon[66572]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-19T00:38:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:37:59 smithi204 ceph-mon[66572]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:01.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 -- 172.21.15.7:0/2344255747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102440 msgr2=0x7f24041028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 --2- 172.21.15.7:0/2344255747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102440 0x7f24041028a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23f80099c0 tx=0x7f23f8009cd0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 -- 172.21.15.7:0/2344255747 shutdown_connections 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 --2- 172.21.15.7:0/2344255747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2404102de0 0x7f2404109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 --2- 172.21.15.7:0/2344255747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2404101240 0x7f2404101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 --2- 172.21.15.7:0/2344255747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102440 0x7f24041028a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 -- 172.21.15.7:0/2344255747 >> 172.21.15.7:0/2344255747 conn(0x7f24040fc7e0 msgr2=0x7f24040fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 -- 172.21.15.7:0/2344255747 shutdown_connections 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.414+0000 7f240c435700 1 -- 172.21.15.7:0/2344255747 wait complete. 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 Processor -- start 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 -- start start 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2404101240 0x7f24041051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2404102440 0x7f2404103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 0x7f2404103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f240410ccd0 con 0x7f2404102de0 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f240410cb50 con 0x7f2404102440 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240c435700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f240410ce50 con 0x7f2404101240 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 0x7f2404103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 0x7f2404103d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44748/0 (socket says 172.21.15.7:44748) 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 -- 172.21.15.7:0/947080634 learned_addr learned my addr 172.21.15.7:0/947080634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a1d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2404101240 0x7f24041051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f24099d0700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2404102440 0x7f2404103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 -- 172.21.15.7:0/947080634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2404101240 msgr2=0x7f24041051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2404101240 0x7f24041051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 -- 172.21.15.7:0/947080634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2404102440 msgr2=0x7f2404103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2404102440 0x7f2404103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.415+0000 7f240a9d2700 1 -- 172.21.15.7:0/947080634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2404104610 con 0x7f2404102de0 2024-03-19T00:38:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.416+0000 7f240a9d2700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 0x7f2404103d60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e940 tx=0x7f23fc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.416+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc004a30 con 0x7f2404102de0 2024-03-19T00:38:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.416+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23fc0054d0 con 0x7f2404102de0 2024-03-19T00:38:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.416+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24041a2610 con 0x7f2404102de0 2024-03-19T00:38:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.416+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00ccf0 con 0x7f2404102de0 2024-03-19T00:38:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.416+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a2b50 con 0x7f2404102de0 2024-03-19T00:38:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.417+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2404068fd0 con 0x7f2404102de0 2024-03-19T00:38:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.418+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f23fc005000 con 0x7f2404102de0 2024-03-19T00:38:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.419+0000 7f24037fe700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f23ec0a2910 0x7f23ec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.419+0000 7f240a1d1700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f23ec0a2910 0x7f23ec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.419+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f23fc0cf090 con 0x7f2404102de0 2024-03-19T00:38:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.419+0000 7f240a1d1700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f23ec0a2910 0x7f23ec0a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f24041022a0 tx=0x7f23f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.423+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f23fc053740 con 0x7f2404102de0 2024-03-19T00:38:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.637+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f24040fea20 con 0x7f23ec0a2910 2024-03-19T00:38:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.643+0000 7f24037fe700 1 -- 172.21.15.7:0/947080634 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f24040fea20 con 0x7f23ec0a2910 2024-03-19T00:38:01.644 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-19T00:38:01.644 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 59s ago 2m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-19T00:38:01.644 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 59s ago 8m smithi007=x;smithi044=y;smithi204=z;count:3 2024-03-19T00:38:01.644 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 59s ago 8m smithi007:172.21.15.7=a;smithi044:172.21.15.44=b;smithi204:172.21.15.204=c;count:3 2024-03-19T00:38:01.644 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 59s ago - 2024-03-19T00:38:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f23ec0a2910 msgr2=0x7f23ec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f23ec0a2910 0x7f23ec0a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f24041022a0 tx=0x7f23f400b040 comp rx=0 tx=0).stop 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 msgr2=0x7f2404103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 0x7f2404103d60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e940 tx=0x7f23fc00ec50 comp rx=0 tx=0).stop 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 shutdown_connections 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f23ec0a2910 0x7f23ec0a4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404102de0 0x7f2404103d60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2404101240 0x7f24041051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 --2- 172.21.15.7:0/947080634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2404102440 0x7f2404103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 >> 172.21.15.7:0/947080634 conn(0x7f24040fc7e0 msgr2=0x7f24040fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 shutdown_connections 2024-03-19T00:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:01.645+0000 7f240c435700 1 -- 172.21.15.7:0/947080634 wait complete. 2024-03-19T00:38:02.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:01 smithi044 ceph-mon[66366]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:01 smithi007 ceph-mon[62854]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:01 smithi204 ceph-mon[66572]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:02.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-19T00:38:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:38:03.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:02 smithi007 ceph-mon[62854]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:03.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:02 smithi044 ceph-mon[66366]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:03.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:02 smithi204 ceph-mon[66572]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:03 smithi044 ceph-mon[66366]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:04.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:03 smithi007 ceph-mon[62854]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:04.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:03 smithi204 ceph-mon[66572]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 -- 172.21.15.7:0/2843358332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c103920 msgr2=0x7fb87c103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2843358332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c103920 0x7fb87c103d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009990 tx=0x7fb86c009ca0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 -- 172.21.15.7:0/2843358332 shutdown_connections 2024-03-19T00:38:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2843358332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c100d40 0x7fb87c10a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2843358332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c103920 0x7fb87c103d40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2843358332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb87c1003a0 0x7fb87c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 -- 172.21.15.7:0/2843358332 >> 172.21.15.7:0/2843358332 conn(0x7fb87c0fbf80 msgr2=0x7fb87c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 -- 172.21.15.7:0/2843358332 shutdown_connections 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.370+0000 7fb8838e3700 1 -- 172.21.15.7:0/2843358332 wait complete. 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 Processor -- start 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 -- start start 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 0x7fb87c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb87c100d40 0x7fb87c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c103920 0x7fb87c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10ddc0 con 0x7fb87c1003a0 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10dc40 con 0x7fb87c100d40 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb8838e3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10df40 con 0x7fb87c103920 2024-03-19T00:38:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 0x7fb87c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 0x7fb87c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44770/0 (socket says 172.21.15.7:44770) 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 -- 172.21.15.7:0/2647229773 learned_addr learned my addr 172.21.15.7:0/2647229773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb880e7e700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb87c100d40 0x7fb87c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb881e80700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c103920 0x7fb87c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 -- 172.21.15.7:0/2647229773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c103920 msgr2=0x7fb87c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c103920 0x7fb87c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 -- 172.21.15.7:0/2647229773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb87c100d40 msgr2=0x7fb87c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb87c100d40 0x7fb87c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb88167f700 1 -- 172.21.15.7:0/2647229773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c1a0110 con 0x7fb87c1003a0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.371+0000 7fb881e80700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c103920 0x7fb87c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.372+0000 7fb88167f700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 0x7fb87c1984b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005b40 tx=0x7fb86c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.372+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c026070 con 0x7fb87c1003a0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.372+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb86c00fbb0 con 0x7fb87c1003a0 2024-03-19T00:38:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.372+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1a03a0 con 0x7fb87c1003a0 2024-03-19T00:38:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.372+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c017650 con 0x7fb87c1003a0 2024-03-19T00:38:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.372+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1a08d0 con 0x7fb87c1003a0 2024-03-19T00:38:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.373+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c068fd0 con 0x7fb87c1003a0 2024-03-19T00:38:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.374+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fb86c029070 con 0x7fb87c1003a0 2024-03-19T00:38:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.374+0000 7fb8727fc700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8680a2910 0x7fb8680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.375+0000 7fb880e7e700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8680a2910 0x7fb8680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.375+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb86c0d00f0 con 0x7fb87c1003a0 2024-03-19T00:38:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.375+0000 7fb880e7e700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8680a2910 0x7fb8680a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb878009c60 tx=0x7fb878009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:04.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.378+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb86c030080 con 0x7fb87c1003a0 2024-03-19T00:38:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.590+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb87c062750 con 0x7fb8680a2910 2024-03-19T00:38:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.592+0000 7fb8727fc700 1 -- 172.21.15.7:0/2647229773 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb87c062750 con 0x7fb8680a2910 2024-03-19T00:38:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-03-19T00:38:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-03-19T00:38:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:smithi044 172.21.15.44 2024-03-19T00:38:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:smithi204 172.21.15.204 2024-03-19T00:38:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8680a2910 msgr2=0x7fb8680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8680a2910 0x7fb8680a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb878009c60 tx=0x7fb878009260 comp rx=0 tx=0).stop 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 msgr2=0x7fb87c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 0x7fb87c1984b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005b40 tx=0x7fb86c005e80 comp rx=0 tx=0).stop 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 shutdown_connections 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8680a2910 0x7fb8680a4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb87c103920 0x7fb87c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c1003a0 0x7fb87c1984b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 --2- 172.21.15.7:0/2647229773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb87c100d40 0x7fb87c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 >> 172.21.15.7:0/2647229773 conn(0x7fb87c0fbf80 msgr2=0x7fb87c106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 shutdown_connections 2024-03-19T00:38:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:04.594+0000 7fb8838e3700 1 -- 172.21.15.7:0/2647229773 wait complete. 2024-03-19T00:38:05.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-19T00:38:05.555 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:38:06.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:05 smithi007 ceph-mon[62854]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:06.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:05 smithi007 ceph-mon[62854]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:06.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:05 smithi204 ceph-mon[66572]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:06.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:05 smithi204 ceph-mon[66572]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:06.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:05 smithi044 ceph-mon[66366]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:06.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:05 smithi044 ceph-mon[66366]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:07.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 -- 172.21.15.7:0/1364690535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0108090 msgr2=0x7efeb010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 --2- 172.21.15.7:0/1364690535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0108090 0x7efeb010a4a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efea8009990 tx=0x7efea8009ca0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 -- 172.21.15.7:0/1364690535 shutdown_connections 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 --2- 172.21.15.7:0/1364690535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0108090 0x7efeb010a4a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 --2- 172.21.15.7:0/1364690535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0102e30 0x7efeb0105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 --2- 172.21.15.7:0/1364690535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb0105760 0x7efeb0107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 -- 172.21.15.7:0/1364690535 >> 172.21.15.7:0/1364690535 conn(0x7efeb00fc7e0 msgr2=0x7efeb00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 -- 172.21.15.7:0/1364690535 shutdown_connections 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.297+0000 7efeb5827700 1 -- 172.21.15.7:0/1364690535 wait complete. 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 Processor -- start 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 -- start start 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 0x7efeb0196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0105760 0x7efeb0196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb0108090 0x7efeb01a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efeb010ddc0 con 0x7efeb0102e30 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efeb010dc40 con 0x7efeb0105760 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeb5827700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efeb010df40 con 0x7efeb0108090 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeaeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 0x7efeb0196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeaeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 0x7efeb0196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52222/0 (socket says 172.21.15.7:52222) 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.298+0000 7efeaeffd700 1 -- 172.21.15.7:0/405768656 learned_addr learned my addr 172.21.15.7:0/405768656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeae7fc700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0105760 0x7efeb0196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaf7fe700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb0108090 0x7efeb01a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaeffd700 1 -- 172.21.15.7:0/405768656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb0108090 msgr2=0x7efeb01a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaeffd700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb0108090 0x7efeb01a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaeffd700 1 -- 172.21.15.7:0/405768656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0105760 msgr2=0x7efeb0196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaeffd700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0105760 0x7efeb0196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaeffd700 1 -- 172.21.15.7:0/405768656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb01a6730 con 0x7efeb0102e30 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeae7fc700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0105760 0x7efeb0196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeaeffd700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 0x7efeb0196300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efea000ef10 tx=0x7efea000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea000c9c0 con 0x7efeb0102e30 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb01a69c0 con 0x7efeb0102e30 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efea0019070 con 0x7efeb0102e30 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0014480 con 0x7efeb0102e30 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.299+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb01a6f00 con 0x7efeb0102e30 2024-03-19T00:38:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.300+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0068fd0 con 0x7efeb0102e30 2024-03-19T00:38:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.301+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7efea0012050 con 0x7efeb0102e30 2024-03-19T00:38:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.302+0000 7efeb4825700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe9c0a2890 0x7efe9c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.302+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efea00cf3e0 con 0x7efeb0102e30 2024-03-19T00:38:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.302+0000 7efeae7fc700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe9c0a2890 0x7efe9c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.303+0000 7efeae7fc700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe9c0a2890 0x7efe9c0a4d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efeb0197880 tx=0x7efe98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:07.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.306+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efea0010040 con 0x7efeb0102e30 2024-03-19T00:38:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.521+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7efeb0062810 con 0x7efe9c0a2890 2024-03-19T00:38:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.523+0000 7efeb4825700 1 -- 172.21.15.7:0/405768656 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7efeb0062810 con 0x7efe9c0a2890 2024-03-19T00:38:07.524 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-19T00:38:07.524 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-19T00:38:07.524 INFO:teuthology.orchestra.run.smithi007.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-19T00:38:07.524 INFO:teuthology.orchestra.run.smithi007.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-19T00:38:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe9c0a2890 msgr2=0x7efe9c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe9c0a2890 0x7efe9c0a4d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efeb0197880 tx=0x7efe98009260 comp rx=0 tx=0).stop 2024-03-19T00:38:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 msgr2=0x7efeb0196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 0x7efeb0196300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efea000ef10 tx=0x7efea000c3a0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 shutdown_connections 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe9c0a2890 0x7efe9c0a4d50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb0108090 0x7efeb01a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0102e30 0x7efeb0196300 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 --2- 172.21.15.7:0/405768656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeb0105760 0x7efeb0196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 >> 172.21.15.7:0/405768656 conn(0x7efeb00fc7e0 msgr2=0x7efeb01025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 shutdown_connections 2024-03-19T00:38:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:07.525+0000 7efeb5827700 1 -- 172.21.15.7:0/405768656 wait complete. 2024-03-19T00:38:08.157 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:07 smithi044 ceph-mon[66366]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:38:08.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:07 smithi007 ceph-mon[62854]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:38:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:07 smithi204 ceph-mon[66572]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:38:08.356 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-19T00:38:08.367 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-19T00:38:08.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-19T00:38:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:38:09.003 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:08 smithi007 ceph-mon[62854]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:09.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:08 smithi204 ceph-mon[66572]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:09.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:08 smithi044 ceph-mon[66366]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:10.112 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:09 smithi007 ceph-mon[62854]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:10.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:09 smithi204 ceph-mon[66572]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 -- 172.21.15.7:0/999193037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 msgr2=0x7f0afc1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 --2- 172.21.15.7:0/999193037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc1039a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80099c0 tx=0x7f0ae8009cd0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 -- 172.21.15.7:0/999193037 shutdown_connections 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 --2- 172.21.15.7:0/999193037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc103ee0 0x7f0afc10a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 --2- 172.21.15.7:0/999193037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0afc102340 0x7f0afc102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 --2- 172.21.15.7:0/999193037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc1039a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 -- 172.21.15.7:0/999193037 >> 172.21.15.7:0/999193037 conn(0x7f0afc0fd920 msgr2=0x7f0afc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 -- 172.21.15.7:0/999193037 shutdown_connections 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.351+0000 7f0b02b74700 1 -- 172.21.15.7:0/999193037 wait complete. 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 Processor -- start 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 -- start start 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 0x7f0afc0760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0afc103ee0 0x7f0afc074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc10dde0 con 0x7f0afc103540 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc10dc60 con 0x7f0afc102340 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b02b74700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc10df60 con 0x7f0afc103ee0 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0afbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b00910700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 0x7f0afc0760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b01111700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0afc103ee0 0x7f0afc074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b00910700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 0x7f0afc0760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56096/0 (socket says 172.21.15.7:56096) 2024-03-19T00:38:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0b00910700 1 -- 172.21.15.7:0/1936629609 learned_addr learned my addr 172.21.15.7:0/1936629609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0afbfff700 1 -- 172.21.15.7:0/1936629609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0afc103ee0 msgr2=0x7f0afc074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0afbfff700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0afc103ee0 0x7f0afc074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.352+0000 7f0afbfff700 1 -- 172.21.15.7:0/1936629609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 msgr2=0x7f0afc0760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0afbfff700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 0x7f0afc0760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0afbfff700 1 -- 172.21.15.7:0/1936629609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc0754f0 con 0x7f0afc103540 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0b00910700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 0x7f0afc0760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0afbfff700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc074700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098e0 tx=0x7f0ae8003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8009e60 con 0x7f0afc103540 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc1a2610 con 0x7f0afc103540 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ae8003e50 con 0x7f0afc103540 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800fbe0 con 0x7f0afc103540 2024-03-19T00:38:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.353+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc1a2af0 con 0x7f0afc103540 2024-03-19T00:38:10.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:09 smithi044 ceph-mon[66366]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.354+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0afc068fd0 con 0x7f0afc103540 2024-03-19T00:38:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.355+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f0ae80052d0 con 0x7f0afc103540 2024-03-19T00:38:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.356+0000 7f0af9ffb700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aec0a2890 0x7f0aec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.356+0000 7f0b00910700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aec0a2890 0x7f0aec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.356+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0ae80cfaa0 con 0x7f0afc103540 2024-03-19T00:38:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.356+0000 7f0b00910700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aec0a2890 0x7f0aec0a4d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0af0005950 tx=0x7f0af0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:10.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.361+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ae8013070 con 0x7f0afc103540 2024-03-19T00:38:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.609+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0afc062f20 con 0x7f0afc103540 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.610+0000 7f0af9ffb700 1 -- 172.21.15.7:0/1936629609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3040 (secure 0 0 0) 0x7f0ae8005940 con 0x7f0afc103540 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:e24 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 24 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-19T00:38:10.611 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-19T00:35:59.576911+0000 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-19T00:36:26.295874+0000 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24521,1=24497,2=24490} 2024-03-19T00:38:10.612 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3,4] 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24521 members: 24521,24490,24497 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/2630641732,v1:172.21.15.204:6837/2630641732] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{1:24497} state up:active seq 11 addr [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.44:6834/1036080774,v1:172.21.15.44:6835/1036080774] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{2:24490} state up:active seq 11 addr [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{2:24454} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/3301743833,v1:172.21.15.204:6835/3301743833] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.7:6834/2506887195,v1:172.21.15.7:6835/2506887195] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.7:6836/520010551,v1:172.21.15.7:6837/520010551] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{-1:14643} state up:standby seq 1 addr [v2:172.21.15.7:6838/1114335301,v1:172.21.15.7:6839/1114335301] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{-1:14655} state up:standby seq 1 addr [v2:172.21.15.7:6840/3178899604,v1:172.21.15.7:6841/3178899604] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{-1:24403} state up:standby seq 1 addr [v2:172.21.15.204:6832/1644274763,v1:172.21.15.204:6833/1644274763] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.44:6832/1889634498,v1:172.21.15.44:6833/1889634498] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aec0a2890 msgr2=0x7f0aec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aec0a2890 0x7f0aec0a4d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0af0005950 tx=0x7f0af0009260 comp rx=0 tx=0).stop 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 msgr2=0x7f0afc074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc074700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098e0 tx=0x7f0ae8003b70 comp rx=0 tx=0).stop 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 shutdown_connections 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0aec0a2890 0x7f0aec0a4d50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0afc103ee0 0x7f0afc074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0afc102340 0x7f0afc0760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 --2- 172.21.15.7:0/1936629609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc103540 0x7f0afc074700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 >> 172.21.15.7:0/1936629609 conn(0x7f0afc0fd920 msgr2=0x7f0afc106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 shutdown_connections 2024-03-19T00:38:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:10.612+0000 7f0b02b74700 1 -- 172.21.15.7:0/1936629609 wait complete. 2024-03-19T00:38:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:38:11.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1936629609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T00:38:11.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:10 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1936629609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T00:38:11.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-19T00:38:11.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:10 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1936629609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T00:38:11.521 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:38:11.931 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:11 smithi007 ceph-mon[62854]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:11 smithi204 ceph-mon[66572]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:11 smithi044 ceph-mon[66366]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 -- 172.21.15.7:0/1991211701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 msgr2=0x7f4e540751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/1991211701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e540751f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009990 tx=0x7f4e48009ca0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 -- 172.21.15.7:0/1991211701 shutdown_connections 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/1991211701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 0x7f4e5410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/1991211701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e540747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/1991211701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e540751f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 -- 172.21.15.7:0/1991211701 >> 172.21.15.7:0/1991211701 conn(0x7f4e540fd920 msgr2=0x7f4e540ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 -- 172.21.15.7:0/1991211701 shutdown_connections 2024-03-19T00:38:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.292+0000 7f4e5a6af700 1 -- 172.21.15.7:0/1991211701 wait complete. 2024-03-19T00:38:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.293+0000 7f4e5a6af700 1 Processor -- start 2024-03-19T00:38:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.293+0000 7f4e5a6af700 1 -- start start 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e5a6af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e5419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e5a6af700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e5419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e5a6af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 0x7f4e541a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e5a6af700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410fff0 con 0x7f4e54074d70 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e5a6af700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410fe70 con 0x7f4e5410a210 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e5a6af700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54110170 con 0x7f4e54076340 2024-03-19T00:38:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e5419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e5419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e5419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:54330/0 (socket says 172.21.15.7:54330) 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e5419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52268/0 (socket says 172.21.15.7:52268) 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 -- 172.21.15.7:0/3346950623 learned_addr learned my addr 172.21.15.7:0/3346950623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e58c4c700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 0x7f4e541a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 -- 172.21.15.7:0/3346950623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 msgr2=0x7f4e541a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 0x7f4e541a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 -- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 msgr2=0x7f4e5419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e5419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 -- 172.21.15.7:0/3346950623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e541a2370 con 0x7f4e54076340 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e53fff700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e5419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e58c4c700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 0x7f4e541a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.294+0000 7f4e537fe700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e5419aca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.295+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48026070 con 0x7f4e54076340 2024-03-19T00:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.295+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e48021410 con 0x7f4e54076340 2024-03-19T00:38:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.295+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e541a2600 con 0x7f4e54076340 2024-03-19T00:38:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.295+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48017650 con 0x7f4e54076340 2024-03-19T00:38:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.295+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a2b30 con 0x7f4e54076340 2024-03-19T00:38:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.296+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e40005360 con 0x7f4e54076340 2024-03-19T00:38:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.297+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f4e48029070 con 0x7f4e54076340 2024-03-19T00:38:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.298+0000 7f4e517fa700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4e3c0a2910 0x7f4e3c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.298+0000 7f4e53fff700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4e3c0a2910 0x7f4e3c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.298+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4e480cfb90 con 0x7f4e54076340 2024-03-19T00:38:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.298+0000 7f4e53fff700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4e3c0a2910 0x7f4e3c0a4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4e54076720 tx=0x7f4e44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.301+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4e48013070 con 0x7f4e54076340 2024-03-19T00:38:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.509+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4e40005920 con 0x7f4e54076340 2024-03-19T00:38:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.510+0000 7f4e517fa700 1 -- 172.21.15.7:0/3346950623 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5543 (secure 0 0 0) 0x7f4e4809c960 con 0x7f4e54076340 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 91 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-19T00:28:02.904625+0000 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-19T00:36:57.940744+0000 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 27 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-03-19T00:38:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/2757458659,v1:172.21.15.7:6803/2757458659] [v2:172.21.15.7:6804/2757458659,v1:172.21.15.7:6805/2757458659] exists,up 7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/274970163,v1:172.21.15.44:6801/274970163] [v2:172.21.15.44:6802/274970163,v1:172.21.15.44:6803/274970163] exists,up baa358f6-e033-488f-9078-2a0c3b3c38f1 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/281046450,v1:172.21.15.204:6801/281046450] [v2:172.21.15.204:6802/281046450,v1:172.21.15.204:6803/281046450] exists,up c4bd1927-870b-4e32-ad51-aff87143cb46 2024-03-19T00:38:13.512 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/375379923,v1:172.21.15.7:6811/375379923] [v2:172.21.15.7:6812/375379923,v1:172.21.15.7:6813/375379923] exists,up 0e3811a3-7436-489a-ac0a-857023269d16 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/3332813314,v1:172.21.15.44:6809/3332813314] [v2:172.21.15.44:6810/3332813314,v1:172.21.15.44:6811/3332813314] exists,up 993175bf-d87f-4369-9ded-8b968d4b9160 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/3713019537,v1:172.21.15.204:6809/3713019537] [v2:172.21.15.204:6810/3713019537,v1:172.21.15.204:6811/3713019537] exists,up 3a893438-b29c-4c46-bbe8-ba8ac823e8dc 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/3051876259,v1:172.21.15.7:6819/3051876259] [v2:172.21.15.7:6820/3051876259,v1:172.21.15.7:6821/3051876259] exists,up 9dfd817f-972c-4c18-8d94-76aebea473aa 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6816/48320224,v1:172.21.15.44:6817/48320224] [v2:172.21.15.44:6818/48320224,v1:172.21.15.44:6819/48320224] exists,up f17cf883-f837-421c-8306-e1091b9b5f73 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 59 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/1654194341,v1:172.21.15.204:6817/1654194341] [v2:172.21.15.204:6818/1654194341,v1:172.21.15.204:6819/1654194341] exists,up 295a9edc-0de0-4302-8887-8717d45ba545 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/1022710322,v1:172.21.15.7:6827/1022710322] [v2:172.21.15.7:6828/1022710322,v1:172.21.15.7:6829/1022710322] exists,up 3844148f-1d07-4f72-a3c0-e249888e8416 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6824/443402515,v1:172.21.15.44:6825/443402515] [v2:172.21.15.44:6826/443402515,v1:172.21.15.44:6827/443402515] exists,up 88ad1139-05b0-45d2-938e-bae6421c1a4c 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/849720251,v1:172.21.15.204:6825/849720251] [v2:172.21.15.204:6826/849720251,v1:172.21.15.204:6827/849720251] exists,up 1b004ee9-0ccb-4258-bc42-4b24baf7931f 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.6 [8,11] 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.8 [2,11] 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.1d [5,9] 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.23 [4,8] 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.2c [4,9] 2024-03-19T00:38:13.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.34 [4,9] 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.e [9,11] 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.1c [9,8] 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.1d [5,10] 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.1f [9,8] 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1277095143 expires 2024-03-20T00:28:55.916728+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/4032873653 expires 2024-03-20T00:28:55.916728+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/501265234 expires 2024-03-20T00:28:55.916728+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/501265234 expires 2024-03-20T00:28:55.916728+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1320706322 expires 2024-03-20T00:28:30.106926+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/237878025 expires 2024-03-20T00:28:30.106926+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/1237634613 expires 2024-03-20T00:28:30.106926+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/1237634613 expires 2024-03-20T00:28:30.106926+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3660465780 expires 2024-03-20T00:28:30.106926+0000 2024-03-19T00:38:13.514 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/97837638 expires 2024-03-20T00:28:55.916728+0000 2024-03-19T00:38:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4e3c0a2910 msgr2=0x7f4e3c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4e3c0a2910 0x7f4e3c0a4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4e54076720 tx=0x7f4e44009260 comp rx=0 tx=0).stop 2024-03-19T00:38:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 msgr2=0x7f4e5419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e5419aca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48009580 comp rx=0 tx=0).stop 2024-03-19T00:38:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 shutdown_connections 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4e3c0a2910 0x7f4e3c0a4dd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e5410a210 0x7f4e541a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e54074d70 0x7f4e5419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 --2- 172.21.15.7:0/3346950623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e54076340 0x7f4e5419aca0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 >> 172.21.15.7:0/3346950623 conn(0x7f4e540fd920 msgr2=0x7f4e54106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 shutdown_connections 2024-03-19T00:38:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:13.512+0000 7f4e5a6af700 1 -- 172.21.15.7:0/3346950623 wait complete. 2024-03-19T00:38:14.179 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:13 smithi007 ceph-mon[62854]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:14.179 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:13 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3346950623' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T00:38:14.207 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-19T00:38:14.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:13 smithi204 ceph-mon[66572]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:13 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3346950623' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 -- 172.21.15.7:0/3039884768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8100750 msgr2=0x7faff8100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 --2- 172.21.15.7:0/3039884768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8100750 0x7faff8100b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009900 tx=0x7fafe802bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 -- 172.21.15.7:0/3039884768 shutdown_connections 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 --2- 172.21.15.7:0/3039884768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faff8102300 0x7faff8108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 --2- 172.21.15.7:0/3039884768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8101940 0x7faff8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 --2- 172.21.15.7:0/3039884768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8100750 0x7faff8100b70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 -- 172.21.15.7:0/3039884768 >> 172.21.15.7:0/3039884768 conn(0x7faff80fbce0 msgr2=0x7faff80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 -- 172.21.15.7:0/3039884768 shutdown_connections 2024-03-19T00:38:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.322+0000 7faffe08c640 1 -- 172.21.15.7:0/3039884768 wait complete. 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 Processor -- start 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 -- start start 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8100750 0x7faff8199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faff8101940 0x7faff819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 0x7faff81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faff810c280 con 0x7faff8102300 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faff810c100 con 0x7faff8100750 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faffe08c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faff810c400 con 0x7faff8101940 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faff7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 0x7faff81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faff7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 0x7faff81a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52276/0 (socket says 172.21.15.7:52276) 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faff7fff640 1 -- 172.21.15.7:0/3254664502 learned_addr learned my addr 172.21.15.7:0/3254664502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faff77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8100750 0x7faff8199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.323+0000 7faff6ffd640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faff8101940 0x7faff819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff7fff640 1 -- 172.21.15.7:0/3254664502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faff8101940 msgr2=0x7faff819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff7fff640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faff8101940 0x7faff819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff7fff640 1 -- 172.21.15.7:0/3254664502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8100750 msgr2=0x7faff8199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff7fff640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8100750 0x7faff8199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff7fff640 1 -- 172.21.15.7:0/3254664502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff81a1900 con 0x7faff8102300 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff7fff640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 0x7faff81a1260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faff000eb10 tx=0x7faff000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0004b20 con 0x7faff8102300 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faff0004cc0 con 0x7faff8102300 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0010690 con 0x7faff8102300 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff81a1bf0 con 0x7faff8102300 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.324+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a2110 con 0x7faff8102300 2024-03-19T00:38:14.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.325+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff81a22e0 con 0x7faff8102300 2024-03-19T00:38:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.326+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7faff0005020 con 0x7faff8102300 2024-03-19T00:38:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.327+0000 7faff4ff9640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafe00a2800 0x7fafe00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.327+0000 7faff77fe640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafe00a2800 0x7fafe00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.327+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faff00cf510 con 0x7faff8102300 2024-03-19T00:38:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.327+0000 7faff77fe640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafe00a2800 0x7fafe00a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005ab0 tx=0x7fafe8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.331+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faff009c250 con 0x7faff8102300 2024-03-19T00:38:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:13 smithi044 ceph-mon[66366]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:13 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3346950623' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T00:38:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.515+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faff81a24b0 con 0x7faff8102300 2024-03-19T00:38:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.517+0000 7faff4ff9640 1 -- 172.21.15.7:0/3254664502 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7faff009bb50 con 0x7faff8102300 2024-03-19T00:38:14.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:14.518 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.518+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafe00a2800 msgr2=0x7fafe00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.518+0000 7faffe08c640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafe00a2800 0x7fafe00a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005ab0 tx=0x7fafe8005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.518+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 msgr2=0x7faff81a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 0x7faff81a1260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faff000eb10 tx=0x7faff000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 shutdown_connections 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafe00a2800 0x7fafe00a4cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faff8102300 0x7faff81a1260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faff8101940 0x7faff819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 --2- 172.21.15.7:0/3254664502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8100750 0x7faff8199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 >> 172.21.15.7:0/3254664502 conn(0x7faff80fbce0 msgr2=0x7faff80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 shutdown_connections 2024-03-19T00:38:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.519+0000 7faffe08c640 1 -- 172.21.15.7:0/3254664502 wait complete. 2024-03-19T00:38:14.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:38:14.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 -- 172.21.15.7:0/2662848827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb268104c90 msgr2=0x7fb268107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 --2- 172.21.15.7:0/2662848827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb268104c90 0x7fb268107080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb250009900 tx=0x7fb25002bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 -- 172.21.15.7:0/2662848827 shutdown_connections 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 --2- 172.21.15.7:0/2662848827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2681075c0 0x7fb2681099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 --2- 172.21.15.7:0/2662848827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb268104c90 0x7fb268107080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 --2- 172.21.15.7:0/2662848827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 -- 172.21.15.7:0/2662848827 >> 172.21.15.7:0/2662848827 conn(0x7fb2680fbcb0 msgr2=0x7fb2680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 -- 172.21.15.7:0/2662848827 shutdown_connections 2024-03-19T00:38:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.636+0000 7fb26cedc640 1 -- 172.21.15.7:0/2662848827 wait complete. 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 Processor -- start 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 -- start start 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb268104c90 0x7fb268198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2681075c0 0x7fb26819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810d340 con 0x7fb268102360 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810d1c0 con 0x7fb2681075c0 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26cedc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810d4c0 con 0x7fb268104c90 2024-03-19T00:38:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46160/0 (socket says 172.21.15.7:46160) 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 -- 172.21.15.7:0/651818810 learned_addr learned my addr 172.21.15.7:0/651818810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb266d9d640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2681075c0 0x7fb26819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb25ffff640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb268104c90 0x7fb268198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 -- 172.21.15.7:0/651818810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb268104c90 msgr2=0x7fb268198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb268104c90 0x7fb268198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 -- 172.21.15.7:0/651818810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2681075c0 msgr2=0x7fb26819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2681075c0 0x7fb26819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.637+0000 7fb26659c640 1 -- 172.21.15.7:0/651818810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26819f700 con 0x7fb268102360 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.638+0000 7fb26659c640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268197ae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb25800e9f0 tx=0x7fb25800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.638+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25800ccc0 con 0x7fb268102360 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.638+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26819f9f0 con 0x7fb268102360 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.638+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2580108e0 con 0x7fb268102360 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.638+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2580187c0 con 0x7fb268102360 2024-03-19T00:38:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.638+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26819ff60 con 0x7fb268102360 2024-03-19T00:38:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.639+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2680fd7a0 con 0x7fb268102360 2024-03-19T00:38:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.640+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fb258010a90 con 0x7fb268102360 2024-03-19T00:38:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.641+0000 7fb25f7fe640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb2540a2800 0x7fb2540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.641+0000 7fb25ffff640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb2540a2800 0x7fb2540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.641+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb2580cf650 con 0x7fb268102360 2024-03-19T00:38:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.641+0000 7fb25ffff640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb2540a2800 0x7fb2540a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb250005ab0 tx=0x7fb250005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.644+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb25809c330 con 0x7fb268102360 2024-03-19T00:38:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.827+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb26802d4a0 con 0x7fb268102360 2024-03-19T00:38:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.827+0000 7fb25f7fe640 1 -- 172.21.15.7:0/651818810 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fb25809bc30 con 0x7fb268102360 2024-03-19T00:38:14.828 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:38:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb2540a2800 msgr2=0x7fb2540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb2540a2800 0x7fb2540a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb250005ab0 tx=0x7fb250005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 msgr2=0x7fb268197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268197ae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb25800e9f0 tx=0x7fb25800edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 shutdown_connections 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb2540a2800 0x7fb2540a4cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2681075c0 0x7fb26819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb268104c90 0x7fb268198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 --2- 172.21.15.7:0/651818810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb268102360 0x7fb268197ae0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 >> 172.21.15.7:0/651818810 conn(0x7fb2680fbcb0 msgr2=0x7fb268104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 shutdown_connections 2024-03-19T00:38:14.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.829+0000 7fb26cedc640 1 -- 172.21.15.7:0/651818810 wait complete. 2024-03-19T00:38:14.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:38:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 -- 172.21.15.7:0/1737800910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04100740 msgr2=0x7f7e04100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 --2- 172.21.15.7:0/1737800910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04100740 0x7f7e04100b60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009900 tx=0x7f7e0002bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 -- 172.21.15.7:0/1737800910 shutdown_connections 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 --2- 172.21.15.7:0/1737800910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e041022f0 0x7f7e04108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 --2- 172.21.15.7:0/1737800910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e04101930 0x7f7e04101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 --2- 172.21.15.7:0/1737800910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04100740 0x7f7e04100b60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 -- 172.21.15.7:0/1737800910 >> 172.21.15.7:0/1737800910 conn(0x7f7e040fbcb0 msgr2=0x7f7e040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 -- 172.21.15.7:0/1737800910 shutdown_connections 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 -- 172.21.15.7:0/1737800910 wait complete. 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 Processor -- start 2024-03-19T00:38:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 -- start start 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e04100740 0x7f7e04197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.947+0000 7f7e0b753640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04101930 0x7f7e04197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e0b753640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 0x7f7e0419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e0b753640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0410c260 con 0x7f7e041022f0 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e0b753640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0410c0e0 con 0x7f7e04101930 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e0b753640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0410c3e0 con 0x7f7e04100740 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 0x7f7e0419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 0x7f7e0419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46172/0 (socket says 172.21.15.7:46172) 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e094ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e04100740 0x7f7e04197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 -- 172.21.15.7:0/2147830032 learned_addr learned my addr 172.21.15.7:0/2147830032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e08cee640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04101930 0x7f7e04197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 -- 172.21.15.7:0/2147830032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e04100740 msgr2=0x7f7e04197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e04100740 0x7f7e04197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 -- 172.21.15.7:0/2147830032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04101930 msgr2=0x7f7e04197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04101930 0x7f7e04197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 -- 172.21.15.7:0/2147830032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0419f6d0 con 0x7f7e041022f0 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e09cf0640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 0x7f7e0419f060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00eb10 tx=0x7f7dfc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc004b20 con 0x7f7e041022f0 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dfc004cc0 con 0x7f7e041022f0 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0419f9c0 con 0x7f7e041022f0 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc010690 con 0x7f7e041022f0 2024-03-19T00:38:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.948+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0419ff10 con 0x7f7e041022f0 2024-03-19T00:38:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.949+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e041a01a0 con 0x7f7e041022f0 2024-03-19T00:38:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.950+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f7dfc005000 con 0x7f7e041022f0 2024-03-19T00:38:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.951+0000 7f7dfa7fc640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df00a2880 0x7f7df00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.951+0000 7f7e094ef640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df00a2880 0x7f7df00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.951+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7dfc0cf510 con 0x7f7e041022f0 2024-03-19T00:38:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.951+0000 7f7e094ef640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df00a2880 0x7f7df00a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e00005ab0 tx=0x7f7e00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:14.954+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7dfc053740 con 0x7f7e041022f0 2024-03-19T00:38:15.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.136+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e041a0370 con 0x7f7e041022f0 2024-03-19T00:38:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.137+0000 7f7dfa7fc640 1 -- 172.21.15.7:0/2147830032 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f7dfc09c1f0 con 0x7f7e041022f0 2024-03-19T00:38:15.138 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df00a2880 msgr2=0x7f7df00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df00a2880 0x7f7df00a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e00005ab0 tx=0x7f7e00005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 msgr2=0x7f7e0419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 0x7f7e0419f060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00eb10 tx=0x7f7dfc00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 shutdown_connections 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df00a2880 0x7f7df00a4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e041022f0 0x7f7e0419f060 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04101930 0x7f7e04197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 --2- 172.21.15.7:0/2147830032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e04100740 0x7f7e04197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 >> 172.21.15.7:0/2147830032 conn(0x7f7e040fbcb0 msgr2=0x7f7e04069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 shutdown_connections 2024-03-19T00:38:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.139+0000 7f7e0b753640 1 -- 172.21.15.7:0/2147830032 wait complete. 2024-03-19T00:38:15.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3254664502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:15.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:14 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/651818810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:15.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:38:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 -- 172.21.15.7:0/2288299859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c101930 msgr2=0x7f038c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 --2- 172.21.15.7:0/2288299859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c101930 0x7f038c101db0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0378009900 tx=0x7f037802bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 -- 172.21.15.7:0/2288299859 shutdown_connections 2024-03-19T00:38:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 --2- 172.21.15.7:0/2288299859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 0x7f038c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 --2- 172.21.15.7:0/2288299859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c101930 0x7f038c101db0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 --2- 172.21.15.7:0/2288299859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f038c100740 0x7f038c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 -- 172.21.15.7:0/2288299859 >> 172.21.15.7:0/2288299859 conn(0x7f038c0fbcb0 msgr2=0x7f038c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 -- 172.21.15.7:0/2288299859 shutdown_connections 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.257+0000 7f03931e8640 1 -- 172.21.15.7:0/2288299859 wait complete. 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 Processor -- start 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 -- start start 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 0x7f038c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f038c101930 0x7f038c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 0x7f038c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f038c10c270 con 0x7f038c100740 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f038c10c0f0 con 0x7f038c1022f0 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f03931e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f038c10c3f0 con 0x7f038c101930 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 0x7f038c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 0x7f038c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46188/0 (socket says 172.21.15.7:46188) 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 -- 172.21.15.7:0/3709420365 learned_addr learned my addr 172.21.15.7:0/3709420365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0391785640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 0x7f038c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f038bfff640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f038c101930 0x7f038c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 -- 172.21.15.7:0/3709420365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f038c101930 msgr2=0x7f038c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f038c101930 0x7f038c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 -- 172.21.15.7:0/3709420365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 msgr2=0x7f038c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 0x7f038c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0390f84640 1 -- 172.21.15.7:0/3709420365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c19f700 con 0x7f038c100740 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.258+0000 7f0391785640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 0x7f038c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.259+0000 7f0390f84640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 0x7f038c197ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f038000e9c0 tx=0x7f038000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:15.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.259+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038000ccc0 con 0x7f038c100740 2024-03-19T00:38:15.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.259+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03800035f0 con 0x7f038c100740 2024-03-19T00:38:15.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.259+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038c19f9f0 con 0x7f038c100740 2024-03-19T00:38:15.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.259+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038000ce60 con 0x7f038c100740 2024-03-19T00:38:15.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.259+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038c19fec0 con 0x7f038c100740 2024-03-19T00:38:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.260+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038c1a0090 con 0x7f038c100740 2024-03-19T00:38:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.261+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f03800037a0 con 0x7f038c100740 2024-03-19T00:38:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.262+0000 7f0389ffb640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f037c0a2800 0x7f037c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.262+0000 7f038bfff640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f037c0a2800 0x7f037c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.262+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f03800cf2f0 con 0x7f038c100740 2024-03-19T00:38:15.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.262+0000 7f038bfff640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f037c0a2800 0x7f037c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0378009230 tx=0x7f0378005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:15.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3254664502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:15.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:14 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/651818810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.266+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f038009bfd0 con 0x7f038c100740 2024-03-19T00:38:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3254664502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:15.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:14 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/651818810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.451+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f038c1a0260 con 0x7f038c100740 2024-03-19T00:38:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.451+0000 7f0389ffb640 1 -- 172.21.15.7:0/3709420365 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f038009b8d0 con 0x7f038c100740 2024-03-19T00:38:15.452 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f037c0a2800 msgr2=0x7f037c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f037c0a2800 0x7f037c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0378009230 tx=0x7f0378005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 msgr2=0x7f038c197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 0x7f038c197ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f038000e9c0 tx=0x7f038000edc0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 shutdown_connections 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f037c0a2800 0x7f037c0a4cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f038c1022f0 0x7f038c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f038c101930 0x7f038c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 --2- 172.21.15.7:0/3709420365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f038c100740 0x7f038c197ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 >> 172.21.15.7:0/3709420365 conn(0x7f038c0fbcb0 msgr2=0x7f038c069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 shutdown_connections 2024-03-19T00:38:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.453+0000 7f03931e8640 1 -- 172.21.15.7:0/3709420365 wait complete. 2024-03-19T00:38:15.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/3827335288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 msgr2=0x7fb7cc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/3827335288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc108b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009ca0 tx=0x7fb7c402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/3827335288 shutdown_connections 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/3827335288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc108b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/3827335288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc101930 0x7fb7cc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/3827335288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7cc100740 0x7fb7cc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/3827335288 >> 172.21.15.7:0/3827335288 conn(0x7fb7cc0fbcb0 msgr2=0x7fb7cc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/3827335288 shutdown_connections 2024-03-19T00:38:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.570+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/3827335288 wait complete. 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 Processor -- start 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 -- start start 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 0x7fb7cc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7cc101930 0x7fb7cc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10c250 con 0x7fb7cc100740 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10c0d0 con 0x7fb7cc1022f0 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7d0e8c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10c3d0 con 0x7fb7cc101930 2024-03-19T00:38:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 0x7fb7cc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 0x7fb7cc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46206/0 (socket says 172.21.15.7:46206) 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7ca59c640 1 -- 172.21.15.7:0/747270918 learned_addr learned my addr 172.21.15.7:0/747270918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7c3fff640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7cc101930 0x7fb7cc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 -- 172.21.15.7:0/747270918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7cc101930 msgr2=0x7fb7cc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7cc101930 0x7fb7cc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 -- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 msgr2=0x7fb7cc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 0x7fb7cc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 -- 172.21.15.7:0/747270918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc19f6c0 con 0x7fb7cc1022f0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7ca59c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 0x7fb7cc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.571+0000 7fb7cad9d640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc19f050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7c402c630 tx=0x7fb7c40328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.572+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4041020 con 0x7fb7cc1022f0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.572+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7c4003e20 con 0x7fb7cc1022f0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.572+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc19f950 con 0x7fb7cc1022f0 2024-03-19T00:38:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.572+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4039040 con 0x7fb7cc1022f0 2024-03-19T00:38:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.572+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc19fe90 con 0x7fb7cc1022f0 2024-03-19T00:38:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.573+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc1a00c0 con 0x7fb7cc1022f0 2024-03-19T00:38:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.574+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fb7c4044020 con 0x7fb7cc1022f0 2024-03-19T00:38:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.574+0000 7fb7c37fe640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7b80a2880 0x7fb7b80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.574+0000 7fb7ca59c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7b80a2880 0x7fb7b80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.574+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7c40f1290 con 0x7fb7cc1022f0 2024-03-19T00:38:15.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.575+0000 7fb7ca59c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7b80a2880 0x7fb7b80a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0058c0 tx=0x7fb7bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.578+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb7c404b020 con 0x7fb7cc1022f0 2024-03-19T00:38:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.761+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb7cc1a0290 con 0x7fb7cc1022f0 2024-03-19T00:38:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.762+0000 7fb7c37fe640 1 -- 172.21.15.7:0/747270918 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fb7c40bdfd0 con 0x7fb7cc1022f0 2024-03-19T00:38:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7b80a2880 msgr2=0x7fb7b80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7b80a2880 0x7fb7b80a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0058c0 tx=0x7fb7bc005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 msgr2=0x7fb7cc19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc19f050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7c402c630 tx=0x7fb7c40328a0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 shutdown_connections 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7b80a2880 0x7fb7b80a4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7cc1022f0 0x7fb7cc19f050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7cc101930 0x7fb7cc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 --2- 172.21.15.7:0/747270918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7cc100740 0x7fb7cc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 >> 172.21.15.7:0/747270918 conn(0x7fb7cc0fbcb0 msgr2=0x7fb7cc0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 shutdown_connections 2024-03-19T00:38:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.764+0000 7fb7d0e8c640 1 -- 172.21.15.7:0/747270918 wait complete. 2024-03-19T00:38:15.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:38:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 -- 172.21.15.7:0/1007430180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704102370 msgr2=0x7f2704104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 --2- 172.21.15.7:0/1007430180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704102370 0x7f2704104760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009900 tx=0x7f26f402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 -- 172.21.15.7:0/1007430180 shutdown_connections 2024-03-19T00:38:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 --2- 172.21.15.7:0/1007430180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27041075d0 0x7f27041099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 --2- 172.21.15.7:0/1007430180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2704104ca0 0x7f2704107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 --2- 172.21.15.7:0/1007430180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704102370 0x7f2704104760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 -- 172.21.15.7:0/1007430180 >> 172.21.15.7:0/1007430180 conn(0x7f27040fbcc0 msgr2=0x7f27040fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 -- 172.21.15.7:0/1007430180 shutdown_connections 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 -- 172.21.15.7:0/1007430180 wait complete. 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 Processor -- start 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.879+0000 7f270a675640 1 -- start start 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f270a675640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704102370 0x7f2704197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f270a675640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 0x7f2704197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f270a675640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27041075d0 0x7f270419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f270a675640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f270410d390 con 0x7f2704104ca0 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f270a675640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f270410d210 con 0x7f27041075d0 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f270a675640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f270410d510 con 0x7f2704102370 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 0x7f2704197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 0x7f2704197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46228/0 (socket says 172.21.15.7:46228) 2024-03-19T00:38:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 -- 172.21.15.7:0/2511322806 learned_addr learned my addr 172.21.15.7:0/2511322806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f2708c12640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27041075d0 0x7f270419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f2703fff640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704102370 0x7f2704197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 -- 172.21.15.7:0/2511322806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704102370 msgr2=0x7f2704197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704102370 0x7f2704197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 -- 172.21.15.7:0/2511322806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27041075d0 msgr2=0x7f270419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27041075d0 0x7f270419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 -- 172.21.15.7:0/2511322806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270419f6b0 con 0x7f2704104ca0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.880+0000 7f27037fe640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 0x7f2704197fd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f26f800e9a0 tx=0x7f26f800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.881+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f800cd80 con 0x7f2704104ca0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.881+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26f80035f0 con 0x7f2704104ca0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.881+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270419f9a0 con 0x7f2704104ca0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.881+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8010690 con 0x7f2704104ca0 2024-03-19T00:38:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.881+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270419ff40 con 0x7f2704104ca0 2024-03-19T00:38:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.882+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27040fd7b0 con 0x7f2704104ca0 2024-03-19T00:38:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.883+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f26f80037a0 con 0x7f2704104ca0 2024-03-19T00:38:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.883+0000 7f27017fa640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2880 0x7f26ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.883+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f26f80cefa0 con 0x7f2704104ca0 2024-03-19T00:38:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.883+0000 7f2703fff640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2880 0x7f26ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.884+0000 7f2703fff640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2880 0x7f26ec0a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009230 tx=0x7f26f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:15.888+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26f809bce0 con 0x7f2704104ca0 2024-03-19T00:38:16.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:15 smithi007 ceph-mon[62854]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:38:16.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2147830032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3709420365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/747270918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.110+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f270402d1f0 con 0x7f2704104ca0 2024-03-19T00:38:16.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.111+0000 7f27017fa640 1 -- 172.21.15.7:0/2511322806 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f26f809b5e0 con 0x7f2704104ca0 2024-03-19T00:38:16.112 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:16.112 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:26.295874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:38:16.112 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:38:16.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2880 msgr2=0x7f26ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2880 0x7f26ec0a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009230 tx=0x7f26f4005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 msgr2=0x7f2704197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 0x7f2704197fd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f26f800e9a0 tx=0x7f26f800ee60 comp rx=0 tx=0).stop 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 shutdown_connections 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2880 0x7f26ec0a4d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27041075d0 0x7f270419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2704104ca0 0x7f2704197fd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 --2- 172.21.15.7:0/2511322806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2704102370 0x7f2704197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 >> 172.21.15.7:0/2511322806 conn(0x7f27040fbcc0 msgr2=0x7f2704102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:16.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 shutdown_connections 2024-03-19T00:38:16.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.113+0000 7f270a675640 1 -- 172.21.15.7:0/2511322806 wait complete. 2024-03-19T00:38:16.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/3463426019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58100750 msgr2=0x7fcb58100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/3463426019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58100750 0x7fcb58100b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009900 tx=0x7fcb5402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/3463426019 shutdown_connections 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/3463426019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58102300 0x7fcb58108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/3463426019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58101940 0x7fcb58101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/3463426019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58100750 0x7fcb58100b70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/3463426019 >> 172.21.15.7:0/3463426019 conn(0x7fcb580fbcc0 msgr2=0x7fcb580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/3463426019 shutdown_connections 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.229+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/3463426019 wait complete. 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 Processor -- start 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 -- start start 2024-03-19T00:38:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58100750 0x7fcb58197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 0x7fcb58197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58102300 0x7fcb5819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5810c2d0 con 0x7fcb58101940 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5810c150 con 0x7fcb58102300 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5fa7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5810c450 con 0x7fcb58100750 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 0x7fcb58197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 0x7fcb58197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46244/0 (socket says 172.21.15.7:46244) 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 -- 172.21.15.7:0/2019308283 learned_addr learned my addr 172.21.15.7:0/2019308283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d81a640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58100750 0x7fcb58197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5e01b640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58102300 0x7fcb5819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 -- 172.21.15.7:0/2019308283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58100750 msgr2=0x7fcb58197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58100750 0x7fcb58197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 -- 172.21.15.7:0/2019308283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58102300 msgr2=0x7fcb5819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58102300 0x7fcb5819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.230+0000 7fcb5d019640 1 -- 172.21.15.7:0/2019308283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5819f680 con 0x7fcb58101940 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.231+0000 7fcb5d019640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 0x7fcb58197fa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800e9f0 tx=0x7fcb4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.231+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800cd80 con 0x7fcb58101940 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.231+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb480108e0 con 0x7fcb58101940 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.231+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5819f970 con 0x7fcb58101940 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.231+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48018920 con 0x7fcb58101940 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.231+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5819fec0 con 0x7fcb58101940 2024-03-19T00:38:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.232+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb581a00f0 con 0x7fcb58101940 2024-03-19T00:38:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.233+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fcb48010a90 con 0x7fcb58101940 2024-03-19T00:38:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.233+0000 7fcb4effd640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcb440a2800 0x7fcb440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.233+0000 7fcb5d81a640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcb440a2800 0x7fcb440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.233+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcb480cf0b0 con 0x7fcb58101940 2024-03-19T00:38:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.234+0000 7fcb5d81a640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcb440a2800 0x7fcb440a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c240 tx=0x7fcb5402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:16.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.238+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcb4809b7a0 con 0x7fcb58101940 2024-03-19T00:38:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:15 smithi204 ceph-mon[66572]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:38:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2147830032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3709420365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/747270918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:15 smithi044 ceph-mon[66366]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:38:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2147830032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3709420365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/747270918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:16.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.423+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb581a03e0 con 0x7fcb58101940 2024-03-19T00:38:16.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.424+0000 7fcb4effd640 1 -- 172.21.15.7:0/2019308283 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7fcb4809f840 con 0x7fcb58101940 2024-03-19T00:38:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:16.426 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:16.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcb440a2800 msgr2=0x7fcb440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcb440a2800 0x7fcb440a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c240 tx=0x7fcb5402c7e0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 msgr2=0x7fcb58197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 0x7fcb58197fa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800e9f0 tx=0x7fcb4800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 shutdown_connections 2024-03-19T00:38:16.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcb440a2800 0x7fcb440a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb58102300 0x7fcb5819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb58101940 0x7fcb58197fa0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 --2- 172.21.15.7:0/2019308283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb58100750 0x7fcb58197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 >> 172.21.15.7:0/2019308283 conn(0x7fcb580fbcc0 msgr2=0x7fcb580fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 shutdown_connections 2024-03-19T00:38:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.426+0000 7fcb5fa7e640 1 -- 172.21.15.7:0/2019308283 wait complete. 2024-03-19T00:38:16.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:38:16.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- 172.21.15.7:0/2040925881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb001022f0 msgr2=0x7fbb00108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- 172.21.15.7:0/2040925881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb001022f0 0x7fbb00108b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ca0 tx=0x7fbaf802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- 172.21.15.7:0/2040925881 shutdown_connections 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- 172.21.15.7:0/2040925881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb001022f0 0x7fbb00108b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- 172.21.15.7:0/2040925881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb00101930 0x7fbb00101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- 172.21.15.7:0/2040925881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb00100740 0x7fbb00100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- 172.21.15.7:0/2040925881 >> 172.21.15.7:0/2040925881 conn(0x7fbb000fbcb0 msgr2=0x7fbb000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- 172.21.15.7:0/2040925881 shutdown_connections 2024-03-19T00:38:16.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- 172.21.15.7:0/2040925881 wait complete. 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 Processor -- start 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- start start 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 0x7fbb00199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb00101930 0x7fbb0019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb001022f0 0x7fbb001a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0010c270 con 0x7fbb00100740 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0010c0f0 con 0x7fbb001022f0 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.544+0000 7fbb076c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0010c3f0 con 0x7fbb00101930 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 0x7fbb00199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 0x7fbb00199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46274/0 (socket says 172.21.15.7:46274) 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 -- 172.21.15.7:0/763548789 learned_addr learned my addr 172.21.15.7:0/763548789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05c63640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb001022f0 0x7fbb001a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb04c61640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb00101930 0x7fbb0019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 -- 172.21.15.7:0/763548789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb00101930 msgr2=0x7fbb0019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb00101930 0x7fbb0019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 -- 172.21.15.7:0/763548789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb001022f0 msgr2=0x7fbb001a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb001022f0 0x7fbb001a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 -- 172.21.15.7:0/763548789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb001a18b0 con 0x7fbb00100740 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb05462640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 0x7fbb00199c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00e9c0 tx=0x7fbafc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00cde0 con 0x7fbb00100740 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb001a1ba0 con 0x7fbb00100740 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbafc0108e0 con 0x7fbb00100740 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc018990 con 0x7fbb00100740 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.545+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb001a20f0 con 0x7fbb00100740 2024-03-19T00:38:16.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.546+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb001a22c0 con 0x7fbb00100740 2024-03-19T00:38:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.547+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fbafc01f020 con 0x7fbb00100740 2024-03-19T00:38:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.548+0000 7fbaf67fc640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbaec0a2800 0x7fbaec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.548+0000 7fbb04c61640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbaec0a2800 0x7fbaec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.548+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbafc0cf5e0 con 0x7fbb00100740 2024-03-19T00:38:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.548+0000 7fbb04c61640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbaec0a2800 0x7fbaec0a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb0019b170 tx=0x7fbaf0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.551+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbafc09c2c0 con 0x7fbb00100740 2024-03-19T00:38:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.734+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb001a2650 con 0x7fbb00100740 2024-03-19T00:38:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.735+0000 7fbaf67fc640 1 -- 172.21.15.7:0/763548789 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fbafc09bbc0 con 0x7fbb00100740 2024-03-19T00:38:16.736 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbaec0a2800 msgr2=0x7fbaec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbaec0a2800 0x7fbaec0a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb0019b170 tx=0x7fbaf0005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 msgr2=0x7fbb00199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 0x7fbb00199c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00e9c0 tx=0x7fbafc00ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 shutdown_connections 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbaec0a2800 0x7fbaec0a4cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb001022f0 0x7fbb001a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb00101930 0x7fbb0019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 --2- 172.21.15.7:0/763548789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb00100740 0x7fbb00199c70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.736+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 >> 172.21.15.7:0/763548789 conn(0x7fbb000fbcb0 msgr2=0x7fbb000fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.737+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 shutdown_connections 2024-03-19T00:38:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.737+0000 7fbb076c6640 1 -- 172.21.15.7:0/763548789 wait complete. 2024-03-19T00:38:16.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:38:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 -- 172.21.15.7:0/1155856877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 msgr2=0x7fb378101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 --2- 172.21.15.7:0/1155856877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 0x7fb378101db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb364009900 tx=0x7fb36402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 -- 172.21.15.7:0/1155856877 shutdown_connections 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 --2- 172.21.15.7:0/1155856877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3781022f0 0x7fb378108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 --2- 172.21.15.7:0/1155856877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 0x7fb378101db0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 --2- 172.21.15.7:0/1155856877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb378100740 0x7fb378100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 -- 172.21.15.7:0/1155856877 >> 172.21.15.7:0/1155856877 conn(0x7fb3780fbcd0 msgr2=0x7fb3780fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 -- 172.21.15.7:0/1155856877 shutdown_connections 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 -- 172.21.15.7:0/1155856877 wait complete. 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 Processor -- start 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.854+0000 7fb37ed89640 1 -- start start 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb378100740 0x7fb378197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 0x7fb378197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 0x7fb37819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810d390 con 0x7fb3781022f0 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810d210 con 0x7fb378101930 2024-03-19T00:38:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810d510 con 0x7fb378100740 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 0x7fb37819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 0x7fb37819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46292/0 (socket says 172.21.15.7:46292) 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 -- 172.21.15.7:0/14053251 learned_addr learned my addr 172.21.15.7:0/14053251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb377fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 0x7fb378197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37cb25640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb378100740 0x7fb378197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 -- 172.21.15.7:0/14053251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb378100740 msgr2=0x7fb378197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb378100740 0x7fb378197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 -- 172.21.15.7:0/14053251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 msgr2=0x7fb378197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 0x7fb378197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 -- 172.21.15.7:0/14053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37819f690 con 0x7fb3781022f0 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37d326640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 0x7fb37819f020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb37000eb10 tx=0x7fb37000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370004b20 con 0x7fb3781022f0 2024-03-19T00:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37819f980 con 0x7fb3781022f0 2024-03-19T00:38:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb370004cc0 con 0x7fb3781022f0 2024-03-19T00:38:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.855+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370010690 con 0x7fb3781022f0 2024-03-19T00:38:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.856+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37819fea0 con 0x7fb3781022f0 2024-03-19T00:38:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.857+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378107ad0 con 0x7fb3781022f0 2024-03-19T00:38:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.858+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fb370005020 con 0x7fb3781022f0 2024-03-19T00:38:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.858+0000 7fb375ffb640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3680a2800 0x7fb3680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.858+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb3700cf480 con 0x7fb3781022f0 2024-03-19T00:38:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.858+0000 7fb37cb25640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3680a2800 0x7fb3680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.859+0000 7fb37cb25640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3680a2800 0x7fb3680a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb378101790 tx=0x7fb36c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:16.863+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb37009c1c0 con 0x7fb3781022f0 2024-03-19T00:38:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.047+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb378062420 con 0x7fb3781022f0 2024-03-19T00:38:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.047+0000 7fb375ffb640 1 -- 172.21.15.7:0/14053251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fb37009bac0 con 0x7fb3781022f0 2024-03-19T00:38:17.048 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:17.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3680a2800 msgr2=0x7fb3680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3680a2800 0x7fb3680a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb378101790 tx=0x7fb36c005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:17.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 msgr2=0x7fb37819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 0x7fb37819f020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb37000eb10 tx=0x7fb37000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 shutdown_connections 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb3680a2800 0x7fb3680a4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3781022f0 0x7fb37819f020 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378101930 0x7fb378197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 --2- 172.21.15.7:0/14053251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb378100740 0x7fb378197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 >> 172.21.15.7:0/14053251 conn(0x7fb3780fbcd0 msgr2=0x7fb378104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 shutdown_connections 2024-03-19T00:38:17.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.049+0000 7fb37ed89640 1 -- 172.21.15.7:0/14053251 wait complete. 2024-03-19T00:38:17.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2511322806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:17.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2019308283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:17.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/763548789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:17.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:38:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- 172.21.15.7:0/2253751579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 msgr2=0x7f5a78108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/2253751579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a78108b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009ca0 tx=0x7f5a7002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- 172.21.15.7:0/2253751579 shutdown_connections 2024-03-19T00:38:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/2253751579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a78108b80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/2253751579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a78101930 0x7f5a78101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/2253751579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 0x7f5a78100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- 172.21.15.7:0/2253751579 >> 172.21.15.7:0/2253751579 conn(0x7f5a780fbcb0 msgr2=0x7f5a780fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- 172.21.15.7:0/2253751579 shutdown_connections 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- 172.21.15.7:0/2253751579 wait complete. 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 Processor -- start 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- start start 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 0x7f5a78197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a78101930 0x7f5a78197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a7819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c250 con 0x7f5a781022f0 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c0d0 con 0x7f5a78100740 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.166+0000 7f5a7f97e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c3d0 con 0x7f5a78101930 2024-03-19T00:38:17.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7df1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a7819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 0x7f5a78197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7df1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a7819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46308/0 (socket says 172.21.15.7:46308) 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7df1b640 1 -- 172.21.15.7:0/3048106887 learned_addr learned my addr 172.21.15.7:0/3048106887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7cf19640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a78101930 0x7f5a78197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 -- 172.21.15.7:0/3048106887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a78101930 msgr2=0x7f5a78197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a78101930 0x7f5a78197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 -- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 msgr2=0x7f5a7819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a7819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 -- 172.21.15.7:0/3048106887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7819f630 con 0x7f5a78100740 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7df1b640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a7819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7d71a640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 0x7f5a78197a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9c0 tx=0x7f5a7400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400ccc0 con 0x7f5a78100740 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a740035f0 con 0x7f5a78100740 2024-03-19T00:38:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7819f920 con 0x7f5a78100740 2024-03-19T00:38:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400ce60 con 0x7f5a78100740 2024-03-19T00:38:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.167+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7819fec0 con 0x7f5a78100740 2024-03-19T00:38:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.168+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a781a0150 con 0x7f5a78100740 2024-03-19T00:38:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.169+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f5a740037a0 con 0x7f5a78100740 2024-03-19T00:38:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.170+0000 7f5a6e7fc640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5a640a2880 0x7f5a640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.170+0000 7f5a7cf19640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5a640a2880 0x7f5a640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.170+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5a740cf150 con 0x7f5a78100740 2024-03-19T00:38:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.170+0000 7f5a7cf19640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5a640a2880 0x7f5a640a4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5a78198f20 tx=0x7f5a68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:17.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.174+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5a7409be90 con 0x7f5a78100740 2024-03-19T00:38:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2511322806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2019308283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:17.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/763548789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:17.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2511322806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:17.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2019308283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:17.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/763548789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.356+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a781a0320 con 0x7f5a78100740 2024-03-19T00:38:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.357+0000 7f5a6e7fc640 1 -- 172.21.15.7:0/3048106887 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f5a7409b790 con 0x7f5a78100740 2024-03-19T00:38:17.358 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5a640a2880 msgr2=0x7f5a640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5a640a2880 0x7f5a640a4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5a78198f20 tx=0x7f5a68005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 msgr2=0x7f5a78197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 0x7f5a78197a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9c0 tx=0x7f5a7400ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 shutdown_connections 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5a640a2880 0x7f5a640a4d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a781022f0 0x7f5a7819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a78101930 0x7f5a78197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 --2- 172.21.15.7:0/3048106887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a78100740 0x7f5a78197a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 >> 172.21.15.7:0/3048106887 conn(0x7f5a780fbcb0 msgr2=0x7f5a780fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 shutdown_connections 2024-03-19T00:38:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.359+0000 7f5a7f97e640 1 -- 172.21.15.7:0/3048106887 wait complete. 2024-03-19T00:38:17.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 -- 172.21.15.7:0/1949590462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c104bf0 msgr2=0x7f397c106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 --2- 172.21.15.7:0/1949590462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c104bf0 0x7f397c106fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3964009900 tx=0x7f396402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 -- 172.21.15.7:0/1949590462 shutdown_connections 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 --2- 172.21.15.7:0/1949590462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f397c107520 0x7f397c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 --2- 172.21.15.7:0/1949590462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c104bf0 0x7f397c106fe0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 --2- 172.21.15.7:0/1949590462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397c1022c0 0x7f397c1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 -- 172.21.15.7:0/1949590462 >> 172.21.15.7:0/1949590462 conn(0x7f397c0fbcb0 msgr2=0x7f397c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 -- 172.21.15.7:0/1949590462 shutdown_connections 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 -- 172.21.15.7:0/1949590462 wait complete. 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.476+0000 7f39817ab640 1 Processor -- start 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 -- start start 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397c1022c0 0x7f397c101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f397c104bf0 0x7f397c100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 0x7f397c1007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10d360 con 0x7f397c107520 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10d1e0 con 0x7f397c1022c0 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10d4e0 con 0x7f397c104bf0 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 0x7f397c1007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 0x7f397c1007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46324/0 (socket says 172.21.15.7:46324) 2024-03-19T00:38:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 -- 172.21.15.7:0/4077290697 learned_addr learned my addr 172.21.15.7:0/4077290697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397affd640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397c1022c0 0x7f397c101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39727fc640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f397c104bf0 0x7f397c100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 -- 172.21.15.7:0/4077290697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f397c104bf0 msgr2=0x7f397c100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f397c104bf0 0x7f397c100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 -- 172.21.15.7:0/4077290697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397c1022c0 msgr2=0x7f397c101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397c1022c0 0x7f397c101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 -- 172.21.15.7:0/4077290697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c101040 con 0x7f397c107520 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f397b7fe640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 0x7f397c1007b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f397400efd0 tx=0x7f397400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974003b70 con 0x7f397c107520 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c1a1c20 con 0x7f397c107520 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.477+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3974003d10 con 0x7f397c107520 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.478+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974010980 con 0x7f397c107520 2024-03-19T00:38:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.478+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c1a20f0 con 0x7f397c107520 2024-03-19T00:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.479+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397c0686a0 con 0x7f397c107520 2024-03-19T00:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.480+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f3974004020 con 0x7f397c107520 2024-03-19T00:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.480+0000 7f3978ff9640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f39680a2800 0x7f39680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.480+0000 7f397affd640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f39680a2800 0x7f39680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.480+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f39740cf2d0 con 0x7f397c107520 2024-03-19T00:38:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.481+0000 7f397affd640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f39680a2800 0x7f39680a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f396c009bd0 tx=0x7f396c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.485+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f397409c010 con 0x7f397c107520 2024-03-19T00:38:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.668+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f397c0624e0 con 0x7f397c107520 2024-03-19T00:38:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.668+0000 7f3978ff9640 1 -- 172.21.15.7:0/4077290697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f397409b910 con 0x7f397c107520 2024-03-19T00:38:17.670 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f39680a2800 msgr2=0x7f39680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f39680a2800 0x7f39680a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f396c009bd0 tx=0x7f396c005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 msgr2=0x7f397c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 0x7f397c1007b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f397400efd0 tx=0x7f397400c6a0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 shutdown_connections 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f39680a2800 0x7f39680a4cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397c107520 0x7f397c1007b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f397c104bf0 0x7f397c100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 --2- 172.21.15.7:0/4077290697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f397c1022c0 0x7f397c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 >> 172.21.15.7:0/4077290697 conn(0x7f397c0fbcb0 msgr2=0x7f397c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 shutdown_connections 2024-03-19T00:38:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.671+0000 7f39817ab640 1 -- 172.21.15.7:0/4077290697 wait complete. 2024-03-19T00:38:17.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:38:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/185543273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a541018b0 msgr2=0x7f7a54101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/185543273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a541018b0 0x7f7a54101cd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009900 tx=0x7f7a4402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/185543273 shutdown_connections 2024-03-19T00:38:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/185543273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a54103460 0x7f7a54109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/185543273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a54102aa0 0x7f7a54102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/185543273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a541018b0 0x7f7a54101cd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/185543273 >> 172.21.15.7:0/185543273 conn(0x7f7a540fce40 msgr2=0x7f7a540ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/185543273 shutdown_connections 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/185543273 wait complete. 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 Processor -- start 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.785+0000 7f7a5a8cc640 1 -- start start 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a541018b0 0x7f7a54197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a54102aa0 0x7f7a54197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a54103460 0x7f7a5419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410d340 con 0x7f7a54102aa0 2024-03-19T00:38:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410d1c0 con 0x7f7a541018b0 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410d4c0 con 0x7f7a54103460 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a54102aa0 0x7f7a54197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a541018b0 0x7f7a54197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a54102aa0 0x7f7a54197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46350/0 (socket says 172.21.15.7:46350) 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a537fe640 1 -- 172.21.15.7:0/1666099668 learned_addr learned my addr 172.21.15.7:0/1666099668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a58e69640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a54103460 0x7f7a5419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 -- 172.21.15.7:0/1666099668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a54103460 msgr2=0x7f7a5419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a54103460 0x7f7a5419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 -- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a54102aa0 msgr2=0x7f7a54197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a54102aa0 0x7f7a54197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 -- 172.21.15.7:0/1666099668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5419f6a0 con 0x7f7a541018b0 2024-03-19T00:38:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a53fff640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a541018b0 0x7f7a54197a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a44036040 tx=0x7f7a44005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44043020 con 0x7f7a541018b0 2024-03-19T00:38:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.786+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5419f930 con 0x7f7a541018b0 2024-03-19T00:38:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.787+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a44004350 con 0x7f7a541018b0 2024-03-19T00:38:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.787+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44003ab0 con 0x7f7a541018b0 2024-03-19T00:38:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.787+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5419fea0 con 0x7f7a541018b0 2024-03-19T00:38:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.787+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a54068af0 con 0x7f7a541018b0 2024-03-19T00:38:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.789+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f7a44046020 con 0x7f7a541018b0 2024-03-19T00:38:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.789+0000 7f7a517fa640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7a3c0a2800 0x7f7a3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.789+0000 7f7a537fe640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7a3c0a2800 0x7f7a3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.789+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7a440f1490 con 0x7f7a541018b0 2024-03-19T00:38:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.790+0000 7f7a537fe640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7a3c0a2800 0x7f7a3c0a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7a480095f0 tx=0x7f7a48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:17.793+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7a4404d020 con 0x7f7a541018b0 2024-03-19T00:38:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:17 smithi007 ceph-mon[62854]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:38:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:17 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/14053251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:17 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3048106887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:17 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4077290697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.016+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a5402d290 con 0x7f7a541018b0 2024-03-19T00:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.016+0000 7f7a517fa640 1 -- 172.21.15.7:0/1666099668 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f7a5402d290 con 0x7f7a541018b0 2024-03-19T00:38:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:18.018 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:36:26.295874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:38:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-19T00:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7a3c0a2800 msgr2=0x7f7a3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7a3c0a2800 0x7f7a3c0a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7a480095f0 tx=0x7f7a48005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a541018b0 msgr2=0x7f7a54197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a541018b0 0x7f7a54197a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a44036040 tx=0x7f7a44005dc0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 shutdown_connections 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7a3c0a2800 0x7f7a3c0a4cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a54103460 0x7f7a5419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a54102aa0 0x7f7a54197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 --2- 172.21.15.7:0/1666099668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a541018b0 0x7f7a54197a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.018+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 >> 172.21.15.7:0/1666099668 conn(0x7f7a540fce40 msgr2=0x7f7a54105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.019+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 shutdown_connections 2024-03-19T00:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.019+0000 7f7a5a8cc640 1 -- 172.21.15.7:0/1666099668 wait complete. 2024-03-19T00:38:18.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 -- 172.21.15.7:0/1888230365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0100750 msgr2=0x7fb5f0100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 --2- 172.21.15.7:0/1888230365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0100750 0x7fb5f0100b70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009900 tx=0x7fb5ec02bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 -- 172.21.15.7:0/1888230365 shutdown_connections 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 --2- 172.21.15.7:0/1888230365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0102300 0x7fb5f0108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 --2- 172.21.15.7:0/1888230365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5f0101940 0x7fb5f0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 --2- 172.21.15.7:0/1888230365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0100750 0x7fb5f0100b70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 -- 172.21.15.7:0/1888230365 >> 172.21.15.7:0/1888230365 conn(0x7fb5f00fbcc0 msgr2=0x7fb5f00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 -- 172.21.15.7:0/1888230365 shutdown_connections 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 -- 172.21.15.7:0/1888230365 wait complete. 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 Processor -- start 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 -- start start 2024-03-19T00:38:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.133+0000 7fb5f8359640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0100750 0x7fb5f0069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 0x7fb5f019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5f0102300 0x7fb5f019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f010c270 con 0x7fb5f0101940 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f010c0f0 con 0x7fb5f0100750 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f010c3f0 con 0x7fb5f0102300 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 0x7fb5f019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f60f5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0100750 0x7fb5f0069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 0x7fb5f019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46368/0 (socket says 172.21.15.7:46368) 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 -- 172.21.15.7:0/3749067683 learned_addr learned my addr 172.21.15.7:0/3749067683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f68f6640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5f0102300 0x7fb5f019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 -- 172.21.15.7:0/3749067683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5f0102300 msgr2=0x7fb5f019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5f0102300 0x7fb5f019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 -- 172.21.15.7:0/3749067683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0100750 msgr2=0x7fb5f0069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0100750 0x7fb5f0069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 -- 172.21.15.7:0/3749067683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f019f6c0 con 0x7fb5f0101940 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f58f4640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 0x7fb5f019c7c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400e9f0 tx=0x7fb5e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400cd80 con 0x7fb5f0101940 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f019f9b0 con 0x7fb5f0101940 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e40054d0 con 0x7fb5f0101940 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4010490 con 0x7fb5f0101940 2024-03-19T00:38:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.134+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f019ff00 con 0x7fb5f0101940 2024-03-19T00:38:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.135+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f0107b90 con 0x7fb5f0101940 2024-03-19T00:38:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.137+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fb5e4005000 con 0x7fb5f0101940 2024-03-19T00:38:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.137+0000 7fb5e37fe640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb5dc0a2800 0x7fb5dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.137+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb5e40cf360 con 0x7fb5f0101940 2024-03-19T00:38:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.137+0000 7fb5f60f5640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb5dc0a2800 0x7fb5dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.138+0000 7fb5f60f5640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb5dc0a2800 0x7fb5dc0a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005ab0 tx=0x7fb5ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.141+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb5e4053740 con 0x7fb5f0101940 2024-03-19T00:38:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:17 smithi204 ceph-mon[66572]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:38:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:17 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/14053251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:17 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3048106887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:17 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4077290697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.325+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5f002d3c0 con 0x7fb5f0101940 2024-03-19T00:38:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.327+0000 7fb5e37fe640 1 -- 172.21.15.7:0/3749067683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7fb5e40052f0 con 0x7fb5f0101940 2024-03-19T00:38:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:18.328 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb5dc0a2800 msgr2=0x7fb5dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb5dc0a2800 0x7fb5dc0a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005ab0 tx=0x7fb5ec005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 msgr2=0x7fb5f019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 0x7fb5f019c7c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400e9f0 tx=0x7fb5e400edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 shutdown_connections 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb5dc0a2800 0x7fb5dc0a4cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5f0102300 0x7fb5f019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5f0101940 0x7fb5f019c7c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 --2- 172.21.15.7:0/3749067683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0100750 0x7fb5f0069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 >> 172.21.15.7:0/3749067683 conn(0x7fb5f00fbcc0 msgr2=0x7fb5f0104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 shutdown_connections 2024-03-19T00:38:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.329+0000 7fb5f8359640 1 -- 172.21.15.7:0/3749067683 wait complete. 2024-03-19T00:38:18.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-19T00:35:59.576911+0000', 'modified': '2024-03-19T00:36:26.295874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24497, 'mds_2': 24490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/3301743833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3301743833}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3301743833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/1036080774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 1036080774}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1036080774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/2630641732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2630641732}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2630641732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6839/945498116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 945498116}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 945498116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.44:6837/3379136750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 3379136750}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 3379136750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/617428839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 617428839}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 617428839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24521, 24490, 24497]} 2024-03-19T00:38:18.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-19T00:38:18.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-19T00:38:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:17 smithi044 ceph-mon[66366]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:38:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:17 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/14053251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:17 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3048106887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:17 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4077290697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 -- 172.21.15.7:0/235921132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 msgr2=0x7f0dc4101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 --2- 172.21.15.7:0/235921132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc4101db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009900 tx=0x7f0db802bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 -- 172.21.15.7:0/235921132 shutdown_connections 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 --2- 172.21.15.7:0/235921132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc41022f0 0x7f0dc4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 --2- 172.21.15.7:0/235921132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc4101db0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 --2- 172.21.15.7:0/235921132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc4100740 0x7f0dc4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 -- 172.21.15.7:0/235921132 >> 172.21.15.7:0/235921132 conn(0x7f0dc40fbcb0 msgr2=0x7f0dc40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 -- 172.21.15.7:0/235921132 shutdown_connections 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.445+0000 7f0dc9873640 1 -- 172.21.15.7:0/235921132 wait complete. 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 Processor -- start 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 -- start start 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4100740 0x7f0dc4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc41022f0 0x7f0dc41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410d390 con 0x7f0dc4101930 2024-03-19T00:38:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410d210 con 0x7f0dc41022f0 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc9873640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410d510 con 0x7f0dc4100740 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46386/0 (socket says 172.21.15.7:46386) 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 -- 172.21.15.7:0/749919651 learned_addr learned my addr 172.21.15.7:0/749919651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc41022f0 0x7f0dc41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4100740 0x7f0dc4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 -- 172.21.15.7:0/749919651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4100740 msgr2=0x7f0dc4199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4100740 0x7f0dc4199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 -- 172.21.15.7:0/749919651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc41022f0 msgr2=0x7f0dc41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc41022f0 0x7f0dc41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc27fc640 1 -- 172.21.15.7:0/749919651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc41a1900 con 0x7f0dc4101930 2024-03-19T00:38:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc41022f0 0x7f0dc41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.446+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4100740 0x7f0dc4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.447+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc419a200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006010 tx=0x7f0db8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.447+0000 7f0dc8871640 1 -- 172.21.15.7:0/749919651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8043020 con 0x7f0dc4101930 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.447+0000 7f0dc8871640 1 -- 172.21.15.7:0/749919651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80043d0 con 0x7f0dc4101930 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.447+0000 7f0dc8871640 1 -- 172.21.15.7:0/749919651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8003b80 con 0x7f0dc4101930 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.447+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc41a1b90 con 0x7f0dc4101930 2024-03-19T00:38:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.447+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a20e0 con 0x7f0dc4101930 2024-03-19T00:38:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.449+0000 7f0dc8871640 1 -- 172.21.15.7:0/749919651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f0db8046020 con 0x7f0dc4101930 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.449+0000 7f0dcc45d740 1 Processor -- start 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.449+0000 7f0dcc45d740 1 -- start start 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.449+0000 7f0dc8871640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dac0a2800 0x7f0dac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa04483410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 0x55aa0448afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa043e7de0 con 0x55aa04488b50 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aa04390c80 con 0x55aa04482ff0 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa043e3e80 con 0x55aa044839e0 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dac0a2800 0x7f0dac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc8871640 1 -- 172.21.15.7:0/749919651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0db80f0fb0 con 0x7f0dc4101930 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 0x55aa0448afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa04483410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 0x55aa0448afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46392/0 (socket says 172.21.15.7:46392) 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 -- 172.21.15.7:0/3176177047 learned_addr learned my addr 172.21.15.7:0/3176177047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 -- 172.21.15.7:0/3176177047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 msgr2=0x55aa044884e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044884e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 -- 172.21.15.7:0/3176177047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 msgr2=0x55aa04483410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa04483410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 -- 172.21.15.7:0/3176177047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa043a2fb0 con 0x55aa04488b50 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa04483410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 0x55aa0448afe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00f5f0 tx=0x7f0dbc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e7a7e1cde034db server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dac0a2800 0x7f0dac0a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00ffb0 tx=0x7f0dbc01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0da9ffb640 1 -- 172.21.15.7:0/3176177047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc02b070 con 0x55aa04488b50 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0da9ffb640 1 -- 172.21.15.7:0/3176177047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dbc010600 con 0x55aa04488b50 2024-03-19T00:38:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.450+0000 7f0da9ffb640 1 -- 172.21.15.7:0/3176177047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc019040 con 0x55aa04488b50 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3176177047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 msgr2=0x55aa0448afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 0x55aa0448afe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00f5f0 tx=0x7f0dbc00fab0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3176177047 shutdown_connections 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04488b50 0x55aa0448afe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3176177047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa04483410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3176177047 >> 172.21.15.7:0/3176177047 conn(0x55aa0447e560 msgr2=0x55aa0447e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3176177047 shutdown_connections 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3176177047 wait complete. 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.451+0000 7f0dcc45d740 1 -- start start 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 Processor -- start 2024-03-19T00:38:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa0448d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 0x55aa0448da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04494a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa0448baa0 con 0x55aa044839e0 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aa043a3fc0 con 0x55aa04482ff0 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa0448bc20 con 0x55aa04488b50 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dcc45d740 10 client.0 _ll_register_callbacks cb 0x55aa043dfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 0x55aa0448da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 0x55aa0448da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46404/0 (socket says 172.21.15.7:46404) 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 -- 172.21.15.7:0/4132795582 learned_addr learned my addr 172.21.15.7:0/4132795582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa0448d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04494a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 -- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 msgr2=0x55aa04494a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04494a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 -- 172.21.15.7:0/4132795582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 msgr2=0x55aa0448d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa0448d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 -- 172.21.15.7:0/4132795582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa040fc2d0 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa0448d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04494a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 0x55aa0448da10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000c00 tx=0x7f0db80bd3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5167382b5d96d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.452+0000 7f0da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0da2ffd640 1 -- 172.21.15.7:0/4132795582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db809ac60 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0da2ffd640 1 -- 172.21.15.7:0/4132795582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0db809a300 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dbc010600 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0da2ffd640 1 -- 172.21.15.7:0/4132795582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0db809aa80 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aa041c66b0 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0da2ffd640 1 -- 172.21.15.7:0/4132795582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dbc010600 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0da2ffd640 1 -- 172.21.15.7:0/4132795582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0db80b9c60 con 0x55aa044839e0 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:38:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 10 client.15414 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 20 client.15414 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa0449a300 0x55aa0449b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 4 client.15414 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa043f5320 con 0x55aa0449a300 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.453+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa0449a300 0x55aa0449b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.454+0000 7f0da2ffd640 10 client.15414 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.454+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa0449a300 0x55aa0449b770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1cd89f1b303cb97 server_cookie=d294d34387240481 in_seq=0 out_seq=0 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.454+0000 7f0da2ffd640 1 -- 172.21.15.7:0/4132795582 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aa043f5320 con 0x55aa0449a300 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.454+0000 7f0da2ffd640 10 client.15414 handle_command_reply: tid=0 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.454+0000 7f0dcc45d740 1 client.15414 shutdown 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 msgr2=0x55aa0448da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 0x55aa0448da10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000c00 tx=0x7f0db80bd3c0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 shutdown_connections 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa0449a300 0x55aa0449b770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04494a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa044839e0 0x55aa0448da10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0da2ffd640 0 client.15414 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:38:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/4132795582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04482ff0 0x55aa0448d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 >> 172.21.15.7:0/4132795582 conn(0x55aa0447e560 msgr2=0x55aa0447d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 shutdown_connections 2024-03-19T00:38:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 1 -- 172.21.15.7:0/4132795582 wait complete. 2024-03-19T00:38:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.455+0000 7f0dcc45d740 20 client.15414 trim_cache size 0 max 16384 2024-03-19T00:38:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 Processor -- start 2024-03-19T00:38:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 -- start start 2024-03-19T00:38:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04498f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa044839e0 0x55aa04494a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04499480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa04499ab0 con 0x55aa04482ff0 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aa0449a180 con 0x55aa044839e0 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa0449a850 con 0x55aa04488b50 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04498f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04498f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46410/0 (socket says 172.21.15.7:46410) 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/2081427130 learned_addr learned my addr 172.21.15.7:0/2081427130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa044839e0 0x55aa04494a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04499480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/2081427130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 msgr2=0x55aa04499480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04499480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/2081427130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa044839e0 msgr2=0x55aa04494a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa044839e0 0x55aa04494a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/2081427130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa04488860 con 0x55aa04482ff0 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.473+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04498f40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4608a0e2ab3cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0da97fa640 1 -- 172.21.15.7:0/2081427130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db400fc80 con 0x55aa04482ff0 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0da97fa640 1 -- 172.21.15.7:0/2081427130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db400fe20 con 0x55aa04482ff0 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0da97fa640 1 -- 172.21.15.7:0/2081427130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40221b0 con 0x55aa04482ff0 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 -- 172.21.15.7:0/2081427130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 msgr2=0x55aa04498f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04498f40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db4005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 -- 172.21.15.7:0/2081427130 shutdown_connections 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa04488b50 0x55aa04499480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa044839e0 0x55aa04494a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/2081427130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04498f40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 -- 172.21.15.7:0/2081427130 >> 172.21.15.7:0/2081427130 conn(0x55aa0447e560 msgr2=0x55aa0447e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 -- 172.21.15.7:0/2081427130 shutdown_connections 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 -- 172.21.15.7:0/2081427130 wait complete. 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.474+0000 7f0dcc45d740 1 -- start start 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 Processor -- start 2024-03-19T00:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04490290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04488b50 0x55aa04493710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aa0449b5f0 con 0x55aa04482ff0 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aa0449af20 con 0x55aa04488b50 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa0449bcc0 con 0x55aa044839e0 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dcc45d740 10 client.0 _ll_register_callbacks cb 0x7f0db8009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04490290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04490290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46416/0 (socket says 172.21.15.7:46416) 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/3148422952 learned_addr learned my addr 172.21.15.7:0/3148422952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04488b50 0x55aa04493710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 msgr2=0x55aa044907d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044907d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/3148422952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04488b50 msgr2=0x55aa04493710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04488b50 0x55aa04493710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 -- 172.21.15.7:0/3148422952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa04493d00 con 0x55aa04482ff0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc37fe640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04488b50 0x55aa04493710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0dc2ffd640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04490290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0db4012970 tx=0x7f0db4012a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ae9c4cc076326 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.475+0000 7f0da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0da2ffd640 1 -- 172.21.15.7:0/3148422952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4005460 con 0x55aa04482ff0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0da2ffd640 1 -- 172.21.15.7:0/3148422952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0db4005600 con 0x55aa04482ff0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db400fe20 con 0x55aa04482ff0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0da2ffd640 1 -- 172.21.15.7:0/3148422952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0db4004020 con 0x55aa04482ff0 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-03-19T00:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55aa045002f0 con 0x55aa04482ff0 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0da2ffd640 1 -- 172.21.15.7:0/3148422952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db400fe20 con 0x55aa04482ff0 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0da2ffd640 1 -- 172.21.15.7:0/3148422952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0db4004960 con 0x55aa04482ff0 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 10 client.15420 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 20 client.15420 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa04504e70 0x55aa04503970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 4 client.15420 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "status"}) -- 0x55aa043ef670 con 0x55aa04504e70 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.476+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa04504e70 0x55aa04503970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0da2ffd640 10 client.15420 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dc27fc640 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa04504e70 0x55aa04503970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1de4509c79414d0 server_cookie=58171ae149708aa4 in_seq=0 out_seq=0 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0da2ffd640 1 -- 172.21.15.7:0/3148422952 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55aa043ef670 con 0x55aa04504e70 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0da2ffd640 10 client.15420 handle_command_reply: tid=0 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 client.15420 shutdown 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 msgr2=0x55aa04490290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04490290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0db4012970 tx=0x7f0db4012a20 comp rx=0 tx=0).stop 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 shutdown_connections 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55aa04504e70 0x55aa04503970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aa04488b50 0x55aa04493710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa044839e0 0x55aa044907d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0da2ffd640 0 client.15420 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 --2- 172.21.15.7:0/3148422952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa04482ff0 0x55aa04490290 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 >> 172.21.15.7:0/3148422952 conn(0x55aa0447e560 msgr2=0x55aa04498f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 shutdown_connections 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 1 -- 172.21.15.7:0/3148422952 wait complete. 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.477+0000 7f0dcc45d740 20 client.15420 trim_cache size 0 max 16384 2024-03-19T00:38:18.488 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24490, 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 113.19178897800001, 2024-03-19T00:38:18.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-03-19T00:38:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 91, 2024-03-19T00:38:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:38:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151.12209581499999 2024-03-19T00:38:18.490 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:38:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dac0a2800 msgr2=0x7f0dac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dac0a2800 0x7f0dac0a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00ffb0 tx=0x7f0dbc01e000 comp rx=0 tx=0).stop 2024-03-19T00:38:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 msgr2=0x7f0dc419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc419a200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006010 tx=0x7f0db8005e40 comp rx=0 tx=0).stop 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 shutdown_connections 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0dac0a2800 0x7f0dac0a4cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0dc41022f0 0x7f0dc41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dc4101930 0x7f0dc419a200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 --2- 172.21.15.7:0/749919651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dc4100740 0x7f0dc4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 >> 172.21.15.7:0/749919651 conn(0x7f0dc40fbcb0 msgr2=0x7f0dc4109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 shutdown_connections 2024-03-19T00:38:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.478+0000 7f0dc9873640 1 -- 172.21.15.7:0/749919651 wait complete. 2024-03-19T00:38:18.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.596+0000 7f65bd122640 1 -- 172.21.15.7:0/2654983630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8107580 msgr2=0x7f65b8109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.596+0000 7f65bd122640 1 --2- 172.21.15.7:0/2654983630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8107580 0x7f65b8109970 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009ca0 tx=0x7f65b002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.596+0000 7f65bd122640 1 -- 172.21.15.7:0/2654983630 shutdown_connections 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.596+0000 7f65bd122640 1 --2- 172.21.15.7:0/2654983630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8107580 0x7f65b8109970 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.596+0000 7f65bd122640 1 --2- 172.21.15.7:0/2654983630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8104c50 0x7f65b8107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.596+0000 7f65bd122640 1 --2- 172.21.15.7:0/2654983630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b8102320 0x7f65b8104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- 172.21.15.7:0/2654983630 >> 172.21.15.7:0/2654983630 conn(0x7f65b80fbcb0 msgr2=0x7f65b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- 172.21.15.7:0/2654983630 shutdown_connections 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- 172.21.15.7:0/2654983630 wait complete. 2024-03-19T00:38:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 Processor -- start 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- start start 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102320 0x7f65b8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 0x7f65b8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b8107580 0x7f65b819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65b810d370 con 0x7f65b8104c50 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65b810d1f0 con 0x7f65b8107580 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.597+0000 7f65bd122640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65b810d4f0 con 0x7f65b8102320 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 0x7f65b8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 0x7f65b8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46424/0 (socket says 172.21.15.7:46424) 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 -- 172.21.15.7:0/378771834 learned_addr learned my addr 172.21.15.7:0/378771834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102320 0x7f65b8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65b759e640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b8107580 0x7f65b819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 -- 172.21.15.7:0/378771834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102320 msgr2=0x7f65b8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102320 0x7f65b8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 -- 172.21.15.7:0/378771834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b8107580 msgr2=0x7f65b819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b8107580 0x7f65b819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 -- 172.21.15.7:0/378771834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b819f670 con 0x7f65b8104c50 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102320 0x7f65b8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65affff640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 0x7f65b8197f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f65a000e9c0 tx=0x7f65a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65b4d99640 1 -- 172.21.15.7:0/378771834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a000cd80 con 0x7f65b8104c50 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65b4d99640 1 -- 172.21.15.7:0/378771834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a00054d0 con 0x7f65b8104c50 2024-03-19T00:38:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b819f960 con 0x7f65b8104c50 2024-03-19T00:38:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65b4d99640 1 -- 172.21.15.7:0/378771834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0010430 con 0x7f65b8104c50 2024-03-19T00:38:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.598+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b819feb0 con 0x7f65b8104c50 2024-03-19T00:38:18.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.600+0000 7f65b4d99640 1 -- 172.21.15.7:0/378771834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f65a0004fd0 con 0x7f65b8104c50 2024-03-19T00:38:18.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 Processor -- start 2024-03-19T00:38:18.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 -- start start 2024-03-19T00:38:18.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b4d99640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f65a40a2800 0x7f65a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b4d99640 1 -- 172.21.15.7:0/378771834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f65a00cf1f0 con 0x7f65b8104c50 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b759e640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f65a40a2800 0x7f65a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x558068989410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806898e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068990fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580688edde0 con 0x55806898eb50 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558068896c80 con 0x558068988ff0 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65bfdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580688e9e80 con 0x5580689899e0 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x558068989410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65affff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806898e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x558068989410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57004/0 (socket says 172.21.15.7:57004) 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b6d9d640 1 -- 172.21.15.7:0/4267479279 learned_addr learned my addr 172.21.15.7:0/4267479279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.601+0000 7f65b759e640 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068990fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b759e640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f65a40a2800 0x7f65a40a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006010 tx=0x7f65b0032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b6d9d640 1 -- 172.21.15.7:0/4267479279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 msgr2=0x55806898e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806898e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b6d9d640 1 -- 172.21.15.7:0/4267479279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 msgr2=0x558068990fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068990fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b6d9d640 1 -- 172.21.15.7:0/4267479279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580688a8fb0 con 0x558068988ff0 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b759e640 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068990fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65affff640 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806898e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x558068989410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65a8011540 tx=0x7f65a8011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ddbda7a225b3dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65ad7fa640 1 -- 172.21.15.7:0/4267479279 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a800f640 con 0x558068988ff0 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65ad7fa640 1 -- 172.21.15.7:0/4267479279 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a800fc90 con 0x558068988ff0 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.602+0000 7f65ad7fa640 1 -- 172.21.15.7:0/4267479279 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8019c80 con 0x558068988ff0 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 -- 172.21.15.7:0/4267479279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 msgr2=0x558068989410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x558068989410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65a8011540 tx=0x7f65a8011a00 comp rx=0 tx=0).stop 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 -- 172.21.15.7:0/4267479279 shutdown_connections 2024-03-19T00:38:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068990fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806898e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 --2- 172.21.15.7:0/4267479279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x558068989410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 -- 172.21.15.7:0/4267479279 >> 172.21.15.7:0/4267479279 conn(0x558068984560 msgr2=0x558068984960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 -- 172.21.15.7:0/4267479279 shutdown_connections 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 -- 172.21.15.7:0/4267479279 wait complete. 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 -- start start 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 Processor -- start 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.603+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068988ff0 0x5580689934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 0x558068993a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x55806899aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558068991b00 con 0x5580689899e0 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5580688a9fc0 con 0x55806898eb50 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558068991c80 con 0x558068988ff0 2024-03-19T00:38:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 10 client.0 _ll_register_callbacks cb 0x5580688e5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 0x558068993a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 0x558068993a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46438/0 (socket says 172.21.15.7:46438) 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 -- 172.21.15.7:0/700221670 learned_addr learned my addr 172.21.15.7:0/700221670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068988ff0 0x5580689934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65b759e640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x55806899aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 -- 172.21.15.7:0/700221670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068988ff0 msgr2=0x5580689934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068988ff0 0x5580689934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 -- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 msgr2=0x55806899aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x55806899aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 -- 172.21.15.7:0/700221670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580686022d0 con 0x5580689899e0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65affff640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 0x558068993a10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65a009b4d0 tx=0x7f65a009b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61930a7073153d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f659e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f659e7fc640 1 -- 172.21.15.7:0/700221670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a00b4060 con 0x5580689899e0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f659e7fc640 1 -- 172.21.15.7:0/700221670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f65a0099ad0 con 0x5580689899e0 2024-03-19T00:38:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a800fc90 con 0x5580689899e0 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f659e7fc640 1 -- 172.21.15.7:0/700221670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f65a009f920 con 0x5580689899e0 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 10 client.15432 fetch_fsmap learned FSMap version 24 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.604+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5580686cc6b0 con 0x5580689899e0 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f659e7fc640 1 -- 172.21.15.7:0/700221670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a800fc90 con 0x5580689899e0 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f659e7fc640 1 -- 172.21.15.7:0/700221670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f65a00afbb0 con 0x5580689899e0 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65bfdef740 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65bfdef740 10 client.15432 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65bfdef740 20 client.15432 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65bfdef740 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5580689a0330 0x5580689a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65bfdef740 4 client.15432 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580688fb320 con 0x5580689a0330 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5580689a0330 0x5580689a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f659e7fc640 10 client.15432 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:38:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.605+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5580689a0330 0x5580689a17a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd48d4c3bbf14021 server_cookie=4237d2e83510f995 in_seq=0 out_seq=0 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.606+0000 7f659e7fc640 1 -- 172.21.15.7:0/700221670 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5580688fb320 con 0x5580689a0330 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.606+0000 7f659e7fc640 10 client.15432 handle_command_reply: tid=0 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.606+0000 7f65bfdef740 1 client.15432 shutdown 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 msgr2=0x558068993a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 0x558068993a10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65a009b4d0 tx=0x7f65a009b130 comp rx=0 tx=0).stop 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 shutdown_connections 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5580689a0330 0x5580689a17a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x55806899aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580689899e0 0x558068993a10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 --2- 172.21.15.7:0/700221670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068988ff0 0x5580689934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f659e7fc640 0 client.15432 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 >> 172.21.15.7:0/700221670 conn(0x558068984560 msgr2=0x5580689837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 shutdown_connections 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 1 -- 172.21.15.7:0/700221670 wait complete. 2024-03-19T00:38:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.607+0000 7f65bfdef740 20 client.15432 trim_cache size 0 max 16384 2024-03-19T00:38:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 Processor -- start 2024-03-19T00:38:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 -- start start 2024-03-19T00:38:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 0x558068982f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x558068983490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55806899fae0 con 0x558068988ff0 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5580689a01b0 con 0x55806898eb50 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65bfdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580689a0880 con 0x5580689899e0 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 0x558068982f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 0x558068982f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46448/0 (socket says 172.21.15.7:46448) 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 -- 172.21.15.7:0/2173771031 learned_addr learned my addr 172.21.15.7:0/2173771031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65affff640 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b759e640 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x558068983490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 -- 172.21.15.7:0/2173771031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 msgr2=0x55806899aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 -- 172.21.15.7:0/2173771031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 msgr2=0x558068983490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x558068983490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.625+0000 7f65b6d9d640 1 -- 172.21.15.7:0/2173771031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806897f670 con 0x558068988ff0 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 0x558068982f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f65a8011510 tx=0x7f65a8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f432f374a955dda server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65acff9640 1 -- 172.21.15.7:0/2173771031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8005040 con 0x558068988ff0 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65acff9640 1 -- 172.21.15.7:0/2173771031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a8019630 con 0x558068988ff0 2024-03-19T00:38:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65acff9640 1 -- 172.21.15.7:0/2173771031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a800f8c0 con 0x558068988ff0 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 -- 172.21.15.7:0/2173771031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 msgr2=0x558068982f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 0x558068982f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f65a8011510 tx=0x7f65a8005850 comp rx=0 tx=0).stop 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 -- 172.21.15.7:0/2173771031 shutdown_connections 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55806898eb50 0x558068983490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 --2- 172.21.15.7:0/2173771031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558068988ff0 0x558068982f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 -- 172.21.15.7:0/2173771031 >> 172.21.15.7:0/2173771031 conn(0x558068984560 msgr2=0x5580689934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 -- 172.21.15.7:0/2173771031 shutdown_connections 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.626+0000 7f65bfdef740 1 -- 172.21.15.7:0/2173771031 wait complete. 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 -- start start 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 Processor -- start 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x55806899a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068996cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580689a1620 con 0x55806898eb50 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5580689a0f50 con 0x558068988ff0 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580689a1cf0 con 0x5580689899e0 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65bfdef740 10 client.0 _ll_register_callbacks cb 0x7f65a00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068996cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x55806899a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x55806899a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57018/0 (socket says 172.21.15.7:57018) 2024-03-19T00:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068996cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46454/0 (socket says 172.21.15.7:46454) 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 -- 172.21.15.7:0/1981922819 learned_addr learned my addr 172.21.15.7:0/1981922819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65affff640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 -- 172.21.15.7:0/1981922819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 msgr2=0x55806899a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 -- 172.21.15.7:0/1981922819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 msgr2=0x558068996cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068996cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b6d9d640 1 -- 172.21.15.7:0/1981922819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558068997360 con 0x558068988ff0 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65b759e640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068996cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.627+0000 7f65affff640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65b6d9d640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x55806899a200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65a801b040 tx=0x7f65a800f6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c644a81e747a76 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f659e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f659e7fc640 1 -- 172.21.15.7:0/1981922819 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8033070 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a8019630 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f659e7fc640 1 -- 172.21.15.7:0/1981922819 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f65a80199f0 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f659e7fc640 1 -- 172.21.15.7:0/1981922819 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f65a803f050 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65bfdef740 10 client.25289 fetch_fsmap learned FSMap version 24 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558068a06c40 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f659e7fc640 1 -- 172.21.15.7:0/1981922819 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a8019630 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f659e7fc640 1 -- 172.21.15.7:0/1981922819 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f65a802b020 con 0x558068988ff0 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65bfdef740 10 client.25289 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65bfdef740 10 client.25289 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.628+0000 7f65bfdef740 20 client.25289 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558068a0b240 0x558068a09f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 4 client.25289 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "status"}) -- 0x5580688f5670 con 0x558068a0b240 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65affff640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558068a0b240 0x558068a09f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65affff640 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558068a0b240 0x558068a09f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c35d0d9616dd083e server_cookie=b3c73f616d09fc48 in_seq=0 out_seq=0 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f659e7fc640 10 client.25289 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f659e7fc640 1 -- 172.21.15.7:0/1981922819 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5580688f5670 con 0x558068a0b240 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f659e7fc640 10 client.25289 handle_command_reply: tid=0 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 client.25289 shutdown 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 msgr2=0x55806899a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x55806899a200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65a801b040 tx=0x7f65a800f6d0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 shutdown_connections 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558068a0b240 0x558068a09f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55806898eb50 0x558068996cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580689899e0 0x55806899a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 --2- 172.21.15.7:0/1981922819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558068988ff0 0x55806899a200 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f659e7fc640 0 client.25289 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 >> 172.21.15.7:0/1981922819 conn(0x558068984560 msgr2=0x5580689877c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 shutdown_connections 2024-03-19T00:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.629+0000 7f65bfdef740 1 -- 172.21.15.7:0/1981922819 wait complete. 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bfdef740 20 client.25289 trim_cache size 0 max 16384 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24497, 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 114.351142915, 2024-03-19T00:38:18.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-03-19T00:38:18.642 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 91, 2024-03-19T00:38:18.642 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:38:18.642 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 154.02015544599999 2024-03-19T00:38:18.642 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f65a40a2800 msgr2=0x7f65a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f65a40a2800 0x7f65a40a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006010 tx=0x7f65b0032340 comp rx=0 tx=0).stop 2024-03-19T00:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 msgr2=0x7f65b8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 0x7f65b8197f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f65a000e9c0 tx=0x7f65a000ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 shutdown_connections 2024-03-19T00:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f65a40a2800 0x7f65a40a4cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65b8107580 0x7f65b819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b8104c50 0x7f65b8197f90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 --2- 172.21.15.7:0/378771834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102320 0x7f65b8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 >> 172.21.15.7:0/378771834 conn(0x7f65b80fbcb0 msgr2=0x7f65b8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 shutdown_connections 2024-03-19T00:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.630+0000 7f65bd122640 1 -- 172.21.15.7:0/378771834 wait complete. 2024-03-19T00:38:18.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 -- 172.21.15.7:0/2393888625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54102340 msgr2=0x7f8d54104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/2393888625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54102340 0x7f8d54104730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009900 tx=0x7f8d4402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 -- 172.21.15.7:0/2393888625 shutdown_connections 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/2393888625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541075a0 0x7f8d54109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/2393888625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54104c70 0x7f8d54107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/2393888625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54102340 0x7f8d54104730 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 -- 172.21.15.7:0/2393888625 >> 172.21.15.7:0/2393888625 conn(0x7f8d540fbcb0 msgr2=0x7f8d540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 -- 172.21.15.7:0/2393888625 shutdown_connections 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 -- 172.21.15.7:0/2393888625 wait complete. 2024-03-19T00:38:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 Processor -- start 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.748+0000 7f8d5a60c640 1 -- start start 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 0x7f8d54197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54104c70 0x7f8d54198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541075a0 0x7f8d5419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410d3e0 con 0x7f8d54104c70 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410d260 con 0x7f8d541075a0 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410d560 con 0x7f8d54102340 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54104c70 0x7f8d54198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 0x7f8d54197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 0x7f8d54197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33864/0 (socket says 172.21.15.7:33864) 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54104c70 0x7f8d54198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46478/0 (socket says 172.21.15.7:46478) 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 -- 172.21.15.7:0/3179331353 learned_addr learned my addr 172.21.15.7:0/3179331353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541075a0 0x7f8d5419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 -- 172.21.15.7:0/3179331353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541075a0 msgr2=0x7f8d5419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541075a0 0x7f8d5419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 -- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54104c70 msgr2=0x7f8d54198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54104c70 0x7f8d54198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 -- 172.21.15.7:0/3179331353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5419f700 con 0x7f8d54102340 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d53fff640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 0x7f8d54197ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000c00 tx=0x7f8d440040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d517fa640 1 -- 172.21.15.7:0/3179331353 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44043020 con 0x7f8d54102340 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5419f990 con 0x7f8d54102340 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d517fa640 1 -- 172.21.15.7:0/3179331353 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d440044c0 con 0x7f8d54102340 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.749+0000 7f8d517fa640 1 -- 172.21.15.7:0/3179331353 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44003c60 con 0x7f8d54102340 2024-03-19T00:38:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.750+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5419ff30 con 0x7f8d54102340 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d517fa640 1 -- 172.21.15.7:0/3179331353 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f8d44046020 con 0x7f8d54102340 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 Processor -- start 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 -- start start 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b2ff0 0x5615164b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 0x5615164bafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561516417de0 con 0x5615164b8b50 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615163c0c80 con 0x5615164b39e0 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561516413e80 con 0x5615164b2ff0 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 0x5615164bafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 0x5615164bafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46494/0 (socket says 172.21.15.7:46494) 2024-03-19T00:38:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 -- 172.21.15.7:0/724057157 learned_addr learned my addr 172.21.15.7:0/724057157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d53fff640 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b2ff0 0x5615164b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d517fa640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d3c0a2800 0x7f8d3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 -- 172.21.15.7:0/724057157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b2ff0 msgr2=0x5615164b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b2ff0 0x5615164b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 -- 172.21.15.7:0/724057157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 msgr2=0x5615164b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d537fe640 1 -- 172.21.15.7:0/724057157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615163d2fb0 con 0x5615164b8b50 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.752+0000 7f8d53fff640 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d3c0a2800 0x7f8d3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d517fa640 1 -- 172.21.15.7:0/3179331353 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d440f0fa0 con 0x7f8d54102340 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d537fe640 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 0x5615164bafe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009c60 tx=0x7f8d48011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d075f5ea9ef8de4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d3a7fc640 1 -- 172.21.15.7:0/724057157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004230 con 0x5615164b8b50 2024-03-19T00:38:18.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d3a7fc640 1 -- 172.21.15.7:0/724057157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d480043d0 con 0x5615164b8b50 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d3a7fc640 1 -- 172.21.15.7:0/724057157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48019670 con 0x5615164b8b50 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/724057157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 msgr2=0x5615164bafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d3c0a2800 0x7f8d3c0a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009fd0 tx=0x7f8d4c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 0x5615164bafe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009c60 tx=0x7f8d48011810 comp rx=0 tx=0).stop 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/724057157 shutdown_connections 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b8b50 0x5615164bafe0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/724057157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b2ff0 0x5615164b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.753+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/724057157 >> 172.21.15.7:0/724057157 conn(0x5615164ae560 msgr2=0x5615164ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/724057157 shutdown_connections 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/724057157 wait complete. 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 -- start start 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 Processor -- start 2024-03-19T00:38:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 0x5615164bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615164bbb00 con 0x5615164b2ff0 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615163d3fc0 con 0x5615164b39e0 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615164bbc80 con 0x5615164b8b50 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d5d1f0740 10 client.0 _ll_register_callbacks cb 0x56151640fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 0x5615164bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 0x5615164bd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46508/0 (socket says 172.21.15.7:46508) 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d53fff640 1 -- 172.21.15.7:0/3153567176 learned_addr learned my addr 172.21.15.7:0/3153567176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d537fe640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.754+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d53fff640 1 -- 172.21.15.7:0/3153567176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 msgr2=0x5615164c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d53fff640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d53fff640 1 -- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 msgr2=0x5615164bd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d53fff640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164bd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d53fff640 1 -- 172.21.15.7:0/3153567176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151612c2d0 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d53fff640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 0x5615164bd4b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8d440ee740 tx=0x7f8d440b92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04ca43b0e356244 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d337fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d337fe640 1 -- 172.21.15.7:0/3153567176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440bd240 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d337fe640 1 -- 172.21.15.7:0/3153567176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d440bba00 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d480043d0 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d337fe640 1 -- 172.21.15.7:0/3153567176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8d440bd060 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d5d1f0740 10 client.15450 fetch_fsmap learned FSMap version 24 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5615161f66b0 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d337fe640 1 -- 172.21.15.7:0/3153567176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d480043d0 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d337fe640 1 -- 172.21.15.7:0/3153567176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8d440f30f0 con 0x5615164b2ff0 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d5d1f0740 10 client.15450 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.755+0000 7f8d5d1f0740 10 client.15450 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d5d1f0740 20 client.15450 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615164ca3b0 0x5615164cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d5d1f0740 4 client.15450 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561516425320 con 0x5615164ca3b0 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d537fe640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615164ca3b0 0x5615164cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d537fe640 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615164ca3b0 0x5615164cb820 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1853acafd9ba2f6 server_cookie=af6d5baeab898d13 in_seq=0 out_seq=0 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.756+0000 7f8d337fe640 10 client.15450 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d337fe640 1 -- 172.21.15.7:0/3153567176 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561516425320 con 0x5615164ca3b0 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d337fe640 10 client.15450 handle_command_reply: tid=0 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 client.15450 shutdown 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 msgr2=0x5615164bd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 0x5615164bd4b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8d440ee740 tx=0x7f8d440b92e0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 shutdown_connections 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b39e0 0x5615164bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615164ca3b0 0x5615164cb820 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/3153567176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b2ff0 0x5615164bd4b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d337fe640 0 client.15450 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 >> 172.21.15.7:0/3153567176 conn(0x5615164ae560 msgr2=0x5615164ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 shutdown_connections 2024-03-19T00:38:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/3153567176 wait complete. 2024-03-19T00:38:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.757+0000 7f8d5d1f0740 20 client.15450 trim_cache size 0 max 16384 2024-03-19T00:38:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 Processor -- start 2024-03-19T00:38:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 -- start start 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615164c9b60 con 0x5615164b39e0 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615164ca230 con 0x5615164b2ff0 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615164ca900 con 0x5615164b8b50 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.775+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46514/0 (socket says 172.21.15.7:46514) 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d537fe640 1 -- 172.21.15.7:0/1661084005 learned_addr learned my addr 172.21.15.7:0/1661084005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 -- 172.21.15.7:0/1661084005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 msgr2=0x5615164ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 -- 172.21.15.7:0/1661084005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 msgr2=0x5615164c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 -- 172.21.15.7:0/1661084005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615164a9670 con 0x5615164b2ff0 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d537fe640 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d53fff640 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164acf50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005ec0 tx=0x7f8d4402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec8957aaa799832 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d39ffb640 1 -- 172.21.15.7:0/1661084005 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440bd240 con 0x5615164b2ff0 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d39ffb640 1 -- 172.21.15.7:0/1661084005 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d440bde00 con 0x5615164b2ff0 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.776+0000 7f8d39ffb640 1 -- 172.21.15.7:0/1661084005 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440bbae0 con 0x5615164b2ff0 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1661084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 msgr2=0x5615164acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164acf50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005ec0 tx=0x7f8d4402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1661084005 shutdown_connections 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1661084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164acf50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1661084005 >> 172.21.15.7:0/1661084005 conn(0x5615164ae560 msgr2=0x5615164c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1661084005 shutdown_connections 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1661084005 wait complete. 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- start start 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 Processor -- start 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615164cb6a0 con 0x5615164b39e0 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615164cafd0 con 0x5615164b2ff0 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615164cbd70 con 0x5615164b8b50 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.777+0000 7f8d5d1f0740 10 client.0 _ll_register_callbacks cb 0x7f8d44009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164c0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57044/0 (socket says 172.21.15.7:57044) 2024-03-19T00:38:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 -- 172.21.15.7:0/1029466385 learned_addr learned my addr 172.21.15.7:0/1029466385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d58ba9640 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 -- 172.21.15.7:0/1029466385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 msgr2=0x5615164c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 -- 172.21.15.7:0/1029466385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 msgr2=0x5615164c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 -- 172.21.15.7:0/1029466385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615164c4370 con 0x5615164b2ff0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d53fff640 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164c0b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d440eeab0 tx=0x7f8d440f33a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e923558578fdca5e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d337fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d337fe640 1 -- 172.21.15.7:0/1029466385 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440bbae0 con 0x5615164b2ff0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d440bde00 con 0x5615164b2ff0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d337fe640 1 -- 172.21.15.7:0/1029466385 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d440c5c20 con 0x5615164b2ff0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d337fe640 1 -- 172.21.15.7:0/1029466385 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8d440bd240 con 0x5615164b2ff0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d5d1f0740 10 client.25307 fetch_fsmap learned FSMap version 24 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5615165308a0 con 0x5615164b2ff0 2024-03-19T00:38:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.778+0000 7f8d337fe640 1 -- 172.21.15.7:0/1029466385 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d440bde00 con 0x5615164b2ff0 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d337fe640 1 -- 172.21.15.7:0/1029466385 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8d440bc7c0 con 0x5615164b2ff0 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 10 client.25307 fetch_fsmap finished waiting for FSMap version 24 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 10 client.25307 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 20 client.25307 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561516534e90 0x5615165319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 4 client.25307 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "status"}) -- 0x56151641f670 con 0x561516534e90 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d537fe640 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561516534e90 0x5615165319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d337fe640 10 client.25307 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d537fe640 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561516534e90 0x5615165319a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b520691df3c435 server_cookie=6e75e84fc3f78f07 in_seq=0 out_seq=0 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d337fe640 1 -- 172.21.15.7:0/1029466385 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56151641f670 con 0x561516534e90 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d337fe640 10 client.25307 handle_command_reply: tid=0 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.779+0000 7f8d5d1f0740 1 client.25307 shutdown 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 msgr2=0x5615164c0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164c0b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d440eeab0 tx=0x7f8d440f33a0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 shutdown_connections 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615164b8b50 0x5615164c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561516534e90 0x5615165319a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615164b39e0 0x5615164c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 --2- 172.21.15.7:0/1029466385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615164b2ff0 0x5615164c0b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d337fe640 0 client.25307 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 >> 172.21.15.7:0/1029466385 conn(0x5615164ae560 msgr2=0x5615164b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 shutdown_connections 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 1 -- 172.21.15.7:0/1029466385 wait complete. 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5d1f0740 20 client.25307 trim_cache size 0 max 16384 2024-03-19T00:38:18.789 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "60839b00-e587-11ee-95c9-87774f69a715", 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24521, 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 138.57674459200001, 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 91, 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 142.74753683399999 2024-03-19T00:38:18.790 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d3c0a2800 msgr2=0x7f8d3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d3c0a2800 0x7f8d3c0a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009fd0 tx=0x7f8d4c004a40 comp rx=0 tx=0).stop 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 msgr2=0x7f8d54197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 0x7f8d54197ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000c00 tx=0x7f8d440040c0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 shutdown_connections 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d3c0a2800 0x7f8d3c0a4cc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541075a0 0x7f8d5419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d54104c70 0x7f8d54198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 --2- 172.21.15.7:0/3179331353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54102340 0x7f8d54197ae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 >> 172.21.15.7:0/3179331353 conn(0x7f8d540fbcb0 msgr2=0x7f8d54102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 shutdown_connections 2024-03-19T00:38:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:18.780+0000 7f8d5a60c640 1 -- 172.21.15.7:0/3179331353 wait complete. 2024-03-19T00:38:18.793 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-19T00:38:18.804 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-19T00:38:18.804 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-19T00:38:18.805 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-19T00:38:18.805 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-19T00:38:18.805 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:18.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:18.805 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-19T00:38:18.820 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-19T00:38:18.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:18.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T00:38:18.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:18.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-19T00:38:18.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1666099668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:18.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3749067683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:18.967 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-19T00:38:18.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-19T00:38:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:38:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:18 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1666099668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:19.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:18 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3749067683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:19.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:18 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1666099668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:19.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:18 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3749067683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:20.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:19 smithi007 ceph-mon[62854]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:19 smithi204 ceph-mon[66572]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:20.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:19 smithi044 ceph-mon[66366]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 -- 172.21.15.7:0/2787616180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc102d20 msgr2=0x7feccc10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 --2- 172.21.15.7:0/2787616180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc102d20 0x7feccc10f160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009fc0 tx=0x7fecc400b330 comp rx=0 tx=0).stop 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 -- 172.21.15.7:0/2787616180 shutdown_connections 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 --2- 172.21.15.7:0/2787616180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc102d20 0x7feccc10f160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 --2- 172.21.15.7:0/2787616180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc108340 0x7feccc108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 --2- 172.21.15.7:0/2787616180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 -- 172.21.15.7:0/2787616180 >> 172.21.15.7:0/2787616180 conn(0x7feccc079440 msgr2=0x7feccc079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 -- 172.21.15.7:0/2787616180 shutdown_connections 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.942+0000 7fecd2f00700 1 -- 172.21.15.7:0/2787616180 wait complete. 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 Processor -- start 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 -- start start 2024-03-19T00:38:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc102d20 0x7feccc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc108340 0x7feccc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1124f0 con 0x7feccc102d20 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feccc112370 con 0x7feccc108340 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd2f00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feccc112670 con 0x7feccc102360 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7feccbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc102d20 0x7feccc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7feccbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc102d20 0x7feccc19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46544/0 (socket says 172.21.15.7:46544) 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd149d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc108340 0x7feccc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33944/0 (socket says 172.21.15.7:33944) 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7feccbfff700 1 -- 172.21.15.7:0/1241614442 learned_addr learned my addr 172.21.15.7:0/1241614442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 -- 172.21.15.7:0/1241614442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc108340 msgr2=0x7feccc1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc108340 0x7feccc1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 -- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc102d20 msgr2=0x7feccc19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc102d20 0x7feccc19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd0c9c700 1 -- 172.21.15.7:0/1241614442 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc198fd0 con 0x7feccc102360 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.943+0000 7fecd149d700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc108340 0x7feccc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.944+0000 7fecd0c9c700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc19e670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecc000eb90 tx=0x7fecc000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.944+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc000cc90 con 0x7feccc102360 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.944+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc000ce30 con 0x7feccc102360 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.944+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0010430 con 0x7feccc102360 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.944+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1992c0 con 0x7feccc102360 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.944+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a54d0 con 0x7feccc102360 2024-03-19T00:38:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.945+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc1039f0 con 0x7feccc102360 2024-03-19T00:38:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.946+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fecc0003680 con 0x7feccc102360 2024-03-19T00:38:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.946+0000 7fecc9ffb700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fecbc0a2890 0x7fecbc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.946+0000 7feccbfff700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fecbc0a2890 0x7fecbc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.946+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fecc00cf070 con 0x7feccc102360 2024-03-19T00:38:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.947+0000 7feccbfff700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fecbc0a2890 0x7fecbc0a4d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feccc079a60 tx=0x7fecb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:20.951+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fecc009bd50 con 0x7feccc102360 2024-03-19T00:38:21.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.205+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feccc199d50 con 0x7feccc102360 2024-03-19T00:38:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.206+0000 7fecc9ffb700 1 -- 172.21.15.7:0/1241614442 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7feccc199d50 con 0x7feccc102360 2024-03-19T00:38:21.207 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fecbc0a2890 msgr2=0x7fecbc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fecbc0a2890 0x7fecbc0a4d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feccc079a60 tx=0x7fecb8009260 comp rx=0 tx=0).stop 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 msgr2=0x7feccc19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc19e670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecc000eb90 tx=0x7fecc000eea0 comp rx=0 tx=0).stop 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 shutdown_connections 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fecbc0a2890 0x7fecbc0a4d50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feccc108340 0x7feccc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feccc102360 0x7feccc19e670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 --2- 172.21.15.7:0/1241614442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feccc102d20 0x7feccc19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 >> 172.21.15.7:0/1241614442 conn(0x7feccc079440 msgr2=0x7feccc101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 shutdown_connections 2024-03-19T00:38:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:21.208+0000 7fecd2f00700 1 -- 172.21.15.7:0/1241614442 wait complete. 2024-03-19T00:38:21.951 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-19T00:38:21.951 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-19T00:38:21.951 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-03-19T00:38:21.951 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-19T00:38:21.951 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:21.951 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-19T00:38:21.952 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-19T00:38:21.952 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-19T00:38:21.952 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-19T00:38:21.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:21.952 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-03-19T00:38:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 40497sec preferred_lft 40497sec 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-03-19T00:38:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-03-19T00:38:21.970 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-19T00:38:21.970 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:38:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:38:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:38:22.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:21 smithi007 ceph-mon[62854]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:38:22.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:21 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1241614442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T00:38:22.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:22.120 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-19T00:38:22.146 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-03-19T00:38:22.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:22.148 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-03-19T00:38:22.203 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-03-19T00:38:22.203 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-19T00:38:22.203 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-19T00:38:22.203 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-19T00:38:22.203 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-19T00:38:22.204 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:38:22.204 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:38:22.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-19T00:38:22.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-19T00:38:22.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-19T00:38:22.204 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:38:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:21 smithi204 ceph-mon[66572]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:38:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:21 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1241614442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T00:38:22.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:38:22.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:38:22.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:22.339 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-19T00:38:22.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:22.354 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-03-19T00:38:22.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:21 smithi044 ceph-mon[66366]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:38:22.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:21 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1241614442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T00:38:22.408 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:38:22.408 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:38:22.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T00:38:22.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-19T00:38:22.408 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:38:22.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:38:22.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:38:22.529 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-19T00:38:22.530 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:38:22.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:38:22.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:38:22.655 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T00:38:22.655 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-19T00:38:22.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-03-19T00:38:22.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-03-19T00:38:22.655 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-19T00:38:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T00:38:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:38:22.736 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-19T00:38:22.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T00:38:22.737 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:22.751 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-19T00:38:22.751 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T00:38:22.751 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:22.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-03-19T00:38:22.873 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-03-19T00:38:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T00:38:22.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/eec6309f9aca95b52ece73994d8f5d8b19c016041c13a5d45577041ea9cf572b/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7e5bdd392ddb07b923686c020cd726eb8fd317947179f61b0ac1a40d7484cd0b/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d9a7c5ec22cb784f8af7efb9e6bb98977c51267bea00372c35bce5265d7204f2/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/b06edf6b7387b2971e764045f906446479b51a30e663b6dc282089af447ed081/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/32ff72bd23cb8453db9034251ada47f5b3f7f0ca02b79e5fc8329e3d2b7a7204/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/6a03db7480c6af03d4d2e805849736a351c69aab279a0c4a53db007a91fe6f02/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/346b7b69993fe11455a1b65856b99e55508617c5081222e7f146a136f8729de1/merged 2024-03-19T00:38:22.931 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/1ba5f749cb32564e79be73311e5836c1e1fb86dfcb258c79c6f2286ac02e6bb7/merged 2024-03-19T00:38:22.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0f9dd57b3e832766d06a403a3b37144dc77bf5b27b2c821d4e450fed440a939a/merged 2024-03-19T00:38:22.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/748dacd042f2c81e25c54a6c38828b7b5b65def2a6cb9624d4b39f23c50d803f/merged 2024-03-19T00:38:22.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-03-19T00:38:22.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T00:38:22.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T00:38:22.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:22.932 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-03-19T00:38:22.984 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-19T00:38:22.984 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/1e7a6851-7e82-4590-b004-54f2635f2344 --client_fs=cephfs) 2024-03-19T00:38:23.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-03-19T00:38:23.052 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T00:38:23.108 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-03-19T00:38:23.109 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T00:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/eec6309f9aca95b52ece73994d8f5d8b19c016041c13a5d45577041ea9cf572b/merged 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7e5bdd392ddb07b923686c020cd726eb8fd317947179f61b0ac1a40d7484cd0b/merged 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d9a7c5ec22cb784f8af7efb9e6bb98977c51267bea00372c35bce5265d7204f2/merged 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/b06edf6b7387b2971e764045f906446479b51a30e663b6dc282089af447ed081/merged 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/32ff72bd23cb8453db9034251ada47f5b3f7f0ca02b79e5fc8329e3d2b7a7204/merged 2024-03-19T00:38:23.111 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/6a03db7480c6af03d4d2e805849736a351c69aab279a0c4a53db007a91fe6f02/merged 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/346b7b69993fe11455a1b65856b99e55508617c5081222e7f146a136f8729de1/merged 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/1ba5f749cb32564e79be73311e5836c1e1fb86dfcb258c79c6f2286ac02e6bb7/merged 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0f9dd57b3e832766d06a403a3b37144dc77bf5b27b2c821d4e450fed440a939a/merged 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/748dacd042f2c81e25c54a6c38828b7b5b65def2a6cb9624d4b39f23c50d803f/merged 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T00:38:23.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:23.113 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-03-19T00:38:23.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2024-03-19T00:38:23.112+0000 7f6c1e03c480 -1 init, newargv = 0x5626ccd4c380 newargc=15 2024-03-19T00:38:23.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[113479]: starting ceph client 2024-03-19T00:38:23.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[113479]: starting fuse 2024-03-19T00:38:23.128 INFO:teuthology.orchestra.run.smithi007.stdout:119 2024-03-19T00:38:23.128 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-03-19T00:38:23.129 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2024-03-19T00:38:23.130 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-03-19T00:38:23.252 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.113479.asok 2024-03-19T00:38:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T00:38:23.262 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113479.asok 2024-03-19T00:38:23.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:23.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113479.asok status 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "b6ae24918b03b1b8a19c9263a50655bba80bb810", 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-19T00:38:23.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "113479", 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/volumes/qa/sv_0/1e7a6851-7e82-4590-b004-54f2635f2344" 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 15468, 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 15468 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 340634494 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:38:23.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 340634494 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.15468 192.168.0.1:0/340634494", 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/340634494", 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 5, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 24, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 91, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-03-19T00:38:23.402 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:38:23.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:23.409 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-03-19T00:38:23.409 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:23.424 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-03-19T00:38:23.424 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:23.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:23.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:23.491 INFO:teuthology.run_tasks:Running task exec... 2024-03-19T00:38:23.502 INFO:teuthology.task.exec:Executing custom commands... 2024-03-19T00:38:23.502 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-19T00:38:23.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 -- 172.21.15.7:0/3611671112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 msgr2=0x7f6ba81009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 --2- 172.21.15.7:0/3611671112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81009d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009900 tx=0x7f6b9802bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 -- 172.21.15.7:0/3611671112 shutdown_connections 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 --2- 172.21.15.7:0/3611671112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8102160 0x7f6ba81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 --2- 172.21.15.7:0/3611671112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ba81017a0 0x7f6ba8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 --2- 172.21.15.7:0/3611671112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81009d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 -- 172.21.15.7:0/3611671112 >> 172.21.15.7:0/3611671112 conn(0x7f6ba80fbb40 msgr2=0x7f6ba80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 -- 172.21.15.7:0/3611671112 shutdown_connections 2024-03-19T00:38:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.634+0000 7f6bad801640 1 -- 172.21.15.7:0/3611671112 wait complete. 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 Processor -- start 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 -- start start 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ba81017a0 0x7f6ba8196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8102160 0x7f6ba819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810c0d0 con 0x7f6ba81005b0 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810bf50 con 0x7f6ba81017a0 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6bad801640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810c250 con 0x7f6ba8102160 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46560/0 (socket says 172.21.15.7:46560) 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 -- 172.21.15.7:0/3675680276 learned_addr learned my addr 172.21.15.7:0/3675680276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6b9ffff640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ba81017a0 0x7f6ba8196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba77fe640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8102160 0x7f6ba819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 -- 172.21.15.7:0/3675680276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8102160 msgr2=0x7f6ba819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8102160 0x7f6ba819dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 -- 172.21.15.7:0/3675680276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ba81017a0 msgr2=0x7f6ba8196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ba81017a0 0x7f6ba8196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 -- 172.21.15.7:0/3675680276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba819e400 con 0x7f6ba81005b0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba6ffd640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81967e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b98005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.635+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98043020 con 0x7f6ba81005b0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.636+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b98004350 con 0x7f6ba81005b0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.636+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba819e690 con 0x7f6ba81005b0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.636+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98003b80 con 0x7f6ba81005b0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.636+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba819ebe0 con 0x7f6ba81005b0 2024-03-19T00:38:23.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.637+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba81079c0 con 0x7f6ba81005b0 2024-03-19T00:38:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.638+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f6b98046020 con 0x7f6ba81005b0 2024-03-19T00:38:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.639+0000 7f6ba4ff9640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b940a27e0 0x7f6b940a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.639+0000 7f6b9ffff640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b940a27e0 0x7f6b940a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.639+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6b980f12a0 con 0x7f6ba81005b0 2024-03-19T00:38:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.639+0000 7f6b9ffff640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b940a27e0 0x7f6b940a4ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8197cf0 tx=0x7f6b90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.643+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b9804d020 con 0x7f6ba81005b0 2024-03-19T00:38:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.869+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f6ba802d1f0 con 0x7f6ba81005b0 2024-03-19T00:38:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.891+0000 7f6ba4ff9640 1 -- 172.21.15.7:0/3675680276 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f6b980bdfe0 con 0x7f6ba81005b0 2024-03-19T00:38:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b940a27e0 msgr2=0x7f6b940a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b940a27e0 0x7f6b940a4ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8197cf0 tx=0x7f6b90005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 msgr2=0x7f6ba81967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81967e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b98005dc0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 shutdown_connections 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b940a27e0 0x7f6b940a4ca0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8102160 0x7f6ba819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ba81017a0 0x7f6ba8196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 --2- 172.21.15.7:0/3675680276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ba81005b0 0x7f6ba81967e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 >> 172.21.15.7:0/3675680276 conn(0x7f6ba80fbb40 msgr2=0x7f6ba8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 shutdown_connections 2024-03-19T00:38:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:23.893+0000 7f6bad801640 1 -- 172.21.15.7:0/3675680276 wait complete. 2024-03-19T00:38:23.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-19T00:38:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 -- 172.21.15.7:0/2780390266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64100790 msgr2=0x7f9c64100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 --2- 172.21.15.7:0/2780390266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64100790 0x7f9c64100bb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009900 tx=0x7f9c5402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 -- 172.21.15.7:0/2780390266 shutdown_connections 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 --2- 172.21.15.7:0/2780390266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c64102340 0x7f9c64108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 --2- 172.21.15.7:0/2780390266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c64101980 0x7f9c64101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 --2- 172.21.15.7:0/2780390266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64100790 0x7f9c64100bb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 -- 172.21.15.7:0/2780390266 >> 172.21.15.7:0/2780390266 conn(0x7f9c640fbd00 msgr2=0x7f9c640fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 -- 172.21.15.7:0/2780390266 shutdown_connections 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 -- 172.21.15.7:0/2780390266 wait complete. 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 Processor -- start 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.009+0000 7f9c694b2640 1 -- start start 2024-03-19T00:38:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c64100790 0x7f9c64197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 0x7f9c64198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c64102340 0x7f9c6419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410d3a0 con 0x7f9c64101980 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410d220 con 0x7f9c64100790 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410d520 con 0x7f9c64102340 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 0x7f9c64198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 0x7f9c64198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46568/0 (socket says 172.21.15.7:46568) 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 -- 172.21.15.7:0/933692200 learned_addr learned my addr 172.21.15.7:0/933692200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c62ffd640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c64100790 0x7f9c64197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c637fe640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c64102340 0x7f9c6419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 -- 172.21.15.7:0/933692200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c64102340 msgr2=0x7f9c6419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c64102340 0x7f9c6419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 -- 172.21.15.7:0/933692200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c64100790 msgr2=0x7f9c64197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c64100790 0x7f9c64197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 -- 172.21.15.7:0/933692200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6419f740 con 0x7f9c64101980 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c627fc640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 0x7f9c64198060 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800e9f0 tx=0x7f9c5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800ccc0 con 0x7f9c64101980 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c580108e0 con 0x7f9c64101980 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c580187c0 con 0x7f9c64101980 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6419fa30 con 0x7f9c64101980 2024-03-19T00:38:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.010+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6419ff30 con 0x7f9c64101980 2024-03-19T00:38:24.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.011+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c64107bd0 con 0x7f9c64101980 2024-03-19T00:38:24.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.012+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f9c58010a90 con 0x7f9c64101980 2024-03-19T00:38:24.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.013+0000 7f9c4bfff640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c4c0a27f0 0x7f9c4c0a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.013+0000 7f9c62ffd640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c4c0a27f0 0x7f9c4c0a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.013+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9c580cf670 con 0x7f9c64101980 2024-03-19T00:38:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.014+0000 7f9c62ffd640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c4c0a27f0 0x7f9c4c0a4cb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.017+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c5809c350 con 0x7f9c64101980 2024-03-19T00:38:24.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:23 smithi007 ceph-mon[62854]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:23 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3675680276' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T00:38:24.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.230+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9c64061dc0 con 0x7f9c4c0a27f0 2024-03-19T00:38:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.247+0000 7f9c4bfff640 1 -- 172.21.15.7:0/933692200 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f9c64061dc0 con 0x7f9c4c0a27f0 2024-03-19T00:38:24.249 INFO:teuthology.orchestra.run.smithi007.stdout:{} 2024-03-19T00:38:24.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c4c0a27f0 msgr2=0x7f9c4c0a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c4c0a27f0 0x7f9c4c0a4cb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c54005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 msgr2=0x7f9c64198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 0x7f9c64198060 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800e9f0 tx=0x7f9c5800edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 shutdown_connections 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c4c0a27f0 0x7f9c4c0a4cb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c64102340 0x7f9c6419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c64101980 0x7f9c64198060 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 --2- 172.21.15.7:0/933692200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c64100790 0x7f9c64197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 >> 172.21.15.7:0/933692200 conn(0x7f9c640fbd00 msgr2=0x7f9c64104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 shutdown_connections 2024-03-19T00:38:24.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.249+0000 7f9c694b2640 1 -- 172.21.15.7:0/933692200 wait complete. 2024-03-19T00:38:24.263 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-19T00:38:24.274 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-19T00:38:24.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:24.323 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:23 smithi204 ceph-mon[66572]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:24.324 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:23 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3675680276' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T00:38:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:23 smithi044 ceph-mon[66366]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:23 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3675680276' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T00:38:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 -- 172.21.15.7:0/3219080157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c1022d0 msgr2=0x7f367c1046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 --2- 172.21.15.7:0/3219080157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c1022d0 0x7f367c1046c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3674009900 tx=0x7f367402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 -- 172.21.15.7:0/3219080157 shutdown_connections 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 --2- 172.21.15.7:0/3219080157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f367c107530 0x7f367c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 --2- 172.21.15.7:0/3219080157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c104c00 0x7f367c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 --2- 172.21.15.7:0/3219080157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c1022d0 0x7f367c1046c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 -- 172.21.15.7:0/3219080157 >> 172.21.15.7:0/3219080157 conn(0x7f367c0fbce0 msgr2=0x7f367c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 -- 172.21.15.7:0/3219080157 shutdown_connections 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.424+0000 7f3681027640 1 -- 172.21.15.7:0/3219080157 wait complete. 2024-03-19T00:38:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 Processor -- start 2024-03-19T00:38:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 -- start start 2024-03-19T00:38:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c1022d0 0x7f367c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f367c104c00 0x7f367c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 0x7f367c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f367c10d380 con 0x7f367c107530 2024-03-19T00:38:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f367c10d200 con 0x7f367c104c00 2024-03-19T00:38:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f3681027640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f367c10d500 con 0x7f367c1022d0 2024-03-19T00:38:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f367b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 0x7f367c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f367b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 0x7f367c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46620/0 (socket says 172.21.15.7:46620) 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f367b59e640 1 -- 172.21.15.7:0/1959333790 learned_addr learned my addr 172.21.15.7:0/1959333790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f367a59c640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f367c104c00 0x7f367c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.425+0000 7f367ad9d640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c1022d0 0x7f367c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367b59e640 1 -- 172.21.15.7:0/1959333790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c1022d0 msgr2=0x7f367c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367b59e640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c1022d0 0x7f367c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367b59e640 1 -- 172.21.15.7:0/1959333790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f367c104c00 msgr2=0x7f367c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367b59e640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f367c104c00 0x7f367c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367b59e640 1 -- 172.21.15.7:0/1959333790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c19f6f0 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367ad9d640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c1022d0 0x7f367c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f367b59e640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 0x7f367c19f080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f366c00efd0 tx=0x7f366c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c003b70 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c19f9e0 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f366c003d10 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c010980 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.426+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c19fee0 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.427+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c0fd7d0 con 0x7f367c107530 2024-03-19T00:38:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.428+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f366c004b40 con 0x7f367c107530 2024-03-19T00:38:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.429+0000 7f3663fff640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36640a2800 0x7f36640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.429+0000 7f367ad9d640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36640a2800 0x7f36640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.429+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f366c0cf600 con 0x7f367c107530 2024-03-19T00:38:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.429+0000 7f367ad9d640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36640a2800 0x7f36640a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3674009230 tx=0x7f3674005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.432+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f366c09c2e0 con 0x7f367c107530 2024-03-19T00:38:24.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.616+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f367c02d440 con 0x7f367c107530 2024-03-19T00:38:24.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.618+0000 7f3663fff640 1 -- 172.21.15.7:0/1959333790 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f366c09bbe0 con 0x7f367c107530 2024-03-19T00:38:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:24.620 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36640a2800 msgr2=0x7f36640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36640a2800 0x7f36640a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3674009230 tx=0x7f3674005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 msgr2=0x7f367c19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 0x7f367c19f080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f366c00efd0 tx=0x7f366c00c6a0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 shutdown_connections 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36640a2800 0x7f36640a4cc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c107530 0x7f367c19f080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f367c104c00 0x7f367c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 --2- 172.21.15.7:0/1959333790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f367c1022d0 0x7f367c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 >> 172.21.15.7:0/1959333790 conn(0x7f367c0fbce0 msgr2=0x7f367c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 shutdown_connections 2024-03-19T00:38:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.620+0000 7f3681027640 1 -- 172.21.15.7:0/1959333790 wait complete. 2024-03-19T00:38:24.632 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:38:24.234+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:38:24.632 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:38:24.234+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:38:24.632 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:38:24.234+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:38:24.632 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:38:24.234+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:38:24.632 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:38:24.234+0000 7fc867e7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T00:38:24.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:38:24.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:38:24.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/2494805325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a81033e0 msgr2=0x7ff5a8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/2494805325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a81033e0 0x7ff5a8109c70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009ca0 tx=0x7ff5a002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/2494805325 shutdown_connections 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/2494805325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a81033e0 0x7ff5a8109c70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/2494805325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a8102a20 0x7ff5a8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/2494805325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5a8101830 0x7ff5a8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/2494805325 >> 172.21.15.7:0/2494805325 conn(0x7ff5a80fcda0 msgr2=0x7ff5a80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/2494805325 shutdown_connections 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.738+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/2494805325 wait complete. 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 Processor -- start 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 -- start start 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 0x7ff5a8197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a8102a20 0x7ff5a8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5a81033e0 0x7ff5a819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810d360 con 0x7ff5a8101830 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810d1e0 con 0x7ff5a8102a20 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810d4e0 con 0x7ff5a81033e0 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 0x7ff5a8197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 0x7ff5a8197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42636/0 (socket says 172.21.15.7:42636) 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 -- 172.21.15.7:0/308579699 learned_addr learned my addr 172.21.15.7:0/308579699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff59ffff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a8102a20 0x7ff5a8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a759e640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5a81033e0 0x7ff5a819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 -- 172.21.15.7:0/308579699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5a81033e0 msgr2=0x7ff5a819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5a81033e0 0x7ff5a819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 -- 172.21.15.7:0/308579699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a8102a20 msgr2=0x7ff5a8197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a8102a20 0x7ff5a8197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 -- 172.21.15.7:0/308579699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a819f650 con 0x7ff5a8101830 2024-03-19T00:38:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a6d9d640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 0x7ff5a8197a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e9c0 tx=0x7ff59800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59800cd80 con 0x7ff5a8101830 2024-03-19T00:38:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5980035f0 con 0x7ff5a8101830 2024-03-19T00:38:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.739+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a819f940 con 0x7ff5a8101830 2024-03-19T00:38:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.740+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598010690 con 0x7ff5a8101830 2024-03-19T00:38:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.740+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a819fe40 con 0x7ff5a8101830 2024-03-19T00:38:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.740+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a8108b50 con 0x7ff5a8101830 2024-03-19T00:38:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.742+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7ff5980037a0 con 0x7ff5a8101830 2024-03-19T00:38:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.742+0000 7ff5a4d99640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff5940a2800 0x7ff5940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.743+0000 7ff59ffff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff5940a2800 0x7ff5940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.743+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff5980cf430 con 0x7ff5a8101830 2024-03-19T00:38:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.743+0000 7ff59ffff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff5940a2800 0x7ff5940a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5900095f0 tx=0x7ff590005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.746+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff59809c170 con 0x7ff5a8101830 2024-03-19T00:38:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.931+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5a802d3a0 con 0x7ff5a8101830 2024-03-19T00:38:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.931+0000 7ff5a4d99640 1 -- 172.21.15.7:0/308579699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ff59809ba70 con 0x7ff5a8101830 2024-03-19T00:38:24.934 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff5940a2800 msgr2=0x7ff5940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff5940a2800 0x7ff5940a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5900095f0 tx=0x7ff590005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 msgr2=0x7ff5a8197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 0x7ff5a8197a30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e9c0 tx=0x7ff59800ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 shutdown_connections 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff5940a2800 0x7ff5940a4cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5a81033e0 0x7ff5a819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5a8102a20 0x7ff5a8197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 --2- 172.21.15.7:0/308579699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a8101830 0x7ff5a8197a30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.933+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 >> 172.21.15.7:0/308579699 conn(0x7ff5a80fcda0 msgr2=0x7ff5a8105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.934+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 shutdown_connections 2024-03-19T00:38:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:24.934+0000 7ff5ad0ff640 1 -- 172.21.15.7:0/308579699 wait complete. 2024-03-19T00:38:24.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- 172.21.15.7:0/35248559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 msgr2=0x7f5504102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- 172.21.15.7:0/35248559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504102ea0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009900 tx=0x7f54f002bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- 172.21.15.7:0/35248559 shutdown_connections 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- 172.21.15.7:0/35248559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55041033e0 0x7f5504109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- 172.21.15.7:0/35248559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504102ea0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- 172.21.15.7:0/35248559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 0x7f5504101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- 172.21.15.7:0/35248559 >> 172.21.15.7:0/35248559 conn(0x7f55040fcda0 msgr2=0x7f55040ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- 172.21.15.7:0/35248559 shutdown_connections 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- 172.21.15.7:0/35248559 wait complete. 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 Processor -- start 2024-03-19T00:38:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- start start 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 0x7f5504197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55041033e0 0x7f550419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f550410d360 con 0x7f5504102a20 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f550410d1e0 con 0x7f5504101830 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.051+0000 7f550b236640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f550410d4e0 con 0x7f55041033e0 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42662/0 (socket says 172.21.15.7:42662) 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 -- 172.21.15.7:0/2055649624 learned_addr learned my addr 172.21.15.7:0/2055649624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5508fd2640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 0x7f5504197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f55097d3640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55041033e0 0x7f550419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 -- 172.21.15.7:0/2055649624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55041033e0 msgr2=0x7f550419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55041033e0 0x7f550419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 -- 172.21.15.7:0/2055649624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 msgr2=0x7f5504197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 0x7f5504197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 -- 172.21.15.7:0/2055649624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550419f660 con 0x7f5504102a20 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5508fd2640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 0x7f5504197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5503fff640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504197f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54f0006010 tx=0x7f54f0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0043020 con 0x7f5504102a20 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f00043d0 con 0x7f5504102a20 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550419f8f0 con 0x7f5504102a20 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0003b80 con 0x7f5504102a20 2024-03-19T00:38:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.052+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550419fe90 con 0x7f5504102a20 2024-03-19T00:38:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.053+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5504108bd0 con 0x7f5504102a20 2024-03-19T00:38:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.054+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f54f0046020 con 0x7f5504102a20 2024-03-19T00:38:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.055+0000 7f5501ffb640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54f40a2880 0x7f54f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.055+0000 7f5508fd2640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54f40a2880 0x7f54f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.055+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f54f00f0f30 con 0x7f5504102a20 2024-03-19T00:38:25.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.056+0000 7f5508fd2640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54f40a2880 0x7f54f40a4d40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5504102880 tx=0x7f54f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.060+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f54f004d020 con 0x7f5504102a20 2024-03-19T00:38:25.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3675680276' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T00:38:25.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:38:25.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:24 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1959333790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:25.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.244+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5504062420 con 0x7f5504102a20 2024-03-19T00:38:25.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.245+0000 7f5501ffb640 1 -- 172.21.15.7:0/2055649624 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f54f00bdc70 con 0x7f5504102a20 2024-03-19T00:38:25.247 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54f40a2880 msgr2=0x7f54f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54f40a2880 0x7f54f40a4d40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5504102880 tx=0x7f54f8005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 msgr2=0x7f5504197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504197f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54f0006010 tx=0x7f54f0005e40 comp rx=0 tx=0).stop 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 shutdown_connections 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54f40a2880 0x7f54f40a4d40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55041033e0 0x7f550419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5504102a20 0x7f5504197f80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 --2- 172.21.15.7:0/2055649624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5504101830 0x7f5504197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 >> 172.21.15.7:0/2055649624 conn(0x7f55040fcda0 msgr2=0x7f5504105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 shutdown_connections 2024-03-19T00:38:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.247+0000 7f550b236640 1 -- 172.21.15.7:0/2055649624 wait complete. 2024-03-19T00:38:25.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:38:25.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3675680276' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T00:38:25.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:24 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:38:25.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:24 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1959333790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:25.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3675680276' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T00:38:25.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:24 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:38:25.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:24 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1959333790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.364+0000 7fbd7a244640 1 -- 172.21.15.7:0/2029052058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd741022e0 msgr2=0x7fbd741046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.364+0000 7fbd7a244640 1 --2- 172.21.15.7:0/2029052058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd741022e0 0x7fbd741046d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009860 tx=0x7fbd6402bb70 comp rx=0 tx=0).stop 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 -- 172.21.15.7:0/2029052058 shutdown_connections 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 --2- 172.21.15.7:0/2029052058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd74109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 --2- 172.21.15.7:0/2029052058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd74104c10 0x7fbd74107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 --2- 172.21.15.7:0/2029052058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd741022e0 0x7fbd741046d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 -- 172.21.15.7:0/2029052058 >> 172.21.15.7:0/2029052058 conn(0x7fbd740fbcb0 msgr2=0x7fbd740fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 -- 172.21.15.7:0/2029052058 shutdown_connections 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 -- 172.21.15.7:0/2029052058 wait complete. 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 Processor -- start 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 -- start start 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd741022e0 0x7fbd74197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd74104c10 0x7fbd74198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd7419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.365+0000 7fbd7a244640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7410d3a0 con 0x7fbd74107540 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd7a244640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7410d220 con 0x7fbd74104c10 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd7a244640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7410d520 con 0x7fbd741022e0 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd7419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd7419f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42678/0 (socket says 172.21.15.7:42678) 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd74104c10 0x7fbd74198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 -- 172.21.15.7:0/3653955059 learned_addr learned my addr 172.21.15.7:0/3653955059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd737fe640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd741022e0 0x7fbd74197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 -- 172.21.15.7:0/3653955059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd741022e0 msgr2=0x7fbd74197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd741022e0 0x7fbd74197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 -- 172.21.15.7:0/3653955059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd74104c10 msgr2=0x7fbd74198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd74104c10 0x7fbd74198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 -- 172.21.15.7:0/3653955059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7419f740 con 0x7fbd74107540 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd72ffd640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd74104c10 0x7fbd74198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd73fff640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd7419f0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00efc0 tx=0x7fbd6c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003970 con 0x7fbd74107540 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd6c004020 con 0x7fbd74107540 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7419fa30 con 0x7fbd74107540 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00b940 con 0x7fbd74107540 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.366+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7419ff30 con 0x7fbd74107540 2024-03-19T00:38:25.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.367+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd740fd7a0 con 0x7fbd74107540 2024-03-19T00:38:25.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.368+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7fbd6c01d020 con 0x7fbd74107540 2024-03-19T00:38:25.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.369+0000 7fbd70ff9640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd5c0a2800 0x7fbd5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.369+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd6c0cec30 con 0x7fbd74107540 2024-03-19T00:38:25.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.369+0000 7fbd737fe640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd5c0a2800 0x7fbd5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.370+0000 7fbd737fe640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd5c0a2800 0x7fbd5c0a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd64000c00 tx=0x7fbd640349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.373+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd6c09b970 con 0x7fbd74107540 2024-03-19T00:38:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.557+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd74062420 con 0x7fbd74107540 2024-03-19T00:38:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.557+0000 7fbd70ff9640 1 -- 172.21.15.7:0/3653955059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fbd6c09b270 con 0x7fbd74107540 2024-03-19T00:38:25.560 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:38:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd5c0a2800 msgr2=0x7fbd5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd5c0a2800 0x7fbd5c0a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd64000c00 tx=0x7fbd640349a0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 msgr2=0x7fbd7419f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd7419f0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00efc0 tx=0x7fbd6c00c800 comp rx=0 tx=0).stop 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 shutdown_connections 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbd5c0a2800 0x7fbd5c0a4cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd74107540 0x7fbd7419f0d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd74104c10 0x7fbd74198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 --2- 172.21.15.7:0/3653955059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd741022e0 0x7fbd74197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 >> 172.21.15.7:0/3653955059 conn(0x7fbd740fbcb0 msgr2=0x7fbd74102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 shutdown_connections 2024-03-19T00:38:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.559+0000 7fbd7a244640 1 -- 172.21.15.7:0/3653955059 wait complete. 2024-03-19T00:38:25.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:38:25.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 -- 172.21.15.7:0/133398454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c41022f0 msgr2=0x7f51c4108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/133398454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c41022f0 0x7f51c4108b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009be0 tx=0x7f51bc02bef0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 -- 172.21.15.7:0/133398454 shutdown_connections 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/133398454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c41022f0 0x7f51c4108b80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/133398454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51c4101930 0x7f51c4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/133398454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c4100740 0x7f51c4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 -- 172.21.15.7:0/133398454 >> 172.21.15.7:0/133398454 conn(0x7f51c40fbcb0 msgr2=0x7f51c40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 -- 172.21.15.7:0/133398454 shutdown_connections 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.676+0000 7f51cb3d4640 1 -- 172.21.15.7:0/133398454 wait complete. 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 Processor -- start 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 -- start start 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 0x7f51c4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c4101930 0x7f51c4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51c41022f0 0x7f51c419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51c410c270 con 0x7f51c4100740 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51c410c0f0 con 0x7f51c41022f0 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51c410c3f0 con 0x7f51c4101930 2024-03-19T00:38:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 0x7f51c4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 0x7f51c4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42698/0 (socket says 172.21.15.7:42698) 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 -- 172.21.15.7:0/31227474 learned_addr learned my addr 172.21.15.7:0/31227474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c896f640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c4101930 0x7f51c4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9971640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51c41022f0 0x7f51c419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 -- 172.21.15.7:0/31227474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c4101930 msgr2=0x7f51c4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c4101930 0x7f51c4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 -- 172.21.15.7:0/31227474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51c41022f0 msgr2=0x7f51c419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51c41022f0 0x7f51c419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 -- 172.21.15.7:0/31227474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c419f660 con 0x7f51c4100740 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51c9170640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 0x7f51c4197a40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e9c0 tx=0x7f51c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c000cd80 con 0x7f51c4100740 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51c00035f0 con 0x7f51c4100740 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0010690 con 0x7f51c4100740 2024-03-19T00:38:25.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.677+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c419f950 con 0x7f51c4100740 2024-03-19T00:38:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.678+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c419fea0 con 0x7f51c4100740 2024-03-19T00:38:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.679+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c41a0130 con 0x7f51c4100740 2024-03-19T00:38:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.679+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137289+0+0 (secure 0 0 0) 0x7f51c00037a0 con 0x7f51c4100740 2024-03-19T00:38:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.680+0000 7f51ba7fc640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51b00a2880 0x7f51b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.680+0000 7f51c896f640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51b00a2880 0x7f51b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.680+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f51c00cf260 con 0x7f51c4100740 2024-03-19T00:38:25.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.680+0000 7f51c896f640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51b00a2880 0x7f51b00a4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51c4198f50 tx=0x7f51b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.684+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f51c009bfa0 con 0x7f51c4100740 2024-03-19T00:38:25.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.867+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f51c41a0300 con 0x7f51c4100740 2024-03-19T00:38:25.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.867+0000 7f51ba7fc640 1 -- 172.21.15.7:0/31227474 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f51c009b8a0 con 0x7f51c4100740 2024-03-19T00:38:25.869 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:25.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51b00a2880 msgr2=0x7f51b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51b00a2880 0x7f51b00a4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51c4198f50 tx=0x7f51b4009260 comp rx=0 tx=0).stop 2024-03-19T00:38:25.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 msgr2=0x7f51c4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 0x7f51c4197a40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e9c0 tx=0x7f51c000ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 shutdown_connections 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51b00a2880 0x7f51b00a4d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51c41022f0 0x7f51c419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c4101930 0x7f51c4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 --2- 172.21.15.7:0/31227474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51c4100740 0x7f51c4197a40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 >> 172.21.15.7:0/31227474 conn(0x7f51c40fbcb0 msgr2=0x7f51c40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 shutdown_connections 2024-03-19T00:38:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.869+0000 7f51cb3d4640 1 -- 172.21.15.7:0/31227474 wait complete. 2024-03-19T00:38:25.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:25.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 -- 172.21.15.7:0/303183723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508102300 msgr2=0x7f2508108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 --2- 172.21.15.7:0/303183723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508102300 0x7f2508108b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2500009ca0 tx=0x7f250002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 -- 172.21.15.7:0/303183723 shutdown_connections 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 --2- 172.21.15.7:0/303183723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508102300 0x7f2508108b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 --2- 172.21.15.7:0/303183723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2508101940 0x7f2508101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 --2- 172.21.15.7:0/303183723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2508100750 0x7f2508100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 -- 172.21.15.7:0/303183723 >> 172.21.15.7:0/303183723 conn(0x7f25080fbcc0 msgr2=0x7f25080fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 -- 172.21.15.7:0/303183723 shutdown_connections 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.987+0000 7f250def8640 1 -- 172.21.15.7:0/303183723 wait complete. 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 Processor -- start 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 -- start start 2024-03-19T00:38:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2508100750 0x7f25081968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 0x7f2508196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2508102300 0x7f250819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f250810c260 con 0x7f2508101940 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f250810c0e0 con 0x7f2508102300 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f250def8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f250810c3e0 con 0x7f2508100750 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 0x7f2508196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 0x7f2508196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42710/0 (socket says 172.21.15.7:42710) 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 -- 172.21.15.7:0/3757839174 learned_addr learned my addr 172.21.15.7:0/3757839174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f25077fe640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2508100750 0x7f25081968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2507fff640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2508102300 0x7f250819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 -- 172.21.15.7:0/3757839174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2508100750 msgr2=0x7f25081968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2508100750 0x7f25081968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 -- 172.21.15.7:0/3757839174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2508102300 msgr2=0x7f250819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2508102300 0x7f250819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.988+0000 7f2506ffd640 1 -- 172.21.15.7:0/3757839174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250819e4d0 con 0x7f2508101940 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.989+0000 7f2506ffd640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 0x7f2508196df0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e9f0 tx=0x7f24f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.989+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800cd80 con 0x7f2508101940 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.989+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24f80108e0 con 0x7f2508101940 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.989+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8018920 con 0x7f2508101940 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.989+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250819e7c0 con 0x7f2508101940 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.989+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250819ed10 con 0x7f2508101940 2024-03-19T00:38:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.990+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2508107a70 con 0x7f2508101940 2024-03-19T00:38:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.991+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f24f8010a90 con 0x7f2508101940 2024-03-19T00:38:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.992+0000 7f2504ff9640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f24f00a2890 0x7f24f00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.992+0000 7f25077fe640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f24f00a2890 0x7f24f00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.992+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f24f80cf240 con 0x7f2508101940 2024-03-19T00:38:25.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.992+0000 7f25077fe640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f24f00a2890 0x7f24f00a4d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25081017a0 tx=0x7f24fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:25.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:25.996+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f24f809b7d0 con 0x7f2508101940 2024-03-19T00:38:26.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.185+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f250802d430 con 0x7f2508101940 2024-03-19T00:38:26.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.187+0000 7f2504ff9640 1 -- 172.21.15.7:0/3757839174 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f24f809f870 con 0x7f2508101940 2024-03-19T00:38:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:26.189 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f24f00a2890 msgr2=0x7f24f00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f24f00a2890 0x7f24f00a4d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25081017a0 tx=0x7f24fc005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 msgr2=0x7f2508196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 0x7f2508196df0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e9f0 tx=0x7f24f800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 shutdown_connections 2024-03-19T00:38:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f24f00a2890 0x7f24f00a4d50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2508102300 0x7f250819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2508101940 0x7f2508196df0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 --2- 172.21.15.7:0/3757839174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2508100750 0x7f25081968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 >> 172.21.15.7:0/3757839174 conn(0x7f25080fbcc0 msgr2=0x7f2508104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 shutdown_connections 2024-03-19T00:38:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.189+0000 7f250def8640 1 -- 172.21.15.7:0/3757839174 wait complete. 2024-03-19T00:38:26.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:25 smithi007 ceph-mon[62854]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-19T00:38:26.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:25 smithi007 ceph-mon[62854]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:26.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/308579699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2055649624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3653955059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.201 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/31227474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.203 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:38:26.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:38:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:25 smithi204 ceph-mon[66572]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-19T00:38:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:25 smithi204 ceph-mon[66572]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:25 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/308579699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:25 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2055649624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:25 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3653955059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:25 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/31227474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 -- 172.21.15.7:0/4142891412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e4101930 msgr2=0x7f96e4101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 --2- 172.21.15.7:0/4142891412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e4101930 0x7f96e4101db0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009900 tx=0x7f96cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 -- 172.21.15.7:0/4142891412 shutdown_connections 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 --2- 172.21.15.7:0/4142891412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e41022f0 0x7f96e4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 --2- 172.21.15.7:0/4142891412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e4101930 0x7f96e4101db0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 --2- 172.21.15.7:0/4142891412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96e4100740 0x7f96e4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 -- 172.21.15.7:0/4142891412 >> 172.21.15.7:0/4142891412 conn(0x7f96e40fbcb0 msgr2=0x7f96e40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 -- 172.21.15.7:0/4142891412 shutdown_connections 2024-03-19T00:38:26.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.307+0000 7f96e8f87640 1 -- 172.21.15.7:0/4142891412 wait complete. 2024-03-19T00:38:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 Processor -- start 2024-03-19T00:38:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 -- start start 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96e4100740 0x7f96e406aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e4101930 0x7f96e419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 0x7f96e419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410c270 con 0x7f96e41022f0 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410c0f0 con 0x7f96e4101930 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e8f87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410c3f0 con 0x7f96e4100740 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 0x7f96e419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 0x7f96e419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42720/0 (socket says 172.21.15.7:42720) 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 -- 172.21.15.7:0/1919976854 learned_addr learned my addr 172.21.15.7:0/1919976854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e259c640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96e4100740 0x7f96e406aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96d9d9b640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e4101930 0x7f96e419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 -- 172.21.15.7:0/1919976854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96e4100740 msgr2=0x7f96e406aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96e4100740 0x7f96e406aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 -- 172.21.15.7:0/1919976854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e4101930 msgr2=0x7f96e419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e4101930 0x7f96e419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.308+0000 7f96e2d9d640 1 -- 172.21.15.7:0/1919976854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e419f6d0 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.309+0000 7f96e2d9d640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 0x7f96e419ee10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00eb10 tx=0x7f96dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.309+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc004b20 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.309+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e419f9c0 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.309+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96dc004cc0 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.309+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc010690 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.309+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e419fe90 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.310+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e4107a80 con 0x7f96e41022f0 2024-03-19T00:38:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.311+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f96dc005020 con 0x7f96e41022f0 2024-03-19T00:38:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.311+0000 7f96dbfff640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96d00a2890 0x7f96d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.311+0000 7f96e259c640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96d00a2890 0x7f96d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.311+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f96dc0cf4c0 con 0x7f96e41022f0 2024-03-19T00:38:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.312+0000 7f96e259c640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96d00a2890 0x7f96d00a4d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f96e4101790 tx=0x7f96d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:26.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.316+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f96dc09c0a0 con 0x7f96e41022f0 2024-03-19T00:38:26.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:25 smithi044 ceph-mon[66366]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-19T00:38:26.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:25 smithi044 ceph-mon[66366]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:38:26.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:25 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/308579699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:25 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2055649624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:25 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3653955059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:25 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/31227474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:26.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.499+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96e402d1f0 con 0x7f96e41022f0 2024-03-19T00:38:26.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.500+0000 7f96dbfff640 1 -- 172.21.15.7:0/1919976854 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f96dc09b9a0 con 0x7f96e41022f0 2024-03-19T00:38:26.502 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:38:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96d00a2890 msgr2=0x7f96d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96d00a2890 0x7f96d00a4d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f96e4101790 tx=0x7f96d4005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:26.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 msgr2=0x7f96e419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 0x7f96e419ee10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00eb10 tx=0x7f96dc00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 shutdown_connections 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96d00a2890 0x7f96d00a4d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96e41022f0 0x7f96e419ee10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e4101930 0x7f96e419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 --2- 172.21.15.7:0/1919976854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96e4100740 0x7f96e406aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 >> 172.21.15.7:0/1919976854 conn(0x7f96e40fbcb0 msgr2=0x7f96e4104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 shutdown_connections 2024-03-19T00:38:26.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.502+0000 7f96e8f87640 1 -- 172.21.15.7:0/1919976854 wait complete. 2024-03-19T00:38:26.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- 172.21.15.7:0/3705751692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 msgr2=0x7f6060100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- 172.21.15.7:0/3705751692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060100b60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6050009900 tx=0x7f605002bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- 172.21.15.7:0/3705751692 shutdown_connections 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- 172.21.15.7:0/3705751692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60601022f0 0x7f6060108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- 172.21.15.7:0/3705751692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6060101930 0x7f6060101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- 172.21.15.7:0/3705751692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060100b60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- 172.21.15.7:0/3705751692 >> 172.21.15.7:0/3705751692 conn(0x7f60600fbcd0 msgr2=0x7f60600fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- 172.21.15.7:0/3705751692 shutdown_connections 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- 172.21.15.7:0/3705751692 wait complete. 2024-03-19T00:38:26.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 Processor -- start 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- start start 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6060101930 0x7f6060197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60601022f0 0x7f606019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d390 con 0x7f6060100740 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d210 con 0x7f6060101930 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.620+0000 7f60655a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d510 con 0x7f60601022f0 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42730/0 (socket says 172.21.15.7:42730) 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 -- 172.21.15.7:0/3548494074 learned_addr learned my addr 172.21.15.7:0/3548494074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605f7fe640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60601022f0 0x7f606019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f6057fff640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6060101930 0x7f6060197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 -- 172.21.15.7:0/3548494074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60601022f0 msgr2=0x7f606019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60601022f0 0x7f606019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 -- 172.21.15.7:0/3548494074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6060101930 msgr2=0x7f6060197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6060101930 0x7f6060197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 -- 172.21.15.7:0/3548494074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606019f6c0 con 0x7f6060100740 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605effd640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060197aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050043020 con 0x7f6060100740 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606019f950 con 0x7f6060100740 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6050004350 con 0x7f6060100740 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050003b80 con 0x7f6060100740 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.621+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606019fef0 con 0x7f6060100740 2024-03-19T00:38:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.622+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6060068af0 con 0x7f6060100740 2024-03-19T00:38:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.623+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f6050046020 con 0x7f6060100740 2024-03-19T00:38:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.624+0000 7f605cff9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f604c0a2890 0x7f604c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.624+0000 7f6057fff640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f604c0a2890 0x7f604c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.624+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f60500f1320 con 0x7f6060100740 2024-03-19T00:38:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.624+0000 7f6057fff640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f604c0a2890 0x7f604c0a4d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f60480095f0 tx=0x7f6048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.627+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f605004d020 con 0x7f6060100740 2024-03-19T00:38:26.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.811+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f606002d1f0 con 0x7f6060100740 2024-03-19T00:38:26.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.811+0000 7f605cff9640 1 -- 172.21.15.7:0/3548494074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f60500be010 con 0x7f6060100740 2024-03-19T00:38:26.813 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f604c0a2890 msgr2=0x7f604c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f604c0a2890 0x7f604c0a4d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f60480095f0 tx=0x7f6048005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 msgr2=0x7f6060197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060197aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050005dc0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 shutdown_connections 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f604c0a2890 0x7f604c0a4d50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60601022f0 0x7f606019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6060101930 0x7f6060197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 --2- 172.21.15.7:0/3548494074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6060100740 0x7f6060197aa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 >> 172.21.15.7:0/3548494074 conn(0x7f60600fbcd0 msgr2=0x7f6060104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 shutdown_connections 2024-03-19T00:38:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.813+0000 7f60655a9640 1 -- 172.21.15.7:0/3548494074 wait complete. 2024-03-19T00:38:26.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.930+0000 7fd49659d640 1 -- 172.21.15.7:0/165926149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd490101930 msgr2=0x7fd490101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.930+0000 7fd49659d640 1 --2- 172.21.15.7:0/165926149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd490101930 0x7fd490101db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd484009900 tx=0x7fd48402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 -- 172.21.15.7:0/165926149 shutdown_connections 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 --2- 172.21.15.7:0/165926149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4901022f0 0x7fd490108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 --2- 172.21.15.7:0/165926149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd490101930 0x7fd490101db0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 --2- 172.21.15.7:0/165926149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd490100740 0x7fd490100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 -- 172.21.15.7:0/165926149 >> 172.21.15.7:0/165926149 conn(0x7fd4900fbcb0 msgr2=0x7fd4900fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 -- 172.21.15.7:0/165926149 shutdown_connections 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 -- 172.21.15.7:0/165926149 wait complete. 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 Processor -- start 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 -- start start 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd490100740 0x7fd490104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.931+0000 7fd49659d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd490101930 0x7fd490102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd49659d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 0x7fd490103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd49659d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd49010c270 con 0x7fd4901022f0 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd49659d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd49010c0f0 con 0x7fd490101930 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd49659d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd49010c3f0 con 0x7fd490100740 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 0x7fd490103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 0x7fd490103250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42760/0 (socket says 172.21.15.7:42760) 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 -- 172.21.15.7:0/1641784428 learned_addr learned my addr 172.21.15.7:0/1641784428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd48ffff640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd490100740 0x7fd490104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd48f7fe640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd490101930 0x7fd490102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 -- 172.21.15.7:0/1641784428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd490100740 msgr2=0x7fd490104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd490100740 0x7fd490104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 -- 172.21.15.7:0/1641784428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd490101930 msgr2=0x7fd490102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd490101930 0x7fd490102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 -- 172.21.15.7:0/1641784428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd490103a50 con 0x7fd4901022f0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd494b3a640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 0x7fd490103250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd48800eb10 tx=0x7fd48800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488004b20 con 0x7fd4901022f0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd488004cc0 con 0x7fd4901022f0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4901a1be0 con 0x7fd4901022f0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488010690 con 0x7fd4901022f0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.932+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4901a2150 con 0x7fd4901022f0 2024-03-19T00:38:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.933+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4901a2380 con 0x7fd4901022f0 2024-03-19T00:38:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.934+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fd488005020 con 0x7fd4901022f0 2024-03-19T00:38:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.935+0000 7fd48d7fa640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd4780a2910 0x7fd4780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.935+0000 7fd48ffff640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd4780a2910 0x7fd4780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.935+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd4880cf5e0 con 0x7fd4901022f0 2024-03-19T00:38:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.935+0000 7fd48ffff640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd4780a2910 0x7fd4780a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd490101790 tx=0x7fd480005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:26.940+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd48809c160 con 0x7fd4901022f0 2024-03-19T00:38:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.124+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd4901a2710 con 0x7fd4901022f0 2024-03-19T00:38:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.124+0000 7fd48d7fa640 1 -- 172.21.15.7:0/1641784428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fd48809ba60 con 0x7fd4901022f0 2024-03-19T00:38:27.126 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.126+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd4780a2910 msgr2=0x7fd4780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.126+0000 7fd49659d640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd4780a2910 0x7fd4780a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd490101790 tx=0x7fd480005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 msgr2=0x7fd490103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 0x7fd490103250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd48800eb10 tx=0x7fd48800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 shutdown_connections 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd4780a2910 0x7fd4780a4dd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4901022f0 0x7fd490103250 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd490101930 0x7fd490102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 --2- 172.21.15.7:0/1641784428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd490100740 0x7fd490104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 >> 172.21.15.7:0/1641784428 conn(0x7fd4900fbcb0 msgr2=0x7fd490068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 shutdown_connections 2024-03-19T00:38:27.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.127+0000 7fd49659d640 1 -- 172.21.15.7:0/1641784428 wait complete. 2024-03-19T00:38:27.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:26 smithi007 ceph-mon[62854]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-19T00:38:27.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3757839174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:27.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1919976854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:27.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:26 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3548494074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:27.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:38:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 -- 172.21.15.7:0/4155427942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce801022f0 msgr2=0x7fce80108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 --2- 172.21.15.7:0/4155427942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce801022f0 0x7fce80108b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fce78009ca0 tx=0x7fce7802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 -- 172.21.15.7:0/4155427942 shutdown_connections 2024-03-19T00:38:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 --2- 172.21.15.7:0/4155427942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce801022f0 0x7fce80108b80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 --2- 172.21.15.7:0/4155427942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80101930 0x7fce80101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 --2- 172.21.15.7:0/4155427942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce80100740 0x7fce80100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 -- 172.21.15.7:0/4155427942 >> 172.21.15.7:0/4155427942 conn(0x7fce800fbcb0 msgr2=0x7fce800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 -- 172.21.15.7:0/4155427942 shutdown_connections 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.243+0000 7fce87a81640 1 -- 172.21.15.7:0/4155427942 wait complete. 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 Processor -- start 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 -- start start 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80100740 0x7fce80069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 0x7fce8019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce801022f0 0x7fce8019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c270 con 0x7fce80101930 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c0f0 con 0x7fce801022f0 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce87a81640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce8010c3f0 con 0x7fce80100740 2024-03-19T00:38:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 0x7fce8019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 0x7fce8019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42776/0 (socket says 172.21.15.7:42776) 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 -- 172.21.15.7:0/3616183352 learned_addr learned my addr 172.21.15.7:0/3616183352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8581d640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80100740 0x7fce80069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8601e640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce801022f0 0x7fce8019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 -- 172.21.15.7:0/3616183352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80100740 msgr2=0x7fce80069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80100740 0x7fce80069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 -- 172.21.15.7:0/3616183352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce801022f0 msgr2=0x7fce8019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce801022f0 0x7fce8019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8501c640 1 -- 172.21.15.7:0/3616183352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8019f670 con 0x7fce80101930 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8581d640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80100740 0x7fce80069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.244+0000 7fce8601e640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce801022f0 0x7fce8019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.245+0000 7fce8501c640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 0x7fce8019c770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e9f0 tx=0x7fce7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.245+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7000cd80 con 0x7fce80101930 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.245+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce700054d0 con 0x7fce80101930 2024-03-19T00:38:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.245+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8019f960 con 0x7fce80101930 2024-03-19T00:38:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.245+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70010490 con 0x7fce80101930 2024-03-19T00:38:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.245+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8019fe60 con 0x7fce80101930 2024-03-19T00:38:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.246+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80107ab0 con 0x7fce80101930 2024-03-19T00:38:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.247+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fce70005000 con 0x7fce80101930 2024-03-19T00:38:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.247+0000 7fce76ffd640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce6c0a2890 0x7fce6c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.247+0000 7fce8581d640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce6c0a2890 0x7fce6c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.247+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fce700cf510 con 0x7fce80101930 2024-03-19T00:38:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.248+0000 7fce8581d640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce6c0a2890 0x7fce6c0a4d50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce80101790 tx=0x7fce7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.251+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fce70053740 con 0x7fce80101930 2024-03-19T00:38:27.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:26 smithi204 ceph-mon[66572]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-19T00:38:27.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3757839174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:27.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1919976854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:27.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:26 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3548494074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:27.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:26 smithi044 ceph-mon[66366]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-19T00:38:27.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3757839174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:27.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1919976854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:27.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:26 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3548494074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.434+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fce8002d380 con 0x7fce80101930 2024-03-19T00:38:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.435+0000 7fce76ffd640 1 -- 172.21.15.7:0/3616183352 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fce700052f0 con 0x7fce80101930 2024-03-19T00:38:27.437 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce6c0a2890 msgr2=0x7fce6c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce6c0a2890 0x7fce6c0a4d50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce80101790 tx=0x7fce7c005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 msgr2=0x7fce8019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 0x7fce8019c770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e9f0 tx=0x7fce7000edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 shutdown_connections 2024-03-19T00:38:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce6c0a2890 0x7fce6c0a4d50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce801022f0 0x7fce8019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce80101930 0x7fce8019c770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 --2- 172.21.15.7:0/3616183352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce80100740 0x7fce80069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 >> 172.21.15.7:0/3616183352 conn(0x7fce800fbcb0 msgr2=0x7fce80104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 shutdown_connections 2024-03-19T00:38:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.437+0000 7fce87a81640 1 -- 172.21.15.7:0/3616183352 wait complete. 2024-03-19T00:38:27.450 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-19T00:38:27.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:38:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 -- 172.21.15.7:0/669624937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64701012b0 msgr2=0x7f6470101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 --2- 172.21.15.7:0/669624937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64701012b0 0x7f6470101b30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6468009be0 tx=0x7f646802bef0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 -- 172.21.15.7:0/669624937 shutdown_connections 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 --2- 172.21.15.7:0/669624937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64701012b0 0x7f6470101b30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 --2- 172.21.15.7:0/669624937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f6470100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 --2- 172.21.15.7:0/669624937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64700ffb90 0x7f64700fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 -- 172.21.15.7:0/669624937 >> 172.21.15.7:0/669624937 conn(0x7f64700fb4a0 msgr2=0x7f64700fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 -- 172.21.15.7:0/669624937 shutdown_connections 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.550+0000 7f6478177640 1 -- 172.21.15.7:0/669624937 wait complete. 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 Processor -- start 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 -- start start 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64700ffb90 0x7f6470199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f647019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64701012b0 0x7f64701a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f647010f590 con 0x7f64701008f0 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f647010f410 con 0x7f64701012b0 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6478177640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f647010f710 con 0x7f64700ffb90 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f647019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f647019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42796/0 (socket says 172.21.15.7:42796) 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 -- 172.21.15.7:0/1488720099 learned_addr learned my addr 172.21.15.7:0/1488720099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6476714640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64701012b0 0x7f64701a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475f13640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64700ffb90 0x7f6470199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 -- 172.21.15.7:0/1488720099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64700ffb90 msgr2=0x7f6470199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64700ffb90 0x7f6470199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 -- 172.21.15.7:0/1488720099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64701012b0 msgr2=0x7f64701a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64701012b0 0x7f64701a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.551+0000 7f6475712640 1 -- 172.21.15.7:0/1488720099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64701a1840 con 0x7f64701008f0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.552+0000 7f6475712640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f647019a170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f646000e9a0 tx=0x7f646000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.552+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646000cd80 con 0x7f64701008f0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.552+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64701a1b30 con 0x7f64701008f0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.552+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64600108e0 con 0x7f64701008f0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.552+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600188a0 con 0x7f64701008f0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.552+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64701a2080 con 0x7f64701008f0 2024-03-19T00:38:27.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.553+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647010b850 con 0x7f64701008f0 2024-03-19T00:38:27.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.554+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f6460010a90 con 0x7f64701008f0 2024-03-19T00:38:27.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.554+0000 7f6466ffd640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f645c0a2890 0x7f645c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.554+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f64600cf9e0 con 0x7f64701008f0 2024-03-19T00:38:27.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.554+0000 7f6475f13640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f645c0a2890 0x7f645c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.555+0000 7f6475f13640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f645c0a2890 0x7f645c0a4d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f647019b100 tx=0x7f646c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:27.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.558+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6460026070 con 0x7f64701008f0 2024-03-19T00:38:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.782+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f647002d460 con 0x7f64701008f0 2024-03-19T00:38:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.782+0000 7f6466ffd640 1 -- 172.21.15.7:0/1488720099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7f646009c5c0 con 0x7f64701008f0 2024-03-19T00:38:27.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:27.785 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:38:23.871158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:38:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-19T00:38:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f645c0a2890 msgr2=0x7f645c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f645c0a2890 0x7f645c0a4d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f647019b100 tx=0x7f646c005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 msgr2=0x7f647019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f647019a170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f646000e9a0 tx=0x7f646000ee60 comp rx=0 tx=0).stop 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 shutdown_connections 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f645c0a2890 0x7f645c0a4d50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64701012b0 0x7f64701a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64701008f0 0x7f647019a170 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 --2- 172.21.15.7:0/1488720099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64700ffb90 0x7f6470199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 >> 172.21.15.7:0/1488720099 conn(0x7f64700fb4a0 msgr2=0x7f64700fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 shutdown_connections 2024-03-19T00:38:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.784+0000 7f6478177640 1 -- 172.21.15.7:0/1488720099 wait complete. 2024-03-19T00:38:27.797 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-19T00:38:27.797 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-19T00:38:27.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 -- 172.21.15.7:0/3141657389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5181007c0 msgr2=0x7fc518100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 --2- 172.21.15.7:0/3141657389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5181007c0 0x7fc518100be0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009900 tx=0x7fc50c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 -- 172.21.15.7:0/3141657389 shutdown_connections 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 --2- 172.21.15.7:0/3141657389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc518102370 0x7fc518108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 --2- 172.21.15.7:0/3141657389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5181019b0 0x7fc518101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 --2- 172.21.15.7:0/3141657389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5181007c0 0x7fc518100be0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 -- 172.21.15.7:0/3141657389 >> 172.21.15.7:0/3141657389 conn(0x7fc5180fbd50 msgr2=0x7fc5180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 -- 172.21.15.7:0/3141657389 shutdown_connections 2024-03-19T00:38:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 -- 172.21.15.7:0/3141657389 wait complete. 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 Processor -- start 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.899+0000 7fc51e31b640 1 -- start start 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5181007c0 0x7fc518197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5181019b0 0x7fc518197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 0x7fc51819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc51810d3a0 con 0x7fc518102370 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc51810d220 con 0x7fc5181019b0 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51810d520 con 0x7fc5181007c0 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 0x7fc51819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 0x7fc51819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42822/0 (socket says 172.21.15.7:42822) 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 -- 172.21.15.7:0/1185554173 learned_addr learned my addr 172.21.15.7:0/1185554173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc517fff640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5181007c0 0x7fc518197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc5177fe640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5181019b0 0x7fc518197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 -- 172.21.15.7:0/1185554173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5181007c0 msgr2=0x7fc518197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5181007c0 0x7fc518197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 -- 172.21.15.7:0/1185554173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5181019b0 msgr2=0x7fc518197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5181019b0 0x7fc518197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 -- 172.21.15.7:0/1185554173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51819f6a0 con 0x7fc518102370 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51c8b8640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 0x7fc51819f030 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc51000eb10 tx=0x7fc51000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510004b20 con 0x7fc518102370 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51819f990 con 0x7fc518102370 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.900+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc510004cc0 con 0x7fc518102370 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.901+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510010690 con 0x7fc518102370 2024-03-19T00:38:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.901+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51819fee0 con 0x7fc518102370 2024-03-19T00:38:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.901+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc518107bb0 con 0x7fc518102370 2024-03-19T00:38:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.902+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fc510005020 con 0x7fc518102370 2024-03-19T00:38:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.903+0000 7fc5157fa640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5000a2890 0x7fc5000a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.903+0000 7fc517fff640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5000a2890 0x7fc5000a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.903+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc5100cf610 con 0x7fc518102370 2024-03-19T00:38:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.903+0000 7fc517fff640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5000a2890 0x7fc5000a4d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005ab0 tx=0x7fc50c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:27.907+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc51009c1f0 con 0x7fc518102370 2024-03-19T00:38:28.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.089+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc518061c60 con 0x7fc5000a2890 2024-03-19T00:38:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.099+0000 7fc5157fa640 1 -- 172.21.15.7:0/1185554173 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340481 (secure 0 0 0) 0x7fc518061c60 con 0x7fc5000a2890 2024-03-19T00:38:28.105 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:28.107 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5000a2890 msgr2=0x7fc5000a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5000a2890 0x7fc5000a4d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005ab0 tx=0x7fc50c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 msgr2=0x7fc51819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 0x7fc51819f030 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc51000eb10 tx=0x7fc51000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 shutdown_connections 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5000a2890 0x7fc5000a4d50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc518102370 0x7fc51819f030 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5181019b0 0x7fc518197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 --2- 172.21.15.7:0/1185554173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5181007c0 0x7fc518197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 >> 172.21.15.7:0/1185554173 conn(0x7fc5180fbd50 msgr2=0x7fc518104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 shutdown_connections 2024-03-19T00:38:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.103+0000 7fc51e31b640 1 -- 172.21.15.7:0/1185554173 wait complete. 2024-03-19T00:38:28.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:27 smithi007 ceph-mon[62854]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:38:28.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:27 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1641784428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:28.115 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:27 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3616183352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:28.115 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:27 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1488720099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:28.116 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-03-19T00:38:28.038034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660364,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":867,"num_read_kb":555,"num_write":366,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404852,"kb_used_data":14100,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536972,"statfs":{"total":1151940427776,"available":1151525859328,"internally_reserved":0,"allocated":14438400,"data_stored":7970807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1482,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":32,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029289+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:37:56.029289+0000","last_peered":"2024-03-19T00:37:56.029289+0000","last_clean":"2024-03-19T00:37:56.029289+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:37:56.029289+0000","last_undegraded":"2024-03-19T00:37:56.029289+0000","last_fullsized":"2024-03-19T00:37:56.029289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039878+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:37:56.039878+0000","last_peered":"2024-03-19T00:37:56.039878+0000","last_clean":"2024-03-19T00:37:56.039878+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:37:56.039878+0000","last_undegraded":"2024-03-19T00:37:56.039878+0000","last_fullsized":"2024-03-19T00:37:56.039878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042753+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:37:56.042753+0000","last_peered":"2024-03-19T00:37:56.042753+0000","last_clean":"2024-03-19T00:37:56.042753+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:37:56.042753+0000","last_undegraded":"2024-03-19T00:37:56.042753+0000","last_fullsized":"2024-03-19T00:37:56.042753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.019845+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:37:56.019845+0000","last_peered":"2024-03-19T00:37:56.019845+0000","last_clean":"2024-03-19T00:37:56.019845+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:37:56.019845+0000","last_undegraded":"2024-03-19T00:37:56.019845+0000","last_fullsized":"2024-03-19T00:37:56.019845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.017353+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:37:56.017353+0000","last_peered":"2024-03-19T00:37:56.017353+0000","last_clean":"2024-03-19T00:37:56.017353+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:37:56.017353+0000","last_undegraded":"2024-03-19T00:37:56.017353+0000","last_fullsized":"2024-03-19T00:37:56.017353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029411+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:37:56.029411+0000","last_peered":"2024-03-19T00:37:56.029411+0000","last_clean":"2024-03-19T00:37:56.029411+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:37:56.029411+0000","last_undegraded":"2024-03-19T00:37:56.029411+0000","last_fullsized":"2024-03-19T00:37:56.029411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.022473+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:37:56.022473+0000","last_peered":"2024-03-19T00:37:56.022473+0000","last_clean":"2024-03-19T00:37:56.022473+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:37:56.022473+0000","last_undegraded":"2024-03-19T00:37:56.022473+0000","last_fullsized":"2024-03-19T00:37:56.022473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.034359+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:37:56.034359+0000","last_peered":"2024-03-19T00:37:56.034359+0000","last_clean":"2024-03-19T00:37:56.034359+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:37:56.034359+0000","last_undegraded":"2024-03-19T00:37:56.034359+0000","last_fullsized":"2024-03-19T00:37:56.034359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041752+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:37:56.041752+0000","last_peered":"2024-03-19T00:37:56.041752+0000","last_clean":"2024-03-19T00:37:56.041752+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:37:56.041752+0000","last_undegraded":"2024-03-19T00:37:56.041752+0000","last_fullsized":"2024-03-19T00:37:56.041752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.035497+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:37:56.035497+0000","last_peered":"2024-03-19T00:37:56.035497+0000","last_clean":"2024-03-19T00:37:56.035497+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:37:56.035497+0000","last_undegraded":"2024-03-19T00:37:56.035497+0000","last_fullsized":"2024-03-19T00:37:56.035497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040785+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:37:56.040785+0000","last_peered":"2024-03-19T00:37:56.040785+0000","last_clean":"2024-03-19T00:37:56.040785+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:37:56.040785+0000","last_undegraded":"2024-03-19T00:37:56.040785+0000","last_fullsized":"2024-03-19T00:37:56.040785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031924+0000","last_change":"2024-03-19T00:36:58.297533+0000","last_active":"2024-03-19T00:37:56.031924+0000","last_peered":"2024-03-19T00:37:56.031924+0000","last_clean":"2024-03-19T00:37:56.031924+0000","last_became_active":"2024-03-19T00:36:58.297218+0000","last_became_peered":"2024-03-19T00:36:58.297218+0000","last_unstale":"2024-03-19T00:37:56.031924+0000","last_undegraded":"2024-03-19T00:37:56.031924+0000","last_fullsized":"2024-03-19T00:37:56.031924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:06.256765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039054+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:37:56.039054+0000","last_peered":"2024-03-19T00:37:56.039054+0000","last_clean":"2024-03-19T00:37:56.039054+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:37:56.039054+0000","last_undegraded":"2024-03-19T00:37:56.039054+0000","last_fullsized":"2024-03-19T00:37:56.039054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039900+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:37:56.039900+0000","last_peered":"2024-03-19T00:37:56.039900+0000","last_clean":"2024-03-19T00:37:56.039900+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:37:56.039900+0000","last_undegraded":"2024-03-19T00:37:56.039900+0000","last_fullsized":"2024-03-19T00:37:56.039900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042705+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:37:56.042705+0000","last_peered":"2024-03-19T00:37:56.042705+0000","last_clean":"2024-03-19T00:37:56.042705+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:37:56.042705+0000","last_undegraded":"2024-03-19T00:37:56.042705+0000","last_fullsized":"2024-03-19T00:37:56.042705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042051+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:37:56.042051+0000","last_peered":"2024-03-19T00:37:56.042051+0000","last_clean":"2024-03-19T00:37:56.042051+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:37:56.042051+0000","last_undegraded":"2024-03-19T00:37:56.042051+0000","last_fullsized":"2024-03-19T00:37:56.042051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029319+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:37:56.029319+0000","last_peered":"2024-03-19T00:37:56.029319+0000","last_clean":"2024-03-19T00:37:56.029319+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:37:56.029319+0000","last_undegraded":"2024-03-19T00:37:56.029319+0000","last_fullsized":"2024-03-19T00:37:56.029319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.032743+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:37:56.032743+0000","last_peered":"2024-03-19T00:37:56.032743+0000","last_clean":"2024-03-19T00:37:56.032743+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:37:56.032743+0000","last_undegraded":"2024-03-19T00:37:56.032743+0000","last_fullsized":"2024-03-19T00:37:56.032743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.030215+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:37:56.030215+0000","last_peered":"2024-03-19T00:37:56.030215+0000","last_clean":"2024-03-19T00:37:56.030215+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:37:56.030215+0000","last_undegraded":"2024-03-19T00:37:56.030215+0000","last_fullsized":"2024-03-19T00:37:56.030215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944493+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:57.944493+0000","last_peered":"2024-03-19T00:36:57.944493+0000","last_clean":"2024-03-19T00:36:57.944493+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:57.944493+0000","last_undegraded":"2024-03-19T00:36:57.944493+0000","last_fullsized":"2024-03-19T00:36:57.944493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943877+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:57.943877+0000","last_peered":"2024-03-19T00:36:57.943877+0000","last_clean":"2024-03-19T00:36:57.943877+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:57.943877+0000","last_undegraded":"2024-03-19T00:36:57.943877+0000","last_fullsized":"2024-03-19T00:36:57.943877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028440+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:37:56.028440+0000","last_peered":"2024-03-19T00:37:56.028440+0000","last_clean":"2024-03-19T00:37:56.028440+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:37:56.028440+0000","last_undegraded":"2024-03-19T00:37:56.028440+0000","last_fullsized":"2024-03-19T00:37:56.028440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943659+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:57.943659+0000","last_peered":"2024-03-19T00:36:57.943659+0000","last_clean":"2024-03-19T00:36:57.943659+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:57.943659+0000","last_undegraded":"2024-03-19T00:36:57.943659+0000","last_fullsized":"2024-03-19T00:36:57.943659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947548+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:57.947548+0000","last_peered":"2024-03-19T00:36:57.947548+0000","last_clean":"2024-03-19T00:36:57.947548+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:57.947548+0000","last_undegraded":"2024-03-19T00:36:57.947548+0000","last_fullsized":"2024-03-19T00:36:57.947548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044072+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:37:56.044072+0000","last_peered":"2024-03-19T00:37:56.044072+0000","last_clean":"2024-03-19T00:37:56.044072+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:37:56.044072+0000","last_undegraded":"2024-03-19T00:37:56.044072+0000","last_fullsized":"2024-03-19T00:37:56.044072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943869+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:57.943869+0000","last_peered":"2024-03-19T00:36:57.943869+0000","last_clean":"2024-03-19T00:36:57.943869+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:57.943869+0000","last_undegraded":"2024-03-19T00:36:57.943869+0000","last_fullsized":"2024-03-19T00:36:57.943869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947966+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:57.947966+0000","last_peered":"2024-03-19T00:36:57.947966+0000","last_clean":"2024-03-19T00:36:57.947966+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:57.947966+0000","last_undegraded":"2024-03-19T00:36:57.947966+0000","last_fullsized":"2024-03-19T00:36:57.947966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044222+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:37:56.044222+0000","last_peered":"2024-03-19T00:37:56.044222+0000","last_clean":"2024-03-19T00:37:56.044222+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:37:56.044222+0000","last_undegraded":"2024-03-19T00:37:56.044222+0000","last_fullsized":"2024-03-19T00:37:56.044222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943744+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:57.943744+0000","last_peered":"2024-03-19T00:36:57.943744+0000","last_clean":"2024-03-19T00:36:57.943744+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:57.943744+0000","last_undegraded":"2024-03-19T00:36:57.943744+0000","last_fullsized":"2024-03-19T00:36:57.943744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948143+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:57.948143+0000","last_peered":"2024-03-19T00:36:57.948143+0000","last_clean":"2024-03-19T00:36:57.948143+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:57.948143+0000","last_undegraded":"2024-03-19T00:36:57.948143+0000","last_fullsized":"2024-03-19T00:36:57.948143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042138+0000","last_change":"2024-03-19T00:36:57.948938+0000","last_active":"2024-03-19T00:37:56.042138+0000","last_peered":"2024-03-19T00:37:56.042138+0000","last_clean":"2024-03-19T00:37:56.042138+0000","last_became_active":"2024-03-19T00:36:57.948519+0000","last_became_peered":"2024-03-19T00:36:57.948519+0000","last_unstale":"2024-03-19T00:37:56.042138+0000","last_undegraded":"2024-03-19T00:37:56.042138+0000","last_fullsized":"2024-03-19T00:37:56.042138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:33:49.099555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.298764+0000","last_change":"2024-03-19T00:36:58.298764+0000","last_active":"2024-03-19T00:36:58.298764+0000","last_peered":"2024-03-19T00:36:58.298764+0000","last_clean":"2024-03-19T00:36:58.298764+0000","last_became_active":"2024-03-19T00:36:58.298265+0000","last_became_peered":"2024-03-19T00:36:58.298265+0000","last_unstale":"2024-03-19T00:36:58.298764+0000","last_undegraded":"2024-03-19T00:36:58.298764+0000","last_fullsized":"2024-03-19T00:36:58.298764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:45.983861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:22.428477+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:38:22.428477+0000","last_peered":"2024-03-19T00:38:22.428477+0000","last_clean":"2024-03-19T00:38:22.428477+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:38:22.428477+0000","last_undegraded":"2024-03-19T00:38:22.428477+0000","last_fullsized":"2024-03-19T00:38:22.428477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1648,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.014746+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:37:56.014746+0000","last_peered":"2024-03-19T00:37:56.014746+0000","last_clean":"2024-03-19T00:37:56.014746+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:37:56.014746+0000","last_undegraded":"2024-03-19T00:37:56.014746+0000","last_fullsized":"2024-03-19T00:37:56.014746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342937+0000","last_change":"2024-03-19T00:36:58.342937+0000","last_active":"2024-03-19T00:36:58.342937+0000","last_peered":"2024-03-19T00:36:58.342937+0000","last_clean":"2024-03-19T00:36:58.342937+0000","last_became_active":"2024-03-19T00:36:58.342474+0000","last_became_peered":"2024-03-19T00:36:58.342474+0000","last_unstale":"2024-03-19T00:36:58.342937+0000","last_undegraded":"2024-03-19T00:36:58.342937+0000","last_fullsized":"2024-03-19T00:36:58.342937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:55.531546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297043+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:58.297043+0000","last_peered":"2024-03-19T00:36:58.297043+0000","last_clean":"2024-03-19T00:36:58.297043+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:58.297043+0000","last_undegraded":"2024-03-19T00:36:58.297043+0000","last_fullsized":"2024-03-19T00:36:58.297043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.027675+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:37:56.027675+0000","last_peered":"2024-03-19T00:37:56.027675+0000","last_clean":"2024-03-19T00:37:56.027675+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:37:56.027675+0000","last_undegraded":"2024-03-19T00:37:56.027675+0000","last_fullsized":"2024-03-19T00:37:56.027675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.951395+0000","last_change":"2024-03-19T00:36:57.950687+0000","last_active":"2024-03-19T00:36:57.951395+0000","last_peered":"2024-03-19T00:36:57.951395+0000","last_clean":"2024-03-19T00:36:57.951395+0000","last_became_active":"2024-03-19T00:36:57.950151+0000","last_became_peered":"2024-03-19T00:36:57.950151+0000","last_unstale":"2024-03-19T00:36:57.951395+0000","last_undegraded":"2024-03-19T00:36:57.951395+0000","last_fullsized":"2024-03-19T00:36:57.951395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:41.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018277+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:58.018277+0000","last_peered":"2024-03-19T00:36:58.018277+0000","last_clean":"2024-03-19T00:36:58.018277+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:58.018277+0000","last_undegraded":"2024-03-19T00:36:58.018277+0000","last_fullsized":"2024-03-19T00:36:58.018277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033347+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:37:56.033347+0000","last_peered":"2024-03-19T00:37:56.033347+0000","last_clean":"2024-03-19T00:37:56.033347+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:37:56.033347+0000","last_undegraded":"2024-03-19T00:37:56.033347+0000","last_fullsized":"2024-03-19T00:37:56.033347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947452+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:57.947452+0000","last_peered":"2024-03-19T00:36:57.947452+0000","last_clean":"2024-03-19T00:36:57.947452+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:57.947452+0000","last_undegraded":"2024-03-19T00:36:57.947452+0000","last_fullsized":"2024-03-19T00:36:57.947452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341398+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:58.341398+0000","last_peered":"2024-03-19T00:36:58.341398+0000","last_clean":"2024-03-19T00:36:58.341398+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:58.341398+0000","last_undegraded":"2024-03-19T00:36:58.341398+0000","last_fullsized":"2024-03-19T00:36:58.341398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033250+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:37:56.033250+0000","last_peered":"2024-03-19T00:37:56.033250+0000","last_clean":"2024-03-19T00:37:56.033250+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:37:56.033250+0000","last_undegraded":"2024-03-19T00:37:56.033250+0000","last_fullsized":"2024-03-19T00:37:56.033250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946524+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:57.946524+0000","last_peered":"2024-03-19T00:36:57.946524+0000","last_clean":"2024-03-19T00:36:57.946524+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:57.946524+0000","last_undegraded":"2024-03-19T00:36:57.946524+0000","last_fullsized":"2024-03-19T00:36:57.946524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.429184+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:38:23.429184+0000","last_peered":"2024-03-19T00:38:23.429184+0000","last_clean":"2024-03-19T00:38:23.429184+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:38:23.429184+0000","last_undegraded":"2024-03-19T00:38:23.429184+0000","last_fullsized":"2024-03-19T00:38:23.429184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044130+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:37:56.044130+0000","last_peered":"2024-03-19T00:37:56.044130+0000","last_clean":"2024-03-19T00:37:56.044130+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:37:56.044130+0000","last_undegraded":"2024-03-19T00:37:56.044130+0000","last_fullsized":"2024-03-19T00:37:56.044130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943224+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:57.943224+0000","last_peered":"2024-03-19T00:36:57.943224+0000","last_clean":"2024-03-19T00:36:57.943224+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:57.943224+0000","last_undegraded":"2024-03-19T00:36:57.943224+0000","last_fullsized":"2024-03-19T00:36:57.943224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092881+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:58.092881+0000","last_peered":"2024-03-19T00:36:58.092881+0000","last_clean":"2024-03-19T00:36:58.092881+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:58.092881+0000","last_undegraded":"2024-03-19T00:36:58.092881+0000","last_fullsized":"2024-03-19T00:36:58.092881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.037121+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:37:56.037121+0000","last_peered":"2024-03-19T00:37:56.037121+0000","last_clean":"2024-03-19T00:37:56.037121+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:37:56.037121+0000","last_undegraded":"2024-03-19T00:37:56.037121+0000","last_fullsized":"2024-03-19T00:37:56.037121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944132+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:57.944132+0000","last_peered":"2024-03-19T00:36:57.944132+0000","last_clean":"2024-03-19T00:36:57.944132+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:57.944132+0000","last_undegraded":"2024-03-19T00:36:57.944132+0000","last_fullsized":"2024-03-19T00:36:57.944132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341884+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:58.341884+0000","last_peered":"2024-03-19T00:36:58.341884+0000","last_clean":"2024-03-19T00:36:58.341884+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:58.341884+0000","last_undegraded":"2024-03-19T00:36:58.341884+0000","last_fullsized":"2024-03-19T00:36:58.341884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042790+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:37:56.042790+0000","last_peered":"2024-03-19T00:37:56.042790+0000","last_clean":"2024-03-19T00:37:56.042790+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:37:56.042790+0000","last_undegraded":"2024-03-19T00:37:56.042790+0000","last_fullsized":"2024-03-19T00:37:56.042790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943327+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:57.943327+0000","last_peered":"2024-03-19T00:36:57.943327+0000","last_clean":"2024-03-19T00:36:57.943327+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:57.943327+0000","last_undegraded":"2024-03-19T00:36:57.943327+0000","last_fullsized":"2024-03-19T00:36:57.943327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":312,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.429728+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:38:23.429728+0000","last_peered":"2024-03-19T00:38:23.429728+0000","last_clean":"2024-03-19T00:38:23.429728+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:38:23.429728+0000","last_undegraded":"2024-03-19T00:38:23.429728+0000","last_fullsized":"2024-03-19T00:38:23.429728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":186,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.018651+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:37:56.018651+0000","last_peered":"2024-03-19T00:37:56.018651+0000","last_clean":"2024-03-19T00:37:56.018651+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:37:56.018651+0000","last_undegraded":"2024-03-19T00:37:56.018651+0000","last_fullsized":"2024-03-19T00:37:56.018651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342057+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:58.342057+0000","last_peered":"2024-03-19T00:36:58.342057+0000","last_clean":"2024-03-19T00:36:58.342057+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:58.342057+0000","last_undegraded":"2024-03-19T00:36:58.342057+0000","last_fullsized":"2024-03-19T00:36:58.342057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945086+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:57.945086+0000","last_peered":"2024-03-19T00:36:57.945086+0000","last_clean":"2024-03-19T00:36:57.945086+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:57.945086+0000","last_undegraded":"2024-03-19T00:36:57.945086+0000","last_fullsized":"2024-03-19T00:36:57.945086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033786+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:37:56.033786+0000","last_peered":"2024-03-19T00:37:56.033786+0000","last_clean":"2024-03-19T00:37:56.033786+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:37:56.033786+0000","last_undegraded":"2024-03-19T00:37:56.033786+0000","last_fullsized":"2024-03-19T00:37:56.033786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945403+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:57.945403+0000","last_peered":"2024-03-19T00:36:57.945403+0000","last_clean":"2024-03-19T00:36:57.945403+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:57.945403+0000","last_undegraded":"2024-03-19T00:36:57.945403+0000","last_fullsized":"2024-03-19T00:36:57.945403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341889+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:58.341889+0000","last_peered":"2024-03-19T00:36:58.341889+0000","last_clean":"2024-03-19T00:36:58.341889+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:58.341889+0000","last_undegraded":"2024-03-19T00:36:58.341889+0000","last_fullsized":"2024-03-19T00:36:58.341889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.035315+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:37:56.035315+0000","last_peered":"2024-03-19T00:37:56.035315+0000","last_clean":"2024-03-19T00:37:56.035315+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:37:56.035315+0000","last_undegraded":"2024-03-19T00:37:56.035315+0000","last_fullsized":"2024-03-19T00:37:56.035315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943691+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:57.943691+0000","last_peered":"2024-03-19T00:36:57.943691+0000","last_clean":"2024-03-19T00:36:57.943691+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:57.943691+0000","last_undegraded":"2024-03-19T00:36:57.943691+0000","last_fullsized":"2024-03-19T00:36:57.943691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.445021+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:38:23.445021+0000","last_peered":"2024-03-19T00:38:23.445021+0000","last_clean":"2024-03-19T00:38:23.445021+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:38:23.445021+0000","last_undegraded":"2024-03-19T00:38:23.445021+0000","last_fullsized":"2024-03-19T00:38:23.445021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1614,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033957+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:37:56.033957+0000","last_peered":"2024-03-19T00:37:56.033957+0000","last_clean":"2024-03-19T00:37:56.033957+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:37:56.033957+0000","last_undegraded":"2024-03-19T00:37:56.033957+0000","last_fullsized":"2024-03-19T00:37:56.033957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018782+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:58.018782+0000","last_peered":"2024-03-19T00:36:58.018782+0000","last_clean":"2024-03-19T00:36:58.018782+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:58.018782+0000","last_undegraded":"2024-03-19T00:36:58.018782+0000","last_fullsized":"2024-03-19T00:36:58.018782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947702+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:57.947702+0000","last_peered":"2024-03-19T00:36:57.947702+0000","last_clean":"2024-03-19T00:36:57.947702+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:57.947702+0000","last_undegraded":"2024-03-19T00:36:57.947702+0000","last_fullsized":"2024-03-19T00:36:57.947702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026513+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:37:56.026513+0000","last_peered":"2024-03-19T00:37:56.026513+0000","last_clean":"2024-03-19T00:37:56.026513+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:37:56.026513+0000","last_undegraded":"2024-03-19T00:37:56.026513+0000","last_fullsized":"2024-03-19T00:37:56.026513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341751+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:58.341751+0000","last_peered":"2024-03-19T00:36:58.341751+0000","last_clean":"2024-03-19T00:36:58.341751+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:58.341751+0000","last_undegraded":"2024-03-19T00:36:58.341751+0000","last_fullsized":"2024-03-19T00:36:58.341751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948779+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:57.948779+0000","last_peered":"2024-03-19T00:36:57.948779+0000","last_clean":"2024-03-19T00:36:57.948779+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:57.948779+0000","last_undegraded":"2024-03-19T00:36:57.948779+0000","last_fullsized":"2024-03-19T00:36:57.948779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.024437+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:37:56.024437+0000","last_peered":"2024-03-19T00:37:56.024437+0000","last_clean":"2024-03-19T00:37:56.024437+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:37:56.024437+0000","last_undegraded":"2024-03-19T00:37:56.024437+0000","last_fullsized":"2024-03-19T00:37:56.024437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947891+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:57.947891+0000","last_peered":"2024-03-19T00:36:57.947891+0000","last_clean":"2024-03-19T00:36:57.947891+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:57.947891+0000","last_undegraded":"2024-03-19T00:36:57.947891+0000","last_fullsized":"2024-03-19T00:36:57.947891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943536+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:57.943536+0000","last_peered":"2024-03-19T00:36:57.943536+0000","last_clean":"2024-03-19T00:36:57.943536+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:57.943536+0000","last_undegraded":"2024-03-19T00:36:57.943536+0000","last_fullsized":"2024-03-19T00:36:57.943536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.037115+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:37:56.037115+0000","last_peered":"2024-03-19T00:37:56.037115+0000","last_clean":"2024-03-19T00:37:56.037115+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:37:56.037115+0000","last_undegraded":"2024-03-19T00:37:56.037115+0000","last_fullsized":"2024-03-19T00:37:56.037115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018669+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:58.018669+0000","last_peered":"2024-03-19T00:36:58.018669+0000","last_clean":"2024-03-19T00:36:58.018669+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:58.018669+0000","last_undegraded":"2024-03-19T00:36:58.018669+0000","last_fullsized":"2024-03-19T00:36:58.018669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092575+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:58.092575+0000","last_peered":"2024-03-19T00:36:58.092575+0000","last_clean":"2024-03-19T00:36:58.092575+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:58.092575+0000","last_undegraded":"2024-03-19T00:36:58.092575+0000","last_fullsized":"2024-03-19T00:36:58.092575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.034570+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:37:56.034570+0000","last_peered":"2024-03-19T00:37:56.034570+0000","last_clean":"2024-03-19T00:37:56.034570+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:37:56.034570+0000","last_undegraded":"2024-03-19T00:37:56.034570+0000","last_fullsized":"2024-03-19T00:37:56.034570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018759+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:58.018759+0000","last_peered":"2024-03-19T00:36:58.018759+0000","last_clean":"2024-03-19T00:36:58.018759+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:58.018759+0000","last_undegraded":"2024-03-19T00:36:58.018759+0000","last_fullsized":"2024-03-19T00:36:58.018759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026479+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:37:56.026479+0000","last_peered":"2024-03-19T00:37:56.026479+0000","last_clean":"2024-03-19T00:37:56.026479+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:37:56.026479+0000","last_undegraded":"2024-03-19T00:37:56.026479+0000","last_fullsized":"2024-03-19T00:37:56.026479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.444526+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:38:23.444526+0000","last_peered":"2024-03-19T00:38:23.444526+0000","last_clean":"2024-03-19T00:38:23.444526+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:38:23.444526+0000","last_undegraded":"2024-03-19T00:38:23.444526+0000","last_fullsized":"2024-03-19T00:38:23.444526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092869+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:58.092869+0000","last_peered":"2024-03-19T00:36:58.092869+0000","last_clean":"2024-03-19T00:36:58.092869+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:58.092869+0000","last_undegraded":"2024-03-19T00:36:58.092869+0000","last_fullsized":"2024-03-19T00:36:58.092869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.043291+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:37:56.043291+0000","last_peered":"2024-03-19T00:37:56.043291+0000","last_clean":"2024-03-19T00:37:56.043291+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:37:56.043291+0000","last_undegraded":"2024-03-19T00:37:56.043291+0000","last_fullsized":"2024-03-19T00:37:56.043291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944026+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:57.944026+0000","last_peered":"2024-03-19T00:36:57.944026+0000","last_clean":"2024-03-19T00:36:57.944026+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:57.944026+0000","last_undegraded":"2024-03-19T00:36:57.944026+0000","last_fullsized":"2024-03-19T00:36:57.944026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092415+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:58.092415+0000","last_peered":"2024-03-19T00:36:58.092415+0000","last_clean":"2024-03-19T00:36:58.092415+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:58.092415+0000","last_undegraded":"2024-03-19T00:36:58.092415+0000","last_fullsized":"2024-03-19T00:36:58.092415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342005+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:58.342005+0000","last_peered":"2024-03-19T00:36:58.342005+0000","last_clean":"2024-03-19T00:36:58.342005+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:58.342005+0000","last_undegraded":"2024-03-19T00:36:58.342005+0000","last_fullsized":"2024-03-19T00:36:58.342005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028043+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:37:56.028043+0000","last_peered":"2024-03-19T00:37:56.028043+0000","last_clean":"2024-03-19T00:37:56.028043+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:37:56.028043+0000","last_undegraded":"2024-03-19T00:37:56.028043+0000","last_fullsized":"2024-03-19T00:37:56.028043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092489+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:58.092489+0000","last_peered":"2024-03-19T00:36:58.092489+0000","last_clean":"2024-03-19T00:36:58.092489+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:58.092489+0000","last_undegraded":"2024-03-19T00:36:58.092489+0000","last_fullsized":"2024-03-19T00:36:58.092489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945808+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:57.945808+0000","last_peered":"2024-03-19T00:36:57.945808+0000","last_clean":"2024-03-19T00:36:57.945808+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:57.945808+0000","last_undegraded":"2024-03-19T00:36:57.945808+0000","last_fullsized":"2024-03-19T00:36:57.945808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948192+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:57.948192+0000","last_peered":"2024-03-19T00:36:57.948192+0000","last_clean":"2024-03-19T00:36:57.948192+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:57.948192+0000","last_undegraded":"2024-03-19T00:36:57.948192+0000","last_fullsized":"2024-03-19T00:36:57.948192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026012+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:37:56.026012+0000","last_peered":"2024-03-19T00:37:56.026012+0000","last_clean":"2024-03-19T00:37:56.026012+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:37:56.026012+0000","last_undegraded":"2024-03-19T00:37:56.026012+0000","last_fullsized":"2024-03-19T00:37:56.026012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946095+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:57.946095+0000","last_peered":"2024-03-19T00:36:57.946095+0000","last_clean":"2024-03-19T00:36:57.946095+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:57.946095+0000","last_undegraded":"2024-03-19T00:36:57.946095+0000","last_fullsized":"2024-03-19T00:36:57.946095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.219170+0000","last_change":"2024-03-19T00:36:57.954788+0000","last_active":"2024-03-19T00:38:23.219170+0000","last_peered":"2024-03-19T00:38:23.219170+0000","last_clean":"2024-03-19T00:38:23.219170+0000","last_became_active":"2024-03-19T00:36:57.949916+0000","last_became_peered":"2024-03-19T00:36:57.949916+0000","last_unstale":"2024-03-19T00:38:23.219170+0000","last_undegraded":"2024-03-19T00:38:23.219170+0000","last_fullsized":"2024-03-19T00:38:23.219170+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:09.959915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033677+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:37:56.033677+0000","last_peered":"2024-03-19T00:37:56.033677+0000","last_clean":"2024-03-19T00:37:56.033677+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:37:56.033677+0000","last_undegraded":"2024-03-19T00:37:56.033677+0000","last_fullsized":"2024-03-19T00:37:56.033677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944256+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:57.944256+0000","last_peered":"2024-03-19T00:36:57.944256+0000","last_clean":"2024-03-19T00:36:57.944256+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:57.944256+0000","last_undegraded":"2024-03-19T00:36:57.944256+0000","last_fullsized":"2024-03-19T00:36:57.944256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945162+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:57.945162+0000","last_peered":"2024-03-19T00:36:57.945162+0000","last_clean":"2024-03-19T00:36:57.945162+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:57.945162+0000","last_undegraded":"2024-03-19T00:36:57.945162+0000","last_fullsized":"2024-03-19T00:36:57.945162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044817+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:37:56.044817+0000","last_peered":"2024-03-19T00:37:56.044817+0000","last_clean":"2024-03-19T00:37:56.044817+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:37:56.044817+0000","last_undegraded":"2024-03-19T00:37:56.044817+0000","last_fullsized":"2024-03-19T00:37:56.044817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.343169+0000","last_change":"2024-03-19T00:36:58.342763+0000","last_active":"2024-03-19T00:36:58.343169+0000","last_peered":"2024-03-19T00:36:58.343169+0000","last_clean":"2024-03-19T00:36:58.343169+0000","last_became_active":"2024-03-19T00:36:58.342305+0000","last_became_peered":"2024-03-19T00:36:58.342305+0000","last_unstale":"2024-03-19T00:36:58.343169+0000","last_undegraded":"2024-03-19T00:36:58.343169+0000","last_fullsized":"2024-03-19T00:36:58.343169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:44.312347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.952606+0000","last_change":"2024-03-19T00:36:57.952182+0000","last_active":"2024-03-19T00:36:57.952606+0000","last_peered":"2024-03-19T00:36:57.952606+0000","last_clean":"2024-03-19T00:36:57.952606+0000","last_became_active":"2024-03-19T00:36:57.948822+0000","last_became_peered":"2024-03-19T00:36:57.948822+0000","last_unstale":"2024-03-19T00:36:57.952606+0000","last_undegraded":"2024-03-19T00:36:57.952606+0000","last_fullsized":"2024-03-19T00:36:57.952606+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.527748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026905+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:37:56.026905+0000","last_peered":"2024-03-19T00:37:56.026905+0000","last_clean":"2024-03-19T00:37:56.026905+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:37:56.026905+0000","last_undegraded":"2024-03-19T00:37:56.026905+0000","last_fullsized":"2024-03-19T00:37:56.026905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943843+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:57.943843+0000","last_peered":"2024-03-19T00:36:57.943843+0000","last_clean":"2024-03-19T00:36:57.943843+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:57.943843+0000","last_undegraded":"2024-03-19T00:36:57.943843+0000","last_fullsized":"2024-03-19T00:36:57.943843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946346+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:57.946346+0000","last_peered":"2024-03-19T00:36:57.946346+0000","last_clean":"2024-03-19T00:36:57.946346+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:57.946346+0000","last_undegraded":"2024-03-19T00:36:57.946346+0000","last_fullsized":"2024-03-19T00:36:57.946346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029217+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:37:56.029217+0000","last_peered":"2024-03-19T00:37:56.029217+0000","last_clean":"2024-03-19T00:37:56.029217+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:37:56.029217+0000","last_undegraded":"2024-03-19T00:37:56.029217+0000","last_fullsized":"2024-03-19T00:37:56.029217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944721+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:57.944721+0000","last_peered":"2024-03-19T00:36:57.944721+0000","last_clean":"2024-03-19T00:36:57.944721+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:57.944721+0000","last_undegraded":"2024-03-19T00:36:57.944721+0000","last_fullsized":"2024-03-19T00:36:57.944721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:08.310182+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:38:08.310182+0000","last_peered":"2024-03-19T00:38:08.310182+0000","last_clean":"2024-03-19T00:38:08.310182+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:38:08.310182+0000","last_undegraded":"2024-03-19T00:38:08.310182+0000","last_fullsized":"2024-03-19T00:38:08.310182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041614+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:37:56.041614+0000","last_peered":"2024-03-19T00:37:56.041614+0000","last_clean":"2024-03-19T00:37:56.041614+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:37:56.041614+0000","last_undegraded":"2024-03-19T00:37:56.041614+0000","last_fullsized":"2024-03-19T00:37:56.041614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944067+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:57.944067+0000","last_peered":"2024-03-19T00:36:57.944067+0000","last_clean":"2024-03-19T00:36:57.944067+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:57.944067+0000","last_undegraded":"2024-03-19T00:36:57.944067+0000","last_fullsized":"2024-03-19T00:36:57.944067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947723+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:57.947723+0000","last_peered":"2024-03-19T00:36:57.947723+0000","last_clean":"2024-03-19T00:36:57.947723+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:57.947723+0000","last_undegraded":"2024-03-19T00:36:57.947723+0000","last_fullsized":"2024-03-19T00:36:57.947723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039981+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:37:56.039981+0000","last_peered":"2024-03-19T00:37:56.039981+0000","last_clean":"2024-03-19T00:37:56.039981+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:37:56.039981+0000","last_undegraded":"2024-03-19T00:37:56.039981+0000","last_fullsized":"2024-03-19T00:37:56.039981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946223+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:57.946223+0000","last_peered":"2024-03-19T00:36:57.946223+0000","last_clean":"2024-03-19T00:36:57.946223+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:57.946223+0000","last_undegraded":"2024-03-19T00:36:57.946223+0000","last_fullsized":"2024-03-19T00:36:57.946223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944993+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:57.944993+0000","last_peered":"2024-03-19T00:36:57.944993+0000","last_clean":"2024-03-19T00:36:57.944993+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:57.944993+0000","last_undegraded":"2024-03-19T00:36:57.944993+0000","last_fullsized":"2024-03-19T00:36:57.944993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040908+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:37:56.040908+0000","last_peered":"2024-03-19T00:37:56.040908+0000","last_clean":"2024-03-19T00:37:56.040908+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:37:56.040908+0000","last_undegraded":"2024-03-19T00:37:56.040908+0000","last_fullsized":"2024-03-19T00:37:56.040908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947224+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:57.947224+0000","last_peered":"2024-03-19T00:36:57.947224+0000","last_clean":"2024-03-19T00:36:57.947224+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:57.947224+0000","last_undegraded":"2024-03-19T00:36:57.947224+0000","last_fullsized":"2024-03-19T00:36:57.947224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341995+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:58.341995+0000","last_peered":"2024-03-19T00:36:58.341995+0000","last_clean":"2024-03-19T00:36:58.341995+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:58.341995+0000","last_undegraded":"2024-03-19T00:36:58.341995+0000","last_fullsized":"2024-03-19T00:36:58.341995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031881+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:37:56.031881+0000","last_peered":"2024-03-19T00:37:56.031881+0000","last_clean":"2024-03-19T00:37:56.031881+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:37:56.031881+0000","last_undegraded":"2024-03-19T00:37:56.031881+0000","last_fullsized":"2024-03-19T00:37:56.031881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341269+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:58.341269+0000","last_peered":"2024-03-19T00:36:58.341269+0000","last_clean":"2024-03-19T00:36:58.341269+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:58.341269+0000","last_undegraded":"2024-03-19T00:36:58.341269+0000","last_fullsized":"2024-03-19T00:36:58.341269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018797+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:58.018797+0000","last_peered":"2024-03-19T00:36:58.018797+0000","last_clean":"2024-03-19T00:36:58.018797+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:58.018797+0000","last_undegraded":"2024-03-19T00:36:58.018797+0000","last_fullsized":"2024-03-19T00:36:58.018797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031169+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:37:56.031169+0000","last_peered":"2024-03-19T00:37:56.031169+0000","last_clean":"2024-03-19T00:37:56.031169+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:37:56.031169+0000","last_undegraded":"2024-03-19T00:37:56.031169+0000","last_fullsized":"2024-03-19T00:37:56.031169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028083+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:37:56.028083+0000","last_peered":"2024-03-19T00:37:56.028083+0000","last_clean":"2024-03-19T00:37:56.028083+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:37:56.028083+0000","last_undegraded":"2024-03-19T00:37:56.028083+0000","last_fullsized":"2024-03-19T00:37:56.028083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.023829+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:37:56.023829+0000","last_peered":"2024-03-19T00:37:56.023829+0000","last_clean":"2024-03-19T00:37:56.023829+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:37:56.023829+0000","last_undegraded":"2024-03-19T00:37:56.023829+0000","last_fullsized":"2024-03-19T00:37:56.023829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040636+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:37:56.040636+0000","last_peered":"2024-03-19T00:37:56.040636+0000","last_clean":"2024-03-19T00:37:56.040636+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:37:56.040636+0000","last_undegraded":"2024-03-19T00:37:56.040636+0000","last_fullsized":"2024-03-19T00:37:56.040636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041821+0000","last_change":"2024-03-19T00:36:58.019535+0000","last_active":"2024-03-19T00:37:56.041821+0000","last_peered":"2024-03-19T00:37:56.041821+0000","last_clean":"2024-03-19T00:37:56.041821+0000","last_became_active":"2024-03-19T00:36:58.019220+0000","last_became_peered":"2024-03-19T00:36:58.019220+0000","last_unstale":"2024-03-19T00:37:56.041821+0000","last_undegraded":"2024-03-19T00:37:56.041821+0000","last_fullsized":"2024-03-19T00:37:56.041821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:04:51.797603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044655+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:37:56.044655+0000","last_peered":"2024-03-19T00:37:56.044655+0000","last_clean":"2024-03-19T00:37:56.044655+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:37:56.044655+0000","last_undegraded":"2024-03-19T00:37:56.044655+0000","last_fullsized":"2024-03-19T00:37:56.044655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.024108+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:37:56.024108+0000","last_peered":"2024-03-19T00:37:56.024108+0000","last_clean":"2024-03-19T00:37:56.024108+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:37:56.024108+0000","last_undegraded":"2024-03-19T00:37:56.024108+0000","last_fullsized":"2024-03-19T00:37:56.024108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042382+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:37:56.042382+0000","last_peered":"2024-03-19T00:37:56.042382+0000","last_clean":"2024-03-19T00:37:56.042382+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:37:56.042382+0000","last_undegraded":"2024-03-19T00:37:56.042382+0000","last_fullsized":"2024-03-19T00:37:56.042382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031736+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:37:56.031736+0000","last_peered":"2024-03-19T00:37:56.031736+0000","last_clean":"2024-03-19T00:37:56.031736+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:37:56.031736+0000","last_undegraded":"2024-03-19T00:37:56.031736+0000","last_fullsized":"2024-03-19T00:37:56.031736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.025787+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:37:56.025787+0000","last_peered":"2024-03-19T00:37:56.025787+0000","last_clean":"2024-03-19T00:37:56.025787+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:37:56.025787+0000","last_undegraded":"2024-03-19T00:37:56.025787+0000","last_fullsized":"2024-03-19T00:37:56.025787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.032629+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:37:56.032629+0000","last_peered":"2024-03-19T00:37:56.032629+0000","last_clean":"2024-03-19T00:37:56.032629+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:37:56.032629+0000","last_undegraded":"2024-03-19T00:37:56.032629+0000","last_fullsized":"2024-03-19T00:37:56.032629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.043776+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:37:56.043776+0000","last_peered":"2024-03-19T00:37:56.043776+0000","last_clean":"2024-03-19T00:37:56.043776+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:37:56.043776+0000","last_undegraded":"2024-03-19T00:37:56.043776+0000","last_fullsized":"2024-03-19T00:37:56.043776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.045485+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:37:56.045485+0000","last_peered":"2024-03-19T00:37:56.045485+0000","last_clean":"2024-03-19T00:37:56.045485+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:37:56.045485+0000","last_undegraded":"2024-03-19T00:37:56.045485+0000","last_fullsized":"2024-03-19T00:37:56.045485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029358+0000","last_change":"2024-03-19T00:36:57.951256+0000","last_active":"2024-03-19T00:37:56.029358+0000","last_peered":"2024-03-19T00:37:56.029358+0000","last_clean":"2024-03-19T00:37:56.029358+0000","last_became_active":"2024-03-19T00:36:57.950879+0000","last_became_peered":"2024-03-19T00:36:57.950879+0000","last_unstale":"2024-03-19T00:37:56.029358+0000","last_undegraded":"2024-03-19T00:37:56.029358+0000","last_fullsized":"2024-03-19T00:37:56.029358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:26.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69762,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":446,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":214773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1153209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":500971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1099323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":564942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":561625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1092609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:38:28.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T00:38:28.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 -- 172.21.15.7:0/4002398799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 msgr2=0x7f6a88100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/4002398799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a88100be0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009900 tx=0x7f6a7c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 -- 172.21.15.7:0/4002398799 shutdown_connections 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/4002398799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a88102370 0x7f6a88108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/4002398799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a881019b0 0x7f6a88101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/4002398799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a88100be0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 -- 172.21.15.7:0/4002398799 >> 172.21.15.7:0/4002398799 conn(0x7f6a880fbd30 msgr2=0x7f6a880fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 -- 172.21.15.7:0/4002398799 shutdown_connections 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.259+0000 7f6a8f09a640 1 -- 172.21.15.7:0/4002398799 wait complete. 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 Processor -- start 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 -- start start 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a8806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881019b0 0x7f6a8819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a88102370 0x7f6a8819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810c250 con 0x7f6a881007c0 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810c0d0 con 0x7f6a88102370 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8f09a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810c3d0 con 0x7f6a881019b0 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a8806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a8806aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42844/0 (socket says 172.21.15.7:42844) 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 -- 172.21.15.7:0/2204534684 learned_addr learned my addr 172.21.15.7:0/2204534684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8d637640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a88102370 0x7f6a8819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a87fff640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881019b0 0x7f6a8819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 -- 172.21.15.7:0/2204534684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881019b0 msgr2=0x7f6a8819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881019b0 0x7f6a8819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 -- 172.21.15.7:0/2204534684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a88102370 msgr2=0x7f6a8819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a88102370 0x7f6a8819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 -- 172.21.15.7:0/2204534684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8819f720 con 0x7f6a881007c0 2024-03-19T00:38:28.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8d637640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a88102370 0x7f6a8819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.260+0000 7f6a8ce36640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a8806aab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.261+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c043020 con 0x7f6a881007c0 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.261+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a7c004350 con 0x7f6a881007c0 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.261+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8819f950 con 0x7f6a881007c0 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.261+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003b80 con 0x7f6a881007c0 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.261+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8819fea0 con 0x7f6a881007c0 2024-03-19T00:38:28.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.262+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a88068af0 con 0x7f6a881007c0 2024-03-19T00:38:28.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:27 smithi204 ceph-mon[66572]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:38:28.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:27 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1641784428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:28.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:27 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3616183352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:28.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:27 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1488720099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:28.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.263+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f6a7c046020 con 0x7f6a881007c0 2024-03-19T00:38:28.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.263+0000 7f6a85ffb640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2890 0x7f6a780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.263+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6a7c0f0fc0 con 0x7f6a881007c0 2024-03-19T00:38:28.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.263+0000 7f6a87fff640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2890 0x7f6a780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.264+0000 7f6a87fff640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2890 0x7f6a780a4d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6a8806a820 tx=0x7f6a74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:28.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.268+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6a7c04d020 con 0x7f6a881007c0 2024-03-19T00:38:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:27 smithi044 ceph-mon[66366]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:38:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:27 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1641784428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:27 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3616183352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:27 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1488720099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.449+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a88061c60 con 0x7f6a780a2890 2024-03-19T00:38:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.459+0000 7f6a85ffb640 1 -- 172.21.15.7:0/2204534684 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340481 (secure 0 0 0) 0x7f6a88061c60 con 0x7f6a780a2890 2024-03-19T00:38:28.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2890 msgr2=0x7f6a780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2890 0x7f6a780a4d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6a8806a820 tx=0x7f6a74005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 msgr2=0x7f6a8806aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a8806aab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 shutdown_connections 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2890 0x7f6a780a4d50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a88102370 0x7f6a8819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881019b0 0x7f6a8819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 --2- 172.21.15.7:0/2204534684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a881007c0 0x7f6a8806aab0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 >> 172.21.15.7:0/2204534684 conn(0x7f6a880fbd30 msgr2=0x7f6a88104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 shutdown_connections 2024-03-19T00:38:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.463+0000 7f6a8f09a640 1 -- 172.21.15.7:0/2204534684 wait complete. 2024-03-19T00:38:28.476 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-03-19T00:38:28.038034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660364,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":867,"num_read_kb":555,"num_write":366,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404852,"kb_used_data":14100,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536972,"statfs":{"total":1151940427776,"available":1151525859328,"internally_reserved":0,"allocated":14438400,"data_stored":7970807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1482,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":32,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029289+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:37:56.029289+0000","last_peered":"2024-03-19T00:37:56.029289+0000","last_clean":"2024-03-19T00:37:56.029289+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:37:56.029289+0000","last_undegraded":"2024-03-19T00:37:56.029289+0000","last_fullsized":"2024-03-19T00:37:56.029289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039878+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:37:56.039878+0000","last_peered":"2024-03-19T00:37:56.039878+0000","last_clean":"2024-03-19T00:37:56.039878+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:37:56.039878+0000","last_undegraded":"2024-03-19T00:37:56.039878+0000","last_fullsized":"2024-03-19T00:37:56.039878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042753+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:37:56.042753+0000","last_peered":"2024-03-19T00:37:56.042753+0000","last_clean":"2024-03-19T00:37:56.042753+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:37:56.042753+0000","last_undegraded":"2024-03-19T00:37:56.042753+0000","last_fullsized":"2024-03-19T00:37:56.042753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.019845+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:37:56.019845+0000","last_peered":"2024-03-19T00:37:56.019845+0000","last_clean":"2024-03-19T00:37:56.019845+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:37:56.019845+0000","last_undegraded":"2024-03-19T00:37:56.019845+0000","last_fullsized":"2024-03-19T00:37:56.019845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.017353+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:37:56.017353+0000","last_peered":"2024-03-19T00:37:56.017353+0000","last_clean":"2024-03-19T00:37:56.017353+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:37:56.017353+0000","last_undegraded":"2024-03-19T00:37:56.017353+0000","last_fullsized":"2024-03-19T00:37:56.017353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029411+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:37:56.029411+0000","last_peered":"2024-03-19T00:37:56.029411+0000","last_clean":"2024-03-19T00:37:56.029411+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:37:56.029411+0000","last_undegraded":"2024-03-19T00:37:56.029411+0000","last_fullsized":"2024-03-19T00:37:56.029411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.022473+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:37:56.022473+0000","last_peered":"2024-03-19T00:37:56.022473+0000","last_clean":"2024-03-19T00:37:56.022473+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:37:56.022473+0000","last_undegraded":"2024-03-19T00:37:56.022473+0000","last_fullsized":"2024-03-19T00:37:56.022473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.034359+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:37:56.034359+0000","last_peered":"2024-03-19T00:37:56.034359+0000","last_clean":"2024-03-19T00:37:56.034359+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:37:56.034359+0000","last_undegraded":"2024-03-19T00:37:56.034359+0000","last_fullsized":"2024-03-19T00:37:56.034359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041752+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:37:56.041752+0000","last_peered":"2024-03-19T00:37:56.041752+0000","last_clean":"2024-03-19T00:37:56.041752+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:37:56.041752+0000","last_undegraded":"2024-03-19T00:37:56.041752+0000","last_fullsized":"2024-03-19T00:37:56.041752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.035497+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:37:56.035497+0000","last_peered":"2024-03-19T00:37:56.035497+0000","last_clean":"2024-03-19T00:37:56.035497+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:37:56.035497+0000","last_undegraded":"2024-03-19T00:37:56.035497+0000","last_fullsized":"2024-03-19T00:37:56.035497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040785+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:37:56.040785+0000","last_peered":"2024-03-19T00:37:56.040785+0000","last_clean":"2024-03-19T00:37:56.040785+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:37:56.040785+0000","last_undegraded":"2024-03-19T00:37:56.040785+0000","last_fullsized":"2024-03-19T00:37:56.040785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031924+0000","last_change":"2024-03-19T00:36:58.297533+0000","last_active":"2024-03-19T00:37:56.031924+0000","last_peered":"2024-03-19T00:37:56.031924+0000","last_clean":"2024-03-19T00:37:56.031924+0000","last_became_active":"2024-03-19T00:36:58.297218+0000","last_became_peered":"2024-03-19T00:36:58.297218+0000","last_unstale":"2024-03-19T00:37:56.031924+0000","last_undegraded":"2024-03-19T00:37:56.031924+0000","last_fullsized":"2024-03-19T00:37:56.031924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:06.256765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039054+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:37:56.039054+0000","last_peered":"2024-03-19T00:37:56.039054+0000","last_clean":"2024-03-19T00:37:56.039054+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:37:56.039054+0000","last_undegraded":"2024-03-19T00:37:56.039054+0000","last_fullsized":"2024-03-19T00:37:56.039054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039900+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:37:56.039900+0000","last_peered":"2024-03-19T00:37:56.039900+0000","last_clean":"2024-03-19T00:37:56.039900+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:37:56.039900+0000","last_undegraded":"2024-03-19T00:37:56.039900+0000","last_fullsized":"2024-03-19T00:37:56.039900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042705+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:37:56.042705+0000","last_peered":"2024-03-19T00:37:56.042705+0000","last_clean":"2024-03-19T00:37:56.042705+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:37:56.042705+0000","last_undegraded":"2024-03-19T00:37:56.042705+0000","last_fullsized":"2024-03-19T00:37:56.042705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042051+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:37:56.042051+0000","last_peered":"2024-03-19T00:37:56.042051+0000","last_clean":"2024-03-19T00:37:56.042051+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:37:56.042051+0000","last_undegraded":"2024-03-19T00:37:56.042051+0000","last_fullsized":"2024-03-19T00:37:56.042051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029319+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:37:56.029319+0000","last_peered":"2024-03-19T00:37:56.029319+0000","last_clean":"2024-03-19T00:37:56.029319+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:37:56.029319+0000","last_undegraded":"2024-03-19T00:37:56.029319+0000","last_fullsized":"2024-03-19T00:37:56.029319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.032743+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:37:56.032743+0000","last_peered":"2024-03-19T00:37:56.032743+0000","last_clean":"2024-03-19T00:37:56.032743+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:37:56.032743+0000","last_undegraded":"2024-03-19T00:37:56.032743+0000","last_fullsized":"2024-03-19T00:37:56.032743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.030215+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:37:56.030215+0000","last_peered":"2024-03-19T00:37:56.030215+0000","last_clean":"2024-03-19T00:37:56.030215+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:37:56.030215+0000","last_undegraded":"2024-03-19T00:37:56.030215+0000","last_fullsized":"2024-03-19T00:37:56.030215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944493+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:57.944493+0000","last_peered":"2024-03-19T00:36:57.944493+0000","last_clean":"2024-03-19T00:36:57.944493+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:57.944493+0000","last_undegraded":"2024-03-19T00:36:57.944493+0000","last_fullsized":"2024-03-19T00:36:57.944493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943877+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:57.943877+0000","last_peered":"2024-03-19T00:36:57.943877+0000","last_clean":"2024-03-19T00:36:57.943877+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:57.943877+0000","last_undegraded":"2024-03-19T00:36:57.943877+0000","last_fullsized":"2024-03-19T00:36:57.943877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028440+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:37:56.028440+0000","last_peered":"2024-03-19T00:37:56.028440+0000","last_clean":"2024-03-19T00:37:56.028440+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:37:56.028440+0000","last_undegraded":"2024-03-19T00:37:56.028440+0000","last_fullsized":"2024-03-19T00:37:56.028440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943659+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:57.943659+0000","last_peered":"2024-03-19T00:36:57.943659+0000","last_clean":"2024-03-19T00:36:57.943659+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:57.943659+0000","last_undegraded":"2024-03-19T00:36:57.943659+0000","last_fullsized":"2024-03-19T00:36:57.943659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947548+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:57.947548+0000","last_peered":"2024-03-19T00:36:57.947548+0000","last_clean":"2024-03-19T00:36:57.947548+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:57.947548+0000","last_undegraded":"2024-03-19T00:36:57.947548+0000","last_fullsized":"2024-03-19T00:36:57.947548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044072+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:37:56.044072+0000","last_peered":"2024-03-19T00:37:56.044072+0000","last_clean":"2024-03-19T00:37:56.044072+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:37:56.044072+0000","last_undegraded":"2024-03-19T00:37:56.044072+0000","last_fullsized":"2024-03-19T00:37:56.044072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943869+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:57.943869+0000","last_peered":"2024-03-19T00:36:57.943869+0000","last_clean":"2024-03-19T00:36:57.943869+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:57.943869+0000","last_undegraded":"2024-03-19T00:36:57.943869+0000","last_fullsized":"2024-03-19T00:36:57.943869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947966+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:57.947966+0000","last_peered":"2024-03-19T00:36:57.947966+0000","last_clean":"2024-03-19T00:36:57.947966+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:57.947966+0000","last_undegraded":"2024-03-19T00:36:57.947966+0000","last_fullsized":"2024-03-19T00:36:57.947966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044222+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:37:56.044222+0000","last_peered":"2024-03-19T00:37:56.044222+0000","last_clean":"2024-03-19T00:37:56.044222+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:37:56.044222+0000","last_undegraded":"2024-03-19T00:37:56.044222+0000","last_fullsized":"2024-03-19T00:37:56.044222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943744+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:57.943744+0000","last_peered":"2024-03-19T00:36:57.943744+0000","last_clean":"2024-03-19T00:36:57.943744+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:57.943744+0000","last_undegraded":"2024-03-19T00:36:57.943744+0000","last_fullsized":"2024-03-19T00:36:57.943744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948143+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:57.948143+0000","last_peered":"2024-03-19T00:36:57.948143+0000","last_clean":"2024-03-19T00:36:57.948143+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:57.948143+0000","last_undegraded":"2024-03-19T00:36:57.948143+0000","last_fullsized":"2024-03-19T00:36:57.948143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042138+0000","last_change":"2024-03-19T00:36:57.948938+0000","last_active":"2024-03-19T00:37:56.042138+0000","last_peered":"2024-03-19T00:37:56.042138+0000","last_clean":"2024-03-19T00:37:56.042138+0000","last_became_active":"2024-03-19T00:36:57.948519+0000","last_became_peered":"2024-03-19T00:36:57.948519+0000","last_unstale":"2024-03-19T00:37:56.042138+0000","last_undegraded":"2024-03-19T00:37:56.042138+0000","last_fullsized":"2024-03-19T00:37:56.042138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:33:49.099555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.298764+0000","last_change":"2024-03-19T00:36:58.298764+0000","last_active":"2024-03-19T00:36:58.298764+0000","last_peered":"2024-03-19T00:36:58.298764+0000","last_clean":"2024-03-19T00:36:58.298764+0000","last_became_active":"2024-03-19T00:36:58.298265+0000","last_became_peered":"2024-03-19T00:36:58.298265+0000","last_unstale":"2024-03-19T00:36:58.298764+0000","last_undegraded":"2024-03-19T00:36:58.298764+0000","last_fullsized":"2024-03-19T00:36:58.298764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:45.983861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:22.428477+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:38:22.428477+0000","last_peered":"2024-03-19T00:38:22.428477+0000","last_clean":"2024-03-19T00:38:22.428477+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:38:22.428477+0000","last_undegraded":"2024-03-19T00:38:22.428477+0000","last_fullsized":"2024-03-19T00:38:22.428477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1648,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.014746+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:37:56.014746+0000","last_peered":"2024-03-19T00:37:56.014746+0000","last_clean":"2024-03-19T00:37:56.014746+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:37:56.014746+0000","last_undegraded":"2024-03-19T00:37:56.014746+0000","last_fullsized":"2024-03-19T00:37:56.014746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342937+0000","last_change":"2024-03-19T00:36:58.342937+0000","last_active":"2024-03-19T00:36:58.342937+0000","last_peered":"2024-03-19T00:36:58.342937+0000","last_clean":"2024-03-19T00:36:58.342937+0000","last_became_active":"2024-03-19T00:36:58.342474+0000","last_became_peered":"2024-03-19T00:36:58.342474+0000","last_unstale":"2024-03-19T00:36:58.342937+0000","last_undegraded":"2024-03-19T00:36:58.342937+0000","last_fullsized":"2024-03-19T00:36:58.342937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:55.531546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297043+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:58.297043+0000","last_peered":"2024-03-19T00:36:58.297043+0000","last_clean":"2024-03-19T00:36:58.297043+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:58.297043+0000","last_undegraded":"2024-03-19T00:36:58.297043+0000","last_fullsized":"2024-03-19T00:36:58.297043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.027675+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:37:56.027675+0000","last_peered":"2024-03-19T00:37:56.027675+0000","last_clean":"2024-03-19T00:37:56.027675+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:37:56.027675+0000","last_undegraded":"2024-03-19T00:37:56.027675+0000","last_fullsized":"2024-03-19T00:37:56.027675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.951395+0000","last_change":"2024-03-19T00:36:57.950687+0000","last_active":"2024-03-19T00:36:57.951395+0000","last_peered":"2024-03-19T00:36:57.951395+0000","last_clean":"2024-03-19T00:36:57.951395+0000","last_became_active":"2024-03-19T00:36:57.950151+0000","last_became_peered":"2024-03-19T00:36:57.950151+0000","last_unstale":"2024-03-19T00:36:57.951395+0000","last_undegraded":"2024-03-19T00:36:57.951395+0000","last_fullsized":"2024-03-19T00:36:57.951395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:41.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018277+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:58.018277+0000","last_peered":"2024-03-19T00:36:58.018277+0000","last_clean":"2024-03-19T00:36:58.018277+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:58.018277+0000","last_undegraded":"2024-03-19T00:36:58.018277+0000","last_fullsized":"2024-03-19T00:36:58.018277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033347+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:37:56.033347+0000","last_peered":"2024-03-19T00:37:56.033347+0000","last_clean":"2024-03-19T00:37:56.033347+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:37:56.033347+0000","last_undegraded":"2024-03-19T00:37:56.033347+0000","last_fullsized":"2024-03-19T00:37:56.033347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947452+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:57.947452+0000","last_peered":"2024-03-19T00:36:57.947452+0000","last_clean":"2024-03-19T00:36:57.947452+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:57.947452+0000","last_undegraded":"2024-03-19T00:36:57.947452+0000","last_fullsized":"2024-03-19T00:36:57.947452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341398+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:58.341398+0000","last_peered":"2024-03-19T00:36:58.341398+0000","last_clean":"2024-03-19T00:36:58.341398+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:58.341398+0000","last_undegraded":"2024-03-19T00:36:58.341398+0000","last_fullsized":"2024-03-19T00:36:58.341398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033250+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:37:56.033250+0000","last_peered":"2024-03-19T00:37:56.033250+0000","last_clean":"2024-03-19T00:37:56.033250+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:37:56.033250+0000","last_undegraded":"2024-03-19T00:37:56.033250+0000","last_fullsized":"2024-03-19T00:37:56.033250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946524+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:57.946524+0000","last_peered":"2024-03-19T00:36:57.946524+0000","last_clean":"2024-03-19T00:36:57.946524+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:57.946524+0000","last_undegraded":"2024-03-19T00:36:57.946524+0000","last_fullsized":"2024-03-19T00:36:57.946524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.429184+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:38:23.429184+0000","last_peered":"2024-03-19T00:38:23.429184+0000","last_clean":"2024-03-19T00:38:23.429184+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:38:23.429184+0000","last_undegraded":"2024-03-19T00:38:23.429184+0000","last_fullsized":"2024-03-19T00:38:23.429184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044130+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:37:56.044130+0000","last_peered":"2024-03-19T00:37:56.044130+0000","last_clean":"2024-03-19T00:37:56.044130+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:37:56.044130+0000","last_undegraded":"2024-03-19T00:37:56.044130+0000","last_fullsized":"2024-03-19T00:37:56.044130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943224+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:57.943224+0000","last_peered":"2024-03-19T00:36:57.943224+0000","last_clean":"2024-03-19T00:36:57.943224+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:57.943224+0000","last_undegraded":"2024-03-19T00:36:57.943224+0000","last_fullsized":"2024-03-19T00:36:57.943224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092881+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:58.092881+0000","last_peered":"2024-03-19T00:36:58.092881+0000","last_clean":"2024-03-19T00:36:58.092881+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:58.092881+0000","last_undegraded":"2024-03-19T00:36:58.092881+0000","last_fullsized":"2024-03-19T00:36:58.092881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.037121+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:37:56.037121+0000","last_peered":"2024-03-19T00:37:56.037121+0000","last_clean":"2024-03-19T00:37:56.037121+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:37:56.037121+0000","last_undegraded":"2024-03-19T00:37:56.037121+0000","last_fullsized":"2024-03-19T00:37:56.037121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944132+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:57.944132+0000","last_peered":"2024-03-19T00:36:57.944132+0000","last_clean":"2024-03-19T00:36:57.944132+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:57.944132+0000","last_undegraded":"2024-03-19T00:36:57.944132+0000","last_fullsized":"2024-03-19T00:36:57.944132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341884+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:58.341884+0000","last_peered":"2024-03-19T00:36:58.341884+0000","last_clean":"2024-03-19T00:36:58.341884+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:58.341884+0000","last_undegraded":"2024-03-19T00:36:58.341884+0000","last_fullsized":"2024-03-19T00:36:58.341884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042790+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:37:56.042790+0000","last_peered":"2024-03-19T00:37:56.042790+0000","last_clean":"2024-03-19T00:37:56.042790+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:37:56.042790+0000","last_undegraded":"2024-03-19T00:37:56.042790+0000","last_fullsized":"2024-03-19T00:37:56.042790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943327+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:57.943327+0000","last_peered":"2024-03-19T00:36:57.943327+0000","last_clean":"2024-03-19T00:36:57.943327+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:57.943327+0000","last_undegraded":"2024-03-19T00:36:57.943327+0000","last_fullsized":"2024-03-19T00:36:57.943327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":312,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.429728+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:38:23.429728+0000","last_peered":"2024-03-19T00:38:23.429728+0000","last_clean":"2024-03-19T00:38:23.429728+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:38:23.429728+0000","last_undegraded":"2024-03-19T00:38:23.429728+0000","last_fullsized":"2024-03-19T00:38:23.429728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":186,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.018651+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:37:56.018651+0000","last_peered":"2024-03-19T00:37:56.018651+0000","last_clean":"2024-03-19T00:37:56.018651+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:37:56.018651+0000","last_undegraded":"2024-03-19T00:37:56.018651+0000","last_fullsized":"2024-03-19T00:37:56.018651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342057+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:58.342057+0000","last_peered":"2024-03-19T00:36:58.342057+0000","last_clean":"2024-03-19T00:36:58.342057+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:58.342057+0000","last_undegraded":"2024-03-19T00:36:58.342057+0000","last_fullsized":"2024-03-19T00:36:58.342057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945086+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:57.945086+0000","last_peered":"2024-03-19T00:36:57.945086+0000","last_clean":"2024-03-19T00:36:57.945086+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:57.945086+0000","last_undegraded":"2024-03-19T00:36:57.945086+0000","last_fullsized":"2024-03-19T00:36:57.945086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033786+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:37:56.033786+0000","last_peered":"2024-03-19T00:37:56.033786+0000","last_clean":"2024-03-19T00:37:56.033786+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:37:56.033786+0000","last_undegraded":"2024-03-19T00:37:56.033786+0000","last_fullsized":"2024-03-19T00:37:56.033786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945403+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:57.945403+0000","last_peered":"2024-03-19T00:36:57.945403+0000","last_clean":"2024-03-19T00:36:57.945403+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:57.945403+0000","last_undegraded":"2024-03-19T00:36:57.945403+0000","last_fullsized":"2024-03-19T00:36:57.945403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341889+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:58.341889+0000","last_peered":"2024-03-19T00:36:58.341889+0000","last_clean":"2024-03-19T00:36:58.341889+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:58.341889+0000","last_undegraded":"2024-03-19T00:36:58.341889+0000","last_fullsized":"2024-03-19T00:36:58.341889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.035315+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:37:56.035315+0000","last_peered":"2024-03-19T00:37:56.035315+0000","last_clean":"2024-03-19T00:37:56.035315+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:37:56.035315+0000","last_undegraded":"2024-03-19T00:37:56.035315+0000","last_fullsized":"2024-03-19T00:37:56.035315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943691+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:57.943691+0000","last_peered":"2024-03-19T00:36:57.943691+0000","last_clean":"2024-03-19T00:36:57.943691+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:57.943691+0000","last_undegraded":"2024-03-19T00:36:57.943691+0000","last_fullsized":"2024-03-19T00:36:57.943691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.445021+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:38:23.445021+0000","last_peered":"2024-03-19T00:38:23.445021+0000","last_clean":"2024-03-19T00:38:23.445021+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:38:23.445021+0000","last_undegraded":"2024-03-19T00:38:23.445021+0000","last_fullsized":"2024-03-19T00:38:23.445021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1614,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033957+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:37:56.033957+0000","last_peered":"2024-03-19T00:37:56.033957+0000","last_clean":"2024-03-19T00:37:56.033957+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:37:56.033957+0000","last_undegraded":"2024-03-19T00:37:56.033957+0000","last_fullsized":"2024-03-19T00:37:56.033957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018782+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:58.018782+0000","last_peered":"2024-03-19T00:36:58.018782+0000","last_clean":"2024-03-19T00:36:58.018782+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:58.018782+0000","last_undegraded":"2024-03-19T00:36:58.018782+0000","last_fullsized":"2024-03-19T00:36:58.018782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947702+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:57.947702+0000","last_peered":"2024-03-19T00:36:57.947702+0000","last_clean":"2024-03-19T00:36:57.947702+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:57.947702+0000","last_undegraded":"2024-03-19T00:36:57.947702+0000","last_fullsized":"2024-03-19T00:36:57.947702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026513+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:37:56.026513+0000","last_peered":"2024-03-19T00:37:56.026513+0000","last_clean":"2024-03-19T00:37:56.026513+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:37:56.026513+0000","last_undegraded":"2024-03-19T00:37:56.026513+0000","last_fullsized":"2024-03-19T00:37:56.026513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341751+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:58.341751+0000","last_peered":"2024-03-19T00:36:58.341751+0000","last_clean":"2024-03-19T00:36:58.341751+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:58.341751+0000","last_undegraded":"2024-03-19T00:36:58.341751+0000","last_fullsized":"2024-03-19T00:36:58.341751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948779+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:57.948779+0000","last_peered":"2024-03-19T00:36:57.948779+0000","last_clean":"2024-03-19T00:36:57.948779+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:57.948779+0000","last_undegraded":"2024-03-19T00:36:57.948779+0000","last_fullsized":"2024-03-19T00:36:57.948779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.024437+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:37:56.024437+0000","last_peered":"2024-03-19T00:37:56.024437+0000","last_clean":"2024-03-19T00:37:56.024437+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:37:56.024437+0000","last_undegraded":"2024-03-19T00:37:56.024437+0000","last_fullsized":"2024-03-19T00:37:56.024437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947891+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:57.947891+0000","last_peered":"2024-03-19T00:36:57.947891+0000","last_clean":"2024-03-19T00:36:57.947891+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:57.947891+0000","last_undegraded":"2024-03-19T00:36:57.947891+0000","last_fullsized":"2024-03-19T00:36:57.947891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943536+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:57.943536+0000","last_peered":"2024-03-19T00:36:57.943536+0000","last_clean":"2024-03-19T00:36:57.943536+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:57.943536+0000","last_undegraded":"2024-03-19T00:36:57.943536+0000","last_fullsized":"2024-03-19T00:36:57.943536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.037115+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:37:56.037115+0000","last_peered":"2024-03-19T00:37:56.037115+0000","last_clean":"2024-03-19T00:37:56.037115+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:37:56.037115+0000","last_undegraded":"2024-03-19T00:37:56.037115+0000","last_fullsized":"2024-03-19T00:37:56.037115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018669+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:58.018669+0000","last_peered":"2024-03-19T00:36:58.018669+0000","last_clean":"2024-03-19T00:36:58.018669+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:58.018669+0000","last_undegraded":"2024-03-19T00:36:58.018669+0000","last_fullsized":"2024-03-19T00:36:58.018669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092575+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:58.092575+0000","last_peered":"2024-03-19T00:36:58.092575+0000","last_clean":"2024-03-19T00:36:58.092575+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:58.092575+0000","last_undegraded":"2024-03-19T00:36:58.092575+0000","last_fullsized":"2024-03-19T00:36:58.092575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.034570+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:37:56.034570+0000","last_peered":"2024-03-19T00:37:56.034570+0000","last_clean":"2024-03-19T00:37:56.034570+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:37:56.034570+0000","last_undegraded":"2024-03-19T00:37:56.034570+0000","last_fullsized":"2024-03-19T00:37:56.034570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018759+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:58.018759+0000","last_peered":"2024-03-19T00:36:58.018759+0000","last_clean":"2024-03-19T00:36:58.018759+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:58.018759+0000","last_undegraded":"2024-03-19T00:36:58.018759+0000","last_fullsized":"2024-03-19T00:36:58.018759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026479+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:37:56.026479+0000","last_peered":"2024-03-19T00:37:56.026479+0000","last_clean":"2024-03-19T00:37:56.026479+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:37:56.026479+0000","last_undegraded":"2024-03-19T00:37:56.026479+0000","last_fullsized":"2024-03-19T00:37:56.026479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.444526+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:38:23.444526+0000","last_peered":"2024-03-19T00:38:23.444526+0000","last_clean":"2024-03-19T00:38:23.444526+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:38:23.444526+0000","last_undegraded":"2024-03-19T00:38:23.444526+0000","last_fullsized":"2024-03-19T00:38:23.444526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092869+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:58.092869+0000","last_peered":"2024-03-19T00:36:58.092869+0000","last_clean":"2024-03-19T00:36:58.092869+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:58.092869+0000","last_undegraded":"2024-03-19T00:36:58.092869+0000","last_fullsized":"2024-03-19T00:36:58.092869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.043291+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:37:56.043291+0000","last_peered":"2024-03-19T00:37:56.043291+0000","last_clean":"2024-03-19T00:37:56.043291+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:37:56.043291+0000","last_undegraded":"2024-03-19T00:37:56.043291+0000","last_fullsized":"2024-03-19T00:37:56.043291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944026+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:57.944026+0000","last_peered":"2024-03-19T00:36:57.944026+0000","last_clean":"2024-03-19T00:36:57.944026+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:57.944026+0000","last_undegraded":"2024-03-19T00:36:57.944026+0000","last_fullsized":"2024-03-19T00:36:57.944026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092415+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:58.092415+0000","last_peered":"2024-03-19T00:36:58.092415+0000","last_clean":"2024-03-19T00:36:58.092415+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:58.092415+0000","last_undegraded":"2024-03-19T00:36:58.092415+0000","last_fullsized":"2024-03-19T00:36:58.092415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342005+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:58.342005+0000","last_peered":"2024-03-19T00:36:58.342005+0000","last_clean":"2024-03-19T00:36:58.342005+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:58.342005+0000","last_undegraded":"2024-03-19T00:36:58.342005+0000","last_fullsized":"2024-03-19T00:36:58.342005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028043+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:37:56.028043+0000","last_peered":"2024-03-19T00:37:56.028043+0000","last_clean":"2024-03-19T00:37:56.028043+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:37:56.028043+0000","last_undegraded":"2024-03-19T00:37:56.028043+0000","last_fullsized":"2024-03-19T00:37:56.028043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092489+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:58.092489+0000","last_peered":"2024-03-19T00:36:58.092489+0000","last_clean":"2024-03-19T00:36:58.092489+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:58.092489+0000","last_undegraded":"2024-03-19T00:36:58.092489+0000","last_fullsized":"2024-03-19T00:36:58.092489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945808+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:57.945808+0000","last_peered":"2024-03-19T00:36:57.945808+0000","last_clean":"2024-03-19T00:36:57.945808+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:57.945808+0000","last_undegraded":"2024-03-19T00:36:57.945808+0000","last_fullsized":"2024-03-19T00:36:57.945808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948192+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:57.948192+0000","last_peered":"2024-03-19T00:36:57.948192+0000","last_clean":"2024-03-19T00:36:57.948192+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:57.948192+0000","last_undegraded":"2024-03-19T00:36:57.948192+0000","last_fullsized":"2024-03-19T00:36:57.948192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026012+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:37:56.026012+0000","last_peered":"2024-03-19T00:37:56.026012+0000","last_clean":"2024-03-19T00:37:56.026012+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:37:56.026012+0000","last_undegraded":"2024-03-19T00:37:56.026012+0000","last_fullsized":"2024-03-19T00:37:56.026012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946095+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:57.946095+0000","last_peered":"2024-03-19T00:36:57.946095+0000","last_clean":"2024-03-19T00:36:57.946095+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:57.946095+0000","last_undegraded":"2024-03-19T00:36:57.946095+0000","last_fullsized":"2024-03-19T00:36:57.946095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.219170+0000","last_change":"2024-03-19T00:36:57.954788+0000","last_active":"2024-03-19T00:38:23.219170+0000","last_peered":"2024-03-19T00:38:23.219170+0000","last_clean":"2024-03-19T00:38:23.219170+0000","last_became_active":"2024-03-19T00:36:57.949916+0000","last_became_peered":"2024-03-19T00:36:57.949916+0000","last_unstale":"2024-03-19T00:38:23.219170+0000","last_undegraded":"2024-03-19T00:38:23.219170+0000","last_fullsized":"2024-03-19T00:38:23.219170+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:09.959915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033677+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:37:56.033677+0000","last_peered":"2024-03-19T00:37:56.033677+0000","last_clean":"2024-03-19T00:37:56.033677+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:37:56.033677+0000","last_undegraded":"2024-03-19T00:37:56.033677+0000","last_fullsized":"2024-03-19T00:37:56.033677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944256+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:57.944256+0000","last_peered":"2024-03-19T00:36:57.944256+0000","last_clean":"2024-03-19T00:36:57.944256+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:57.944256+0000","last_undegraded":"2024-03-19T00:36:57.944256+0000","last_fullsized":"2024-03-19T00:36:57.944256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945162+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:57.945162+0000","last_peered":"2024-03-19T00:36:57.945162+0000","last_clean":"2024-03-19T00:36:57.945162+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:57.945162+0000","last_undegraded":"2024-03-19T00:36:57.945162+0000","last_fullsized":"2024-03-19T00:36:57.945162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044817+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:37:56.044817+0000","last_peered":"2024-03-19T00:37:56.044817+0000","last_clean":"2024-03-19T00:37:56.044817+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:37:56.044817+0000","last_undegraded":"2024-03-19T00:37:56.044817+0000","last_fullsized":"2024-03-19T00:37:56.044817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.343169+0000","last_change":"2024-03-19T00:36:58.342763+0000","last_active":"2024-03-19T00:36:58.343169+0000","last_peered":"2024-03-19T00:36:58.343169+0000","last_clean":"2024-03-19T00:36:58.343169+0000","last_became_active":"2024-03-19T00:36:58.342305+0000","last_became_peered":"2024-03-19T00:36:58.342305+0000","last_unstale":"2024-03-19T00:36:58.343169+0000","last_undegraded":"2024-03-19T00:36:58.343169+0000","last_fullsized":"2024-03-19T00:36:58.343169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:44.312347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.952606+0000","last_change":"2024-03-19T00:36:57.952182+0000","last_active":"2024-03-19T00:36:57.952606+0000","last_peered":"2024-03-19T00:36:57.952606+0000","last_clean":"2024-03-19T00:36:57.952606+0000","last_became_active":"2024-03-19T00:36:57.948822+0000","last_became_peered":"2024-03-19T00:36:57.948822+0000","last_unstale":"2024-03-19T00:36:57.952606+0000","last_undegraded":"2024-03-19T00:36:57.952606+0000","last_fullsized":"2024-03-19T00:36:57.952606+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.527748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026905+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:37:56.026905+0000","last_peered":"2024-03-19T00:37:56.026905+0000","last_clean":"2024-03-19T00:37:56.026905+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:37:56.026905+0000","last_undegraded":"2024-03-19T00:37:56.026905+0000","last_fullsized":"2024-03-19T00:37:56.026905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943843+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:57.943843+0000","last_peered":"2024-03-19T00:36:57.943843+0000","last_clean":"2024-03-19T00:36:57.943843+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:57.943843+0000","last_undegraded":"2024-03-19T00:36:57.943843+0000","last_fullsized":"2024-03-19T00:36:57.943843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946346+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:57.946346+0000","last_peered":"2024-03-19T00:36:57.946346+0000","last_clean":"2024-03-19T00:36:57.946346+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:57.946346+0000","last_undegraded":"2024-03-19T00:36:57.946346+0000","last_fullsized":"2024-03-19T00:36:57.946346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029217+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:37:56.029217+0000","last_peered":"2024-03-19T00:37:56.029217+0000","last_clean":"2024-03-19T00:37:56.029217+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:37:56.029217+0000","last_undegraded":"2024-03-19T00:37:56.029217+0000","last_fullsized":"2024-03-19T00:37:56.029217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944721+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:57.944721+0000","last_peered":"2024-03-19T00:36:57.944721+0000","last_clean":"2024-03-19T00:36:57.944721+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:57.944721+0000","last_undegraded":"2024-03-19T00:36:57.944721+0000","last_fullsized":"2024-03-19T00:36:57.944721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:08.310182+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:38:08.310182+0000","last_peered":"2024-03-19T00:38:08.310182+0000","last_clean":"2024-03-19T00:38:08.310182+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:38:08.310182+0000","last_undegraded":"2024-03-19T00:38:08.310182+0000","last_fullsized":"2024-03-19T00:38:08.310182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041614+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:37:56.041614+0000","last_peered":"2024-03-19T00:37:56.041614+0000","last_clean":"2024-03-19T00:37:56.041614+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:37:56.041614+0000","last_undegraded":"2024-03-19T00:37:56.041614+0000","last_fullsized":"2024-03-19T00:37:56.041614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944067+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:57.944067+0000","last_peered":"2024-03-19T00:36:57.944067+0000","last_clean":"2024-03-19T00:36:57.944067+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:57.944067+0000","last_undegraded":"2024-03-19T00:36:57.944067+0000","last_fullsized":"2024-03-19T00:36:57.944067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947723+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:57.947723+0000","last_peered":"2024-03-19T00:36:57.947723+0000","last_clean":"2024-03-19T00:36:57.947723+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:57.947723+0000","last_undegraded":"2024-03-19T00:36:57.947723+0000","last_fullsized":"2024-03-19T00:36:57.947723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039981+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:37:56.039981+0000","last_peered":"2024-03-19T00:37:56.039981+0000","last_clean":"2024-03-19T00:37:56.039981+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:37:56.039981+0000","last_undegraded":"2024-03-19T00:37:56.039981+0000","last_fullsized":"2024-03-19T00:37:56.039981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946223+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:57.946223+0000","last_peered":"2024-03-19T00:36:57.946223+0000","last_clean":"2024-03-19T00:36:57.946223+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:57.946223+0000","last_undegraded":"2024-03-19T00:36:57.946223+0000","last_fullsized":"2024-03-19T00:36:57.946223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944993+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:57.944993+0000","last_peered":"2024-03-19T00:36:57.944993+0000","last_clean":"2024-03-19T00:36:57.944993+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:57.944993+0000","last_undegraded":"2024-03-19T00:36:57.944993+0000","last_fullsized":"2024-03-19T00:36:57.944993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040908+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:37:56.040908+0000","last_peered":"2024-03-19T00:37:56.040908+0000","last_clean":"2024-03-19T00:37:56.040908+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:37:56.040908+0000","last_undegraded":"2024-03-19T00:37:56.040908+0000","last_fullsized":"2024-03-19T00:37:56.040908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947224+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:57.947224+0000","last_peered":"2024-03-19T00:36:57.947224+0000","last_clean":"2024-03-19T00:36:57.947224+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:57.947224+0000","last_undegraded":"2024-03-19T00:36:57.947224+0000","last_fullsized":"2024-03-19T00:36:57.947224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341995+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:58.341995+0000","last_peered":"2024-03-19T00:36:58.341995+0000","last_clean":"2024-03-19T00:36:58.341995+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:58.341995+0000","last_undegraded":"2024-03-19T00:36:58.341995+0000","last_fullsized":"2024-03-19T00:36:58.341995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031881+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:37:56.031881+0000","last_peered":"2024-03-19T00:37:56.031881+0000","last_clean":"2024-03-19T00:37:56.031881+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:37:56.031881+0000","last_undegraded":"2024-03-19T00:37:56.031881+0000","last_fullsized":"2024-03-19T00:37:56.031881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341269+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:58.341269+0000","last_peered":"2024-03-19T00:36:58.341269+0000","last_clean":"2024-03-19T00:36:58.341269+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:58.341269+0000","last_undegraded":"2024-03-19T00:36:58.341269+0000","last_fullsized":"2024-03-19T00:36:58.341269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018797+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:58.018797+0000","last_peered":"2024-03-19T00:36:58.018797+0000","last_clean":"2024-03-19T00:36:58.018797+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:58.018797+0000","last_undegraded":"2024-03-19T00:36:58.018797+0000","last_fullsized":"2024-03-19T00:36:58.018797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031169+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:37:56.031169+0000","last_peered":"2024-03-19T00:37:56.031169+0000","last_clean":"2024-03-19T00:37:56.031169+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:37:56.031169+0000","last_undegraded":"2024-03-19T00:37:56.031169+0000","last_fullsized":"2024-03-19T00:37:56.031169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028083+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:37:56.028083+0000","last_peered":"2024-03-19T00:37:56.028083+0000","last_clean":"2024-03-19T00:37:56.028083+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:37:56.028083+0000","last_undegraded":"2024-03-19T00:37:56.028083+0000","last_fullsized":"2024-03-19T00:37:56.028083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.023829+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:37:56.023829+0000","last_peered":"2024-03-19T00:37:56.023829+0000","last_clean":"2024-03-19T00:37:56.023829+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:37:56.023829+0000","last_undegraded":"2024-03-19T00:37:56.023829+0000","last_fullsized":"2024-03-19T00:37:56.023829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040636+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:37:56.040636+0000","last_peered":"2024-03-19T00:37:56.040636+0000","last_clean":"2024-03-19T00:37:56.040636+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:37:56.040636+0000","last_undegraded":"2024-03-19T00:37:56.040636+0000","last_fullsized":"2024-03-19T00:37:56.040636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041821+0000","last_change":"2024-03-19T00:36:58.019535+0000","last_active":"2024-03-19T00:37:56.041821+0000","last_peered":"2024-03-19T00:37:56.041821+0000","last_clean":"2024-03-19T00:37:56.041821+0000","last_became_active":"2024-03-19T00:36:58.019220+0000","last_became_peered":"2024-03-19T00:36:58.019220+0000","last_unstale":"2024-03-19T00:37:56.041821+0000","last_undegraded":"2024-03-19T00:37:56.041821+0000","last_fullsized":"2024-03-19T00:37:56.041821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:04:51.797603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044655+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:37:56.044655+0000","last_peered":"2024-03-19T00:37:56.044655+0000","last_clean":"2024-03-19T00:37:56.044655+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:37:56.044655+0000","last_undegraded":"2024-03-19T00:37:56.044655+0000","last_fullsized":"2024-03-19T00:37:56.044655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.024108+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:37:56.024108+0000","last_peered":"2024-03-19T00:37:56.024108+0000","last_clean":"2024-03-19T00:37:56.024108+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:37:56.024108+0000","last_undegraded":"2024-03-19T00:37:56.024108+0000","last_fullsized":"2024-03-19T00:37:56.024108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042382+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:37:56.042382+0000","last_peered":"2024-03-19T00:37:56.042382+0000","last_clean":"2024-03-19T00:37:56.042382+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:37:56.042382+0000","last_undegraded":"2024-03-19T00:37:56.042382+0000","last_fullsized":"2024-03-19T00:37:56.042382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031736+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:37:56.031736+0000","last_peered":"2024-03-19T00:37:56.031736+0000","last_clean":"2024-03-19T00:37:56.031736+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:37:56.031736+0000","last_undegraded":"2024-03-19T00:37:56.031736+0000","last_fullsized":"2024-03-19T00:37:56.031736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.025787+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:37:56.025787+0000","last_peered":"2024-03-19T00:37:56.025787+0000","last_clean":"2024-03-19T00:37:56.025787+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:37:56.025787+0000","last_undegraded":"2024-03-19T00:37:56.025787+0000","last_fullsized":"2024-03-19T00:37:56.025787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.032629+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:37:56.032629+0000","last_peered":"2024-03-19T00:37:56.032629+0000","last_clean":"2024-03-19T00:37:56.032629+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:37:56.032629+0000","last_undegraded":"2024-03-19T00:37:56.032629+0000","last_fullsized":"2024-03-19T00:37:56.032629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.043776+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:37:56.043776+0000","last_peered":"2024-03-19T00:37:56.043776+0000","last_clean":"2024-03-19T00:37:56.043776+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:37:56.043776+0000","last_undegraded":"2024-03-19T00:37:56.043776+0000","last_fullsized":"2024-03-19T00:37:56.043776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.045485+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:37:56.045485+0000","last_peered":"2024-03-19T00:37:56.045485+0000","last_clean":"2024-03-19T00:37:56.045485+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:37:56.045485+0000","last_undegraded":"2024-03-19T00:37:56.045485+0000","last_fullsized":"2024-03-19T00:37:56.045485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029358+0000","last_change":"2024-03-19T00:36:57.951256+0000","last_active":"2024-03-19T00:37:56.029358+0000","last_peered":"2024-03-19T00:37:56.029358+0000","last_clean":"2024-03-19T00:37:56.029358+0000","last_became_active":"2024-03-19T00:36:57.950879+0000","last_became_peered":"2024-03-19T00:36:57.950879+0000","last_unstale":"2024-03-19T00:37:56.029358+0000","last_undegraded":"2024-03-19T00:37:56.029358+0000","last_fullsized":"2024-03-19T00:37:56.029358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:26.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69762,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":446,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":214773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1153209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":500971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1099323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":564942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":561625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1092609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:38:28.482 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-19T00:38:28.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 -- 172.21.15.7:0/3552038033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc102370 msgr2=0x7f95cc108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 --2- 172.21.15.7:0/3552038033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc102370 0x7f95cc108c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009be0 tx=0x7f95c402bef0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 -- 172.21.15.7:0/3552038033 shutdown_connections 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 --2- 172.21.15.7:0/3552038033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc102370 0x7f95cc108c00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 --2- 172.21.15.7:0/3552038033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc1019b0 0x7f95cc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 --2- 172.21.15.7:0/3552038033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc1007c0 0x7f95cc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 -- 172.21.15.7:0/3552038033 >> 172.21.15.7:0/3552038033 conn(0x7f95cc0fbd50 msgr2=0x7f95cc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 -- 172.21.15.7:0/3552038033 shutdown_connections 2024-03-19T00:38:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.618+0000 7f95d0f44640 1 -- 172.21.15.7:0/3552038033 wait complete. 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 Processor -- start 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 -- start start 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 0x7f95cc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc1019b0 0x7f95cc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc102370 0x7f95cc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc10d360 con 0x7f95cc102370 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc10d1e0 con 0x7f95cc1007c0 2024-03-19T00:38:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95d0f44640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc10d4e0 con 0x7f95cc1019b0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 0x7f95cc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95cad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc102370 0x7f95cc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 0x7f95cc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54598/0 (socket says 172.21.15.7:54598) 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 -- 172.21.15.7:0/914965165 learned_addr learned my addr 172.21.15.7:0/914965165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95c9d9b640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc1019b0 0x7f95cc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 -- 172.21.15.7:0/914965165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc1019b0 msgr2=0x7f95cc197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc1019b0 0x7f95cc197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 -- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc102370 msgr2=0x7f95cc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc102370 0x7f95cc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 -- 172.21.15.7:0/914965165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc19f620 con 0x7f95cc1007c0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95cad9d640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc102370 0x7f95cc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.619+0000 7f95ca59c640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 0x7f95cc197a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.620+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00ccc0 con 0x7f95cc1007c0 2024-03-19T00:38:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.620+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95bc0035f0 con 0x7f95cc1007c0 2024-03-19T00:38:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.620+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc19f910 con 0x7f95cc1007c0 2024-03-19T00:38:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.620+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00ce60 con 0x7f95cc1007c0 2024-03-19T00:38:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.620+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc19fe60 con 0x7f95cc1007c0 2024-03-19T00:38:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.621+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc068af0 con 0x7f95cc1007c0 2024-03-19T00:38:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.622+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f95bc0037a0 con 0x7f95cc1007c0 2024-03-19T00:38:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.622+0000 7f95bb7fe640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95b40a2890 0x7f95b40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.622+0000 7f95c9d9b640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95b40a2890 0x7f95b40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.622+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f95bc0cf1b0 con 0x7f95cc1007c0 2024-03-19T00:38:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.623+0000 7f95c9d9b640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95b40a2890 0x7f95b40a4d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f95cc198f10 tx=0x7f95c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.626+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f95bc09bd90 con 0x7f95cc1007c0 2024-03-19T00:38:28.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.806+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 --> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95cc061c60 con 0x7f95b40a2890 2024-03-19T00:38:28.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.815+0000 7f95bb7fe640 1 -- 172.21.15.7:0/914965165 <== mgr.14152 v2:172.21.15.7:6800/1790452807 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340481 (secure 0 0 0) 0x7f95cc061c60 con 0x7f95b40a2890 2024-03-19T00:38:28.821 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-19T00:38:28.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95b40a2890 msgr2=0x7f95b40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95b40a2890 0x7f95b40a4d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f95cc198f10 tx=0x7f95c0005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 msgr2=0x7f95cc197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 0x7f95cc197a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 shutdown_connections 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f95b40a2890 0x7f95b40a4d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95cc102370 0x7f95cc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc1019b0 0x7f95cc197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 --2- 172.21.15.7:0/914965165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95cc1007c0 0x7f95cc197a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 >> 172.21.15.7:0/914965165 conn(0x7f95cc0fbd50 msgr2=0x7f95cc104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 shutdown_connections 2024-03-19T00:38:28.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.819+0000 7f95d0f44640 1 -- 172.21.15.7:0/914965165 wait complete. 2024-03-19T00:38:28.832 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-03-19T00:38:28.038034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660364,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":867,"num_read_kb":555,"num_write":366,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404852,"kb_used_data":14100,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536972,"statfs":{"total":1151940427776,"available":1151525859328,"internally_reserved":0,"allocated":14438400,"data_stored":7970807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1482,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":32,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029289+0000","last_change":"2024-03-19T00:36:00.218949+0000","last_active":"2024-03-19T00:37:56.029289+0000","last_peered":"2024-03-19T00:37:56.029289+0000","last_clean":"2024-03-19T00:37:56.029289+0000","last_became_active":"2024-03-19T00:36:00.218204+0000","last_became_peered":"2024-03-19T00:36:00.218204+0000","last_unstale":"2024-03-19T00:37:56.029289+0000","last_undegraded":"2024-03-19T00:37:56.029289+0000","last_fullsized":"2024-03-19T00:37:56.029289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039878+0000","last_change":"2024-03-19T00:36:00.216763+0000","last_active":"2024-03-19T00:37:56.039878+0000","last_peered":"2024-03-19T00:37:56.039878+0000","last_clean":"2024-03-19T00:37:56.039878+0000","last_became_active":"2024-03-19T00:36:00.216413+0000","last_became_peered":"2024-03-19T00:36:00.216413+0000","last_unstale":"2024-03-19T00:37:56.039878+0000","last_undegraded":"2024-03-19T00:37:56.039878+0000","last_fullsized":"2024-03-19T00:37:56.039878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:48.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042753+0000","last_change":"2024-03-19T00:36:00.224944+0000","last_active":"2024-03-19T00:37:56.042753+0000","last_peered":"2024-03-19T00:37:56.042753+0000","last_clean":"2024-03-19T00:37:56.042753+0000","last_became_active":"2024-03-19T00:36:00.223698+0000","last_became_peered":"2024-03-19T00:36:00.223698+0000","last_unstale":"2024-03-19T00:37:56.042753+0000","last_undegraded":"2024-03-19T00:37:56.042753+0000","last_fullsized":"2024-03-19T00:37:56.042753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:32:08.687419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.019845+0000","last_change":"2024-03-19T00:36:00.219914+0000","last_active":"2024-03-19T00:37:56.019845+0000","last_peered":"2024-03-19T00:37:56.019845+0000","last_clean":"2024-03-19T00:37:56.019845+0000","last_became_active":"2024-03-19T00:36:00.219592+0000","last_became_peered":"2024-03-19T00:36:00.219592+0000","last_unstale":"2024-03-19T00:37:56.019845+0000","last_undegraded":"2024-03-19T00:37:56.019845+0000","last_fullsized":"2024-03-19T00:37:56.019845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:42:43.174917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.017353+0000","last_change":"2024-03-19T00:36:00.210066+0000","last_active":"2024-03-19T00:37:56.017353+0000","last_peered":"2024-03-19T00:37:56.017353+0000","last_clean":"2024-03-19T00:37:56.017353+0000","last_became_active":"2024-03-19T00:36:00.209743+0000","last_became_peered":"2024-03-19T00:36:00.209743+0000","last_unstale":"2024-03-19T00:37:56.017353+0000","last_undegraded":"2024-03-19T00:37:56.017353+0000","last_fullsized":"2024-03-19T00:37:56.017353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:30.364729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029411+0000","last_change":"2024-03-19T00:36:00.214392+0000","last_active":"2024-03-19T00:37:56.029411+0000","last_peered":"2024-03-19T00:37:56.029411+0000","last_clean":"2024-03-19T00:37:56.029411+0000","last_became_active":"2024-03-19T00:36:00.212107+0000","last_became_peered":"2024-03-19T00:36:00.212107+0000","last_unstale":"2024-03-19T00:37:56.029411+0000","last_undegraded":"2024-03-19T00:37:56.029411+0000","last_fullsized":"2024-03-19T00:37:56.029411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:43.146316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.022473+0000","last_change":"2024-03-19T00:36:00.219684+0000","last_active":"2024-03-19T00:37:56.022473+0000","last_peered":"2024-03-19T00:37:56.022473+0000","last_clean":"2024-03-19T00:37:56.022473+0000","last_became_active":"2024-03-19T00:36:00.219363+0000","last_became_peered":"2024-03-19T00:36:00.219363+0000","last_unstale":"2024-03-19T00:37:56.022473+0000","last_undegraded":"2024-03-19T00:37:56.022473+0000","last_fullsized":"2024-03-19T00:37:56.022473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:16:11.878499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.034359+0000","last_change":"2024-03-19T00:36:00.208335+0000","last_active":"2024-03-19T00:37:56.034359+0000","last_peered":"2024-03-19T00:37:56.034359+0000","last_clean":"2024-03-19T00:37:56.034359+0000","last_became_active":"2024-03-19T00:36:00.207988+0000","last_became_peered":"2024-03-19T00:36:00.207988+0000","last_unstale":"2024-03-19T00:37:56.034359+0000","last_undegraded":"2024-03-19T00:37:56.034359+0000","last_fullsized":"2024-03-19T00:37:56.034359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:28.486972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041752+0000","last_change":"2024-03-19T00:36:00.209241+0000","last_active":"2024-03-19T00:37:56.041752+0000","last_peered":"2024-03-19T00:37:56.041752+0000","last_clean":"2024-03-19T00:37:56.041752+0000","last_became_active":"2024-03-19T00:36:00.208933+0000","last_became_peered":"2024-03-19T00:36:00.208933+0000","last_unstale":"2024-03-19T00:37:56.041752+0000","last_undegraded":"2024-03-19T00:37:56.041752+0000","last_fullsized":"2024-03-19T00:37:56.041752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:17.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.035497+0000","last_change":"2024-03-19T00:36:00.214597+0000","last_active":"2024-03-19T00:37:56.035497+0000","last_peered":"2024-03-19T00:37:56.035497+0000","last_clean":"2024-03-19T00:37:56.035497+0000","last_became_active":"2024-03-19T00:36:00.212324+0000","last_became_peered":"2024-03-19T00:36:00.212324+0000","last_unstale":"2024-03-19T00:37:56.035497+0000","last_undegraded":"2024-03-19T00:37:56.035497+0000","last_fullsized":"2024-03-19T00:37:56.035497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:49:51.146255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040785+0000","last_change":"2024-03-19T00:36:00.217915+0000","last_active":"2024-03-19T00:37:56.040785+0000","last_peered":"2024-03-19T00:37:56.040785+0000","last_clean":"2024-03-19T00:37:56.040785+0000","last_became_active":"2024-03-19T00:36:00.217579+0000","last_became_peered":"2024-03-19T00:36:00.217579+0000","last_unstale":"2024-03-19T00:37:56.040785+0000","last_undegraded":"2024-03-19T00:37:56.040785+0000","last_fullsized":"2024-03-19T00:37:56.040785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:20.314797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031924+0000","last_change":"2024-03-19T00:36:58.297533+0000","last_active":"2024-03-19T00:37:56.031924+0000","last_peered":"2024-03-19T00:37:56.031924+0000","last_clean":"2024-03-19T00:37:56.031924+0000","last_became_active":"2024-03-19T00:36:58.297218+0000","last_became_peered":"2024-03-19T00:36:58.297218+0000","last_unstale":"2024-03-19T00:37:56.031924+0000","last_undegraded":"2024-03-19T00:37:56.031924+0000","last_fullsized":"2024-03-19T00:37:56.031924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:06.256765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039054+0000","last_change":"2024-03-19T00:36:00.224036+0000","last_active":"2024-03-19T00:37:56.039054+0000","last_peered":"2024-03-19T00:37:56.039054+0000","last_clean":"2024-03-19T00:37:56.039054+0000","last_became_active":"2024-03-19T00:36:00.223714+0000","last_became_peered":"2024-03-19T00:36:00.223714+0000","last_unstale":"2024-03-19T00:37:56.039054+0000","last_undegraded":"2024-03-19T00:37:56.039054+0000","last_fullsized":"2024-03-19T00:37:56.039054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:52:35.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039900+0000","last_change":"2024-03-19T00:36:00.216507+0000","last_active":"2024-03-19T00:37:56.039900+0000","last_peered":"2024-03-19T00:37:56.039900+0000","last_clean":"2024-03-19T00:37:56.039900+0000","last_became_active":"2024-03-19T00:36:00.216120+0000","last_became_peered":"2024-03-19T00:36:00.216120+0000","last_unstale":"2024-03-19T00:37:56.039900+0000","last_undegraded":"2024-03-19T00:37:56.039900+0000","last_fullsized":"2024-03-19T00:37:56.039900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:27.176281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042705+0000","last_change":"2024-03-19T00:36:00.221837+0000","last_active":"2024-03-19T00:37:56.042705+0000","last_peered":"2024-03-19T00:37:56.042705+0000","last_clean":"2024-03-19T00:37:56.042705+0000","last_became_active":"2024-03-19T00:36:00.221440+0000","last_became_peered":"2024-03-19T00:36:00.221440+0000","last_unstale":"2024-03-19T00:37:56.042705+0000","last_undegraded":"2024-03-19T00:37:56.042705+0000","last_fullsized":"2024-03-19T00:37:56.042705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:12.049261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042051+0000","last_change":"2024-03-19T00:36:00.218744+0000","last_active":"2024-03-19T00:37:56.042051+0000","last_peered":"2024-03-19T00:37:56.042051+0000","last_clean":"2024-03-19T00:37:56.042051+0000","last_became_active":"2024-03-19T00:36:00.218107+0000","last_became_peered":"2024-03-19T00:36:00.218107+0000","last_unstale":"2024-03-19T00:37:56.042051+0000","last_undegraded":"2024-03-19T00:37:56.042051+0000","last_fullsized":"2024-03-19T00:37:56.042051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:10.660075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029319+0000","last_change":"2024-03-19T00:36:00.219211+0000","last_active":"2024-03-19T00:37:56.029319+0000","last_peered":"2024-03-19T00:37:56.029319+0000","last_clean":"2024-03-19T00:37:56.029319+0000","last_became_active":"2024-03-19T00:36:00.218629+0000","last_became_peered":"2024-03-19T00:36:00.218629+0000","last_unstale":"2024-03-19T00:37:56.029319+0000","last_undegraded":"2024-03-19T00:37:56.029319+0000","last_fullsized":"2024-03-19T00:37:56.029319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:54.906533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.032743+0000","last_change":"2024-03-19T00:36:00.211725+0000","last_active":"2024-03-19T00:37:56.032743+0000","last_peered":"2024-03-19T00:37:56.032743+0000","last_clean":"2024-03-19T00:37:56.032743+0000","last_became_active":"2024-03-19T00:36:00.211397+0000","last_became_peered":"2024-03-19T00:36:00.211397+0000","last_unstale":"2024-03-19T00:37:56.032743+0000","last_undegraded":"2024-03-19T00:37:56.032743+0000","last_fullsized":"2024-03-19T00:37:56.032743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:46:19.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.030215+0000","last_change":"2024-03-19T00:36:00.213981+0000","last_active":"2024-03-19T00:37:56.030215+0000","last_peered":"2024-03-19T00:37:56.030215+0000","last_clean":"2024-03-19T00:37:56.030215+0000","last_became_active":"2024-03-19T00:36:00.212476+0000","last_became_peered":"2024-03-19T00:36:00.212476+0000","last_unstale":"2024-03-19T00:37:56.030215+0000","last_undegraded":"2024-03-19T00:37:56.030215+0000","last_fullsized":"2024-03-19T00:37:56.030215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:26.681101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944493+0000","last_change":"2024-03-19T00:36:04.251355+0000","last_active":"2024-03-19T00:36:57.944493+0000","last_peered":"2024-03-19T00:36:57.944493+0000","last_clean":"2024-03-19T00:36:57.944493+0000","last_became_active":"2024-03-19T00:36:04.250925+0000","last_became_peered":"2024-03-19T00:36:04.250925+0000","last_unstale":"2024-03-19T00:36:57.944493+0000","last_undegraded":"2024-03-19T00:36:57.944493+0000","last_fullsized":"2024-03-19T00:36:57.944493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:57:54.858890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943877+0000","last_change":"2024-03-19T00:35:59.210770+0000","last_active":"2024-03-19T00:36:57.943877+0000","last_peered":"2024-03-19T00:36:57.943877+0000","last_clean":"2024-03-19T00:36:57.943877+0000","last_became_active":"2024-03-19T00:35:59.210366+0000","last_became_peered":"2024-03-19T00:35:59.210366+0000","last_unstale":"2024-03-19T00:36:57.943877+0000","last_undegraded":"2024-03-19T00:36:57.943877+0000","last_fullsized":"2024-03-19T00:36:57.943877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:25.001114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028440+0000","last_change":"2024-03-19T00:36:00.208633+0000","last_active":"2024-03-19T00:37:56.028440+0000","last_peered":"2024-03-19T00:37:56.028440+0000","last_clean":"2024-03-19T00:37:56.028440+0000","last_became_active":"2024-03-19T00:36:00.207922+0000","last_became_peered":"2024-03-19T00:36:00.207922+0000","last_unstale":"2024-03-19T00:37:56.028440+0000","last_undegraded":"2024-03-19T00:37:56.028440+0000","last_fullsized":"2024-03-19T00:37:56.028440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:27:28.304339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943659+0000","last_change":"2024-03-19T00:36:04.246964+0000","last_active":"2024-03-19T00:36:57.943659+0000","last_peered":"2024-03-19T00:36:57.943659+0000","last_clean":"2024-03-19T00:36:57.943659+0000","last_became_active":"2024-03-19T00:36:04.246589+0000","last_became_peered":"2024-03-19T00:36:04.246589+0000","last_unstale":"2024-03-19T00:36:57.943659+0000","last_undegraded":"2024-03-19T00:36:57.943659+0000","last_fullsized":"2024-03-19T00:36:57.943659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:37.465066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947548+0000","last_change":"2024-03-19T00:35:59.211356+0000","last_active":"2024-03-19T00:36:57.947548+0000","last_peered":"2024-03-19T00:36:57.947548+0000","last_clean":"2024-03-19T00:36:57.947548+0000","last_became_active":"2024-03-19T00:35:59.211035+0000","last_became_peered":"2024-03-19T00:35:59.211035+0000","last_unstale":"2024-03-19T00:36:57.947548+0000","last_undegraded":"2024-03-19T00:36:57.947548+0000","last_fullsized":"2024-03-19T00:36:57.947548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:53.403587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044072+0000","last_change":"2024-03-19T00:36:00.223186+0000","last_active":"2024-03-19T00:37:56.044072+0000","last_peered":"2024-03-19T00:37:56.044072+0000","last_clean":"2024-03-19T00:37:56.044072+0000","last_became_active":"2024-03-19T00:36:00.222646+0000","last_became_peered":"2024-03-19T00:36:00.222646+0000","last_unstale":"2024-03-19T00:37:56.044072+0000","last_undegraded":"2024-03-19T00:37:56.044072+0000","last_fullsized":"2024-03-19T00:37:56.044072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:29.585764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943869+0000","last_change":"2024-03-19T00:36:04.249491+0000","last_active":"2024-03-19T00:36:57.943869+0000","last_peered":"2024-03-19T00:36:57.943869+0000","last_clean":"2024-03-19T00:36:57.943869+0000","last_became_active":"2024-03-19T00:36:04.249147+0000","last_became_peered":"2024-03-19T00:36:04.249147+0000","last_unstale":"2024-03-19T00:36:57.943869+0000","last_undegraded":"2024-03-19T00:36:57.943869+0000","last_fullsized":"2024-03-19T00:36:57.943869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:50:25.396793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947966+0000","last_change":"2024-03-19T00:35:59.214643+0000","last_active":"2024-03-19T00:36:57.947966+0000","last_peered":"2024-03-19T00:36:57.947966+0000","last_clean":"2024-03-19T00:36:57.947966+0000","last_became_active":"2024-03-19T00:35:59.214326+0000","last_became_peered":"2024-03-19T00:35:59.214326+0000","last_unstale":"2024-03-19T00:36:57.947966+0000","last_undegraded":"2024-03-19T00:36:57.947966+0000","last_fullsized":"2024-03-19T00:36:57.947966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:56.992293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044222+0000","last_change":"2024-03-19T00:36:00.217490+0000","last_active":"2024-03-19T00:37:56.044222+0000","last_peered":"2024-03-19T00:37:56.044222+0000","last_clean":"2024-03-19T00:37:56.044222+0000","last_became_active":"2024-03-19T00:36:00.217161+0000","last_became_peered":"2024-03-19T00:36:00.217161+0000","last_unstale":"2024-03-19T00:37:56.044222+0000","last_undegraded":"2024-03-19T00:37:56.044222+0000","last_fullsized":"2024-03-19T00:37:56.044222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:43.743065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943744+0000","last_change":"2024-03-19T00:36:04.244757+0000","last_active":"2024-03-19T00:36:57.943744+0000","last_peered":"2024-03-19T00:36:57.943744+0000","last_clean":"2024-03-19T00:36:57.943744+0000","last_became_active":"2024-03-19T00:36:04.244060+0000","last_became_peered":"2024-03-19T00:36:04.244060+0000","last_unstale":"2024-03-19T00:36:57.943744+0000","last_undegraded":"2024-03-19T00:36:57.943744+0000","last_fullsized":"2024-03-19T00:36:57.943744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:00:09.193379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948143+0000","last_change":"2024-03-19T00:35:59.210617+0000","last_active":"2024-03-19T00:36:57.948143+0000","last_peered":"2024-03-19T00:36:57.948143+0000","last_clean":"2024-03-19T00:36:57.948143+0000","last_became_active":"2024-03-19T00:35:59.210246+0000","last_became_peered":"2024-03-19T00:35:59.210246+0000","last_unstale":"2024-03-19T00:36:57.948143+0000","last_undegraded":"2024-03-19T00:36:57.948143+0000","last_fullsized":"2024-03-19T00:36:57.948143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:44:44.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042138+0000","last_change":"2024-03-19T00:36:57.948938+0000","last_active":"2024-03-19T00:37:56.042138+0000","last_peered":"2024-03-19T00:37:56.042138+0000","last_clean":"2024-03-19T00:37:56.042138+0000","last_became_active":"2024-03-19T00:36:57.948519+0000","last_became_peered":"2024-03-19T00:36:57.948519+0000","last_unstale":"2024-03-19T00:37:56.042138+0000","last_undegraded":"2024-03-19T00:37:56.042138+0000","last_fullsized":"2024-03-19T00:37:56.042138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:33:49.099555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.298764+0000","last_change":"2024-03-19T00:36:58.298764+0000","last_active":"2024-03-19T00:36:58.298764+0000","last_peered":"2024-03-19T00:36:58.298764+0000","last_clean":"2024-03-19T00:36:58.298764+0000","last_became_active":"2024-03-19T00:36:58.298265+0000","last_became_peered":"2024-03-19T00:36:58.298265+0000","last_unstale":"2024-03-19T00:36:58.298764+0000","last_undegraded":"2024-03-19T00:36:58.298764+0000","last_fullsized":"2024-03-19T00:36:58.298764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:48:45.983861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:22.428477+0000","last_change":"2024-03-19T00:35:59.210469+0000","last_active":"2024-03-19T00:38:22.428477+0000","last_peered":"2024-03-19T00:38:22.428477+0000","last_clean":"2024-03-19T00:38:22.428477+0000","last_became_active":"2024-03-19T00:35:59.210133+0000","last_became_peered":"2024-03-19T00:35:59.210133+0000","last_unstale":"2024-03-19T00:38:22.428477+0000","last_undegraded":"2024-03-19T00:38:22.428477+0000","last_fullsized":"2024-03-19T00:38:22.428477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:15.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1648,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.014746+0000","last_change":"2024-03-19T00:36:00.203962+0000","last_active":"2024-03-19T00:37:56.014746+0000","last_peered":"2024-03-19T00:37:56.014746+0000","last_clean":"2024-03-19T00:37:56.014746+0000","last_became_active":"2024-03-19T00:36:00.203617+0000","last_became_peered":"2024-03-19T00:36:00.203617+0000","last_unstale":"2024-03-19T00:37:56.014746+0000","last_undegraded":"2024-03-19T00:37:56.014746+0000","last_fullsized":"2024-03-19T00:37:56.014746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:12.880023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342937+0000","last_change":"2024-03-19T00:36:58.342937+0000","last_active":"2024-03-19T00:36:58.342937+0000","last_peered":"2024-03-19T00:36:58.342937+0000","last_clean":"2024-03-19T00:36:58.342937+0000","last_became_active":"2024-03-19T00:36:58.342474+0000","last_became_peered":"2024-03-19T00:36:58.342474+0000","last_unstale":"2024-03-19T00:36:58.342937+0000","last_undegraded":"2024-03-19T00:36:58.342937+0000","last_fullsized":"2024-03-19T00:36:58.342937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:55.531546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.297043+0000","last_change":"2024-03-19T00:35:59.210849+0000","last_active":"2024-03-19T00:36:58.297043+0000","last_peered":"2024-03-19T00:36:58.297043+0000","last_clean":"2024-03-19T00:36:58.297043+0000","last_became_active":"2024-03-19T00:35:59.210497+0000","last_became_peered":"2024-03-19T00:35:59.210497+0000","last_unstale":"2024-03-19T00:36:58.297043+0000","last_undegraded":"2024-03-19T00:36:58.297043+0000","last_fullsized":"2024-03-19T00:36:58.297043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:08:48.627812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.027675+0000","last_change":"2024-03-19T00:36:00.213418+0000","last_active":"2024-03-19T00:37:56.027675+0000","last_peered":"2024-03-19T00:37:56.027675+0000","last_clean":"2024-03-19T00:37:56.027675+0000","last_became_active":"2024-03-19T00:36:00.213051+0000","last_became_peered":"2024-03-19T00:36:00.213051+0000","last_unstale":"2024-03-19T00:37:56.027675+0000","last_undegraded":"2024-03-19T00:37:56.027675+0000","last_fullsized":"2024-03-19T00:37:56.027675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:27:20.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.951395+0000","last_change":"2024-03-19T00:36:57.950687+0000","last_active":"2024-03-19T00:36:57.951395+0000","last_peered":"2024-03-19T00:36:57.951395+0000","last_clean":"2024-03-19T00:36:57.951395+0000","last_became_active":"2024-03-19T00:36:57.950151+0000","last_became_peered":"2024-03-19T00:36:57.950151+0000","last_unstale":"2024-03-19T00:36:57.951395+0000","last_undegraded":"2024-03-19T00:36:57.951395+0000","last_fullsized":"2024-03-19T00:36:57.951395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:41.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018277+0000","last_change":"2024-03-19T00:35:59.215507+0000","last_active":"2024-03-19T00:36:58.018277+0000","last_peered":"2024-03-19T00:36:58.018277+0000","last_clean":"2024-03-19T00:36:58.018277+0000","last_became_active":"2024-03-19T00:35:59.215175+0000","last_became_peered":"2024-03-19T00:35:59.215175+0000","last_unstale":"2024-03-19T00:36:58.018277+0000","last_undegraded":"2024-03-19T00:36:58.018277+0000","last_fullsized":"2024-03-19T00:36:58.018277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:35:43.796787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033347+0000","last_change":"2024-03-19T00:36:00.211280+0000","last_active":"2024-03-19T00:37:56.033347+0000","last_peered":"2024-03-19T00:37:56.033347+0000","last_clean":"2024-03-19T00:37:56.033347+0000","last_became_active":"2024-03-19T00:36:00.210929+0000","last_became_peered":"2024-03-19T00:36:00.210929+0000","last_unstale":"2024-03-19T00:37:56.033347+0000","last_undegraded":"2024-03-19T00:37:56.033347+0000","last_fullsized":"2024-03-19T00:37:56.033347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:24.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947452+0000","last_change":"2024-03-19T00:36:04.244625+0000","last_active":"2024-03-19T00:36:57.947452+0000","last_peered":"2024-03-19T00:36:57.947452+0000","last_clean":"2024-03-19T00:36:57.947452+0000","last_became_active":"2024-03-19T00:36:04.244224+0000","last_became_peered":"2024-03-19T00:36:04.244224+0000","last_unstale":"2024-03-19T00:36:57.947452+0000","last_undegraded":"2024-03-19T00:36:57.947452+0000","last_fullsized":"2024-03-19T00:36:57.947452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:56.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341398+0000","last_change":"2024-03-19T00:35:59.211701+0000","last_active":"2024-03-19T00:36:58.341398+0000","last_peered":"2024-03-19T00:36:58.341398+0000","last_clean":"2024-03-19T00:36:58.341398+0000","last_became_active":"2024-03-19T00:35:59.211409+0000","last_became_peered":"2024-03-19T00:35:59.211409+0000","last_unstale":"2024-03-19T00:36:58.341398+0000","last_undegraded":"2024-03-19T00:36:58.341398+0000","last_fullsized":"2024-03-19T00:36:58.341398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:42:54.326039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033250+0000","last_change":"2024-03-19T00:36:00.212310+0000","last_active":"2024-03-19T00:37:56.033250+0000","last_peered":"2024-03-19T00:37:56.033250+0000","last_clean":"2024-03-19T00:37:56.033250+0000","last_became_active":"2024-03-19T00:36:00.211922+0000","last_became_peered":"2024-03-19T00:36:00.211922+0000","last_unstale":"2024-03-19T00:37:56.033250+0000","last_undegraded":"2024-03-19T00:37:56.033250+0000","last_fullsized":"2024-03-19T00:37:56.033250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:47.090262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946524+0000","last_change":"2024-03-19T00:36:04.247150+0000","last_active":"2024-03-19T00:36:57.946524+0000","last_peered":"2024-03-19T00:36:57.946524+0000","last_clean":"2024-03-19T00:36:57.946524+0000","last_became_active":"2024-03-19T00:36:04.246522+0000","last_became_peered":"2024-03-19T00:36:04.246522+0000","last_unstale":"2024-03-19T00:36:57.946524+0000","last_undegraded":"2024-03-19T00:36:57.946524+0000","last_fullsized":"2024-03-19T00:36:57.946524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:09:52.129337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.429184+0000","last_change":"2024-03-19T00:35:59.210005+0000","last_active":"2024-03-19T00:38:23.429184+0000","last_peered":"2024-03-19T00:38:23.429184+0000","last_clean":"2024-03-19T00:38:23.429184+0000","last_became_active":"2024-03-19T00:35:59.209661+0000","last_became_peered":"2024-03-19T00:35:59.209661+0000","last_unstale":"2024-03-19T00:38:23.429184+0000","last_undegraded":"2024-03-19T00:38:23.429184+0000","last_fullsized":"2024-03-19T00:38:23.429184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:58:26.184415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044130+0000","last_change":"2024-03-19T00:36:00.207400+0000","last_active":"2024-03-19T00:37:56.044130+0000","last_peered":"2024-03-19T00:37:56.044130+0000","last_clean":"2024-03-19T00:37:56.044130+0000","last_became_active":"2024-03-19T00:36:00.207086+0000","last_became_peered":"2024-03-19T00:36:00.207086+0000","last_unstale":"2024-03-19T00:37:56.044130+0000","last_undegraded":"2024-03-19T00:37:56.044130+0000","last_fullsized":"2024-03-19T00:37:56.044130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:40.561764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943224+0000","last_change":"2024-03-19T00:36:04.245347+0000","last_active":"2024-03-19T00:36:57.943224+0000","last_peered":"2024-03-19T00:36:57.943224+0000","last_clean":"2024-03-19T00:36:57.943224+0000","last_became_active":"2024-03-19T00:36:04.244851+0000","last_became_peered":"2024-03-19T00:36:04.244851+0000","last_unstale":"2024-03-19T00:36:57.943224+0000","last_undegraded":"2024-03-19T00:36:57.943224+0000","last_fullsized":"2024-03-19T00:36:57.943224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:06.488682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092881+0000","last_change":"2024-03-19T00:35:59.208601+0000","last_active":"2024-03-19T00:36:58.092881+0000","last_peered":"2024-03-19T00:36:58.092881+0000","last_clean":"2024-03-19T00:36:58.092881+0000","last_became_active":"2024-03-19T00:35:59.208302+0000","last_became_peered":"2024-03-19T00:35:59.208302+0000","last_unstale":"2024-03-19T00:36:58.092881+0000","last_undegraded":"2024-03-19T00:36:58.092881+0000","last_fullsized":"2024-03-19T00:36:58.092881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:17:28.885192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.037121+0000","last_change":"2024-03-19T00:36:00.228917+0000","last_active":"2024-03-19T00:37:56.037121+0000","last_peered":"2024-03-19T00:37:56.037121+0000","last_clean":"2024-03-19T00:37:56.037121+0000","last_became_active":"2024-03-19T00:36:00.228002+0000","last_became_peered":"2024-03-19T00:36:00.228002+0000","last_unstale":"2024-03-19T00:37:56.037121+0000","last_undegraded":"2024-03-19T00:37:56.037121+0000","last_fullsized":"2024-03-19T00:37:56.037121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:23.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944132+0000","last_change":"2024-03-19T00:36:04.247187+0000","last_active":"2024-03-19T00:36:57.944132+0000","last_peered":"2024-03-19T00:36:57.944132+0000","last_clean":"2024-03-19T00:36:57.944132+0000","last_became_active":"2024-03-19T00:36:04.246759+0000","last_became_peered":"2024-03-19T00:36:04.246759+0000","last_unstale":"2024-03-19T00:36:57.944132+0000","last_undegraded":"2024-03-19T00:36:57.944132+0000","last_fullsized":"2024-03-19T00:36:57.944132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:37.357962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341884+0000","last_change":"2024-03-19T00:35:59.215583+0000","last_active":"2024-03-19T00:36:58.341884+0000","last_peered":"2024-03-19T00:36:58.341884+0000","last_clean":"2024-03-19T00:36:58.341884+0000","last_became_active":"2024-03-19T00:35:59.215235+0000","last_became_peered":"2024-03-19T00:35:59.215235+0000","last_unstale":"2024-03-19T00:36:58.341884+0000","last_undegraded":"2024-03-19T00:36:58.341884+0000","last_fullsized":"2024-03-19T00:36:58.341884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:03.013401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042790+0000","last_change":"2024-03-19T00:36:00.221007+0000","last_active":"2024-03-19T00:37:56.042790+0000","last_peered":"2024-03-19T00:37:56.042790+0000","last_clean":"2024-03-19T00:37:56.042790+0000","last_became_active":"2024-03-19T00:36:00.220694+0000","last_became_peered":"2024-03-19T00:36:00.220694+0000","last_unstale":"2024-03-19T00:37:56.042790+0000","last_undegraded":"2024-03-19T00:37:56.042790+0000","last_fullsized":"2024-03-19T00:37:56.042790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:33:43.140936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943327+0000","last_change":"2024-03-19T00:36:04.241753+0000","last_active":"2024-03-19T00:36:57.943327+0000","last_peered":"2024-03-19T00:36:57.943327+0000","last_clean":"2024-03-19T00:36:57.943327+0000","last_became_active":"2024-03-19T00:36:04.241340+0000","last_became_peered":"2024-03-19T00:36:04.241340+0000","last_unstale":"2024-03-19T00:36:57.943327+0000","last_undegraded":"2024-03-19T00:36:57.943327+0000","last_fullsized":"2024-03-19T00:36:57.943327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:31:46.607295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":312,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.429728+0000","last_change":"2024-03-19T00:35:59.212712+0000","last_active":"2024-03-19T00:38:23.429728+0000","last_peered":"2024-03-19T00:38:23.429728+0000","last_clean":"2024-03-19T00:38:23.429728+0000","last_became_active":"2024-03-19T00:35:59.212384+0000","last_became_peered":"2024-03-19T00:35:59.212384+0000","last_unstale":"2024-03-19T00:38:23.429728+0000","last_undegraded":"2024-03-19T00:38:23.429728+0000","last_fullsized":"2024-03-19T00:38:23.429728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:20:12.984898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":186,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.018651+0000","last_change":"2024-03-19T00:36:00.215379+0000","last_active":"2024-03-19T00:37:56.018651+0000","last_peered":"2024-03-19T00:37:56.018651+0000","last_clean":"2024-03-19T00:37:56.018651+0000","last_became_active":"2024-03-19T00:36:00.215064+0000","last_became_peered":"2024-03-19T00:36:00.215064+0000","last_unstale":"2024-03-19T00:37:56.018651+0000","last_undegraded":"2024-03-19T00:37:56.018651+0000","last_fullsized":"2024-03-19T00:37:56.018651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:32:59.130677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342057+0000","last_change":"2024-03-19T00:36:04.249112+0000","last_active":"2024-03-19T00:36:58.342057+0000","last_peered":"2024-03-19T00:36:58.342057+0000","last_clean":"2024-03-19T00:36:58.342057+0000","last_became_active":"2024-03-19T00:36:04.248766+0000","last_became_peered":"2024-03-19T00:36:04.248766+0000","last_unstale":"2024-03-19T00:36:58.342057+0000","last_undegraded":"2024-03-19T00:36:58.342057+0000","last_fullsized":"2024-03-19T00:36:58.342057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:28:08.370446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945086+0000","last_change":"2024-03-19T00:35:59.218879+0000","last_active":"2024-03-19T00:36:57.945086+0000","last_peered":"2024-03-19T00:36:57.945086+0000","last_clean":"2024-03-19T00:36:57.945086+0000","last_became_active":"2024-03-19T00:35:59.218329+0000","last_became_peered":"2024-03-19T00:35:59.218329+0000","last_unstale":"2024-03-19T00:36:57.945086+0000","last_undegraded":"2024-03-19T00:36:57.945086+0000","last_fullsized":"2024-03-19T00:36:57.945086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:20:05.196425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033786+0000","last_change":"2024-03-19T00:36:00.230603+0000","last_active":"2024-03-19T00:37:56.033786+0000","last_peered":"2024-03-19T00:37:56.033786+0000","last_clean":"2024-03-19T00:37:56.033786+0000","last_became_active":"2024-03-19T00:36:00.230290+0000","last_became_peered":"2024-03-19T00:36:00.230290+0000","last_unstale":"2024-03-19T00:37:56.033786+0000","last_undegraded":"2024-03-19T00:37:56.033786+0000","last_fullsized":"2024-03-19T00:37:56.033786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:46.271993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945403+0000","last_change":"2024-03-19T00:36:04.244953+0000","last_active":"2024-03-19T00:36:57.945403+0000","last_peered":"2024-03-19T00:36:57.945403+0000","last_clean":"2024-03-19T00:36:57.945403+0000","last_became_active":"2024-03-19T00:36:04.244514+0000","last_became_peered":"2024-03-19T00:36:04.244514+0000","last_unstale":"2024-03-19T00:36:57.945403+0000","last_undegraded":"2024-03-19T00:36:57.945403+0000","last_fullsized":"2024-03-19T00:36:57.945403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:28:05.118590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341889+0000","last_change":"2024-03-19T00:35:59.210784+0000","last_active":"2024-03-19T00:36:58.341889+0000","last_peered":"2024-03-19T00:36:58.341889+0000","last_clean":"2024-03-19T00:36:58.341889+0000","last_became_active":"2024-03-19T00:35:59.210416+0000","last_became_peered":"2024-03-19T00:35:59.210416+0000","last_unstale":"2024-03-19T00:36:58.341889+0000","last_undegraded":"2024-03-19T00:36:58.341889+0000","last_fullsized":"2024-03-19T00:36:58.341889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:18:55.462710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.035315+0000","last_change":"2024-03-19T00:36:00.220906+0000","last_active":"2024-03-19T00:37:56.035315+0000","last_peered":"2024-03-19T00:37:56.035315+0000","last_clean":"2024-03-19T00:37:56.035315+0000","last_became_active":"2024-03-19T00:36:00.220588+0000","last_became_peered":"2024-03-19T00:36:00.220588+0000","last_unstale":"2024-03-19T00:37:56.035315+0000","last_undegraded":"2024-03-19T00:37:56.035315+0000","last_fullsized":"2024-03-19T00:37:56.035315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:46.289240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943691+0000","last_change":"2024-03-19T00:36:04.249559+0000","last_active":"2024-03-19T00:36:57.943691+0000","last_peered":"2024-03-19T00:36:57.943691+0000","last_clean":"2024-03-19T00:36:57.943691+0000","last_became_active":"2024-03-19T00:36:04.249124+0000","last_became_peered":"2024-03-19T00:36:04.249124+0000","last_unstale":"2024-03-19T00:36:57.943691+0000","last_undegraded":"2024-03-19T00:36:57.943691+0000","last_fullsized":"2024-03-19T00:36:57.943691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:04:11.475520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.445021+0000","last_change":"2024-03-19T00:35:59.207618+0000","last_active":"2024-03-19T00:38:23.445021+0000","last_peered":"2024-03-19T00:38:23.445021+0000","last_clean":"2024-03-19T00:38:23.445021+0000","last_became_active":"2024-03-19T00:35:59.207284+0000","last_became_peered":"2024-03-19T00:35:59.207284+0000","last_unstale":"2024-03-19T00:38:23.445021+0000","last_undegraded":"2024-03-19T00:38:23.445021+0000","last_fullsized":"2024-03-19T00:38:23.445021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:59:55.487295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1614,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033957+0000","last_change":"2024-03-19T00:36:00.217728+0000","last_active":"2024-03-19T00:37:56.033957+0000","last_peered":"2024-03-19T00:37:56.033957+0000","last_clean":"2024-03-19T00:37:56.033957+0000","last_became_active":"2024-03-19T00:36:00.217431+0000","last_became_peered":"2024-03-19T00:36:00.217431+0000","last_unstale":"2024-03-19T00:37:56.033957+0000","last_undegraded":"2024-03-19T00:37:56.033957+0000","last_fullsized":"2024-03-19T00:37:56.033957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:36:07.809439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018782+0000","last_change":"2024-03-19T00:36:04.251803+0000","last_active":"2024-03-19T00:36:58.018782+0000","last_peered":"2024-03-19T00:36:58.018782+0000","last_clean":"2024-03-19T00:36:58.018782+0000","last_became_active":"2024-03-19T00:36:04.250959+0000","last_became_peered":"2024-03-19T00:36:04.250959+0000","last_unstale":"2024-03-19T00:36:58.018782+0000","last_undegraded":"2024-03-19T00:36:58.018782+0000","last_fullsized":"2024-03-19T00:36:58.018782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:09:27.311016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947702+0000","last_change":"2024-03-19T00:35:59.212287+0000","last_active":"2024-03-19T00:36:57.947702+0000","last_peered":"2024-03-19T00:36:57.947702+0000","last_clean":"2024-03-19T00:36:57.947702+0000","last_became_active":"2024-03-19T00:35:59.211715+0000","last_became_peered":"2024-03-19T00:35:59.211715+0000","last_unstale":"2024-03-19T00:36:57.947702+0000","last_undegraded":"2024-03-19T00:36:57.947702+0000","last_fullsized":"2024-03-19T00:36:57.947702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:31.034885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026513+0000","last_change":"2024-03-19T00:36:00.212497+0000","last_active":"2024-03-19T00:37:56.026513+0000","last_peered":"2024-03-19T00:37:56.026513+0000","last_clean":"2024-03-19T00:37:56.026513+0000","last_became_active":"2024-03-19T00:36:00.212179+0000","last_became_peered":"2024-03-19T00:36:00.212179+0000","last_unstale":"2024-03-19T00:37:56.026513+0000","last_undegraded":"2024-03-19T00:37:56.026513+0000","last_fullsized":"2024-03-19T00:37:56.026513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:44:53.854548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341751+0000","last_change":"2024-03-19T00:36:04.243285+0000","last_active":"2024-03-19T00:36:58.341751+0000","last_peered":"2024-03-19T00:36:58.341751+0000","last_clean":"2024-03-19T00:36:58.341751+0000","last_became_active":"2024-03-19T00:36:04.242980+0000","last_became_peered":"2024-03-19T00:36:04.242980+0000","last_unstale":"2024-03-19T00:36:58.341751+0000","last_undegraded":"2024-03-19T00:36:58.341751+0000","last_fullsized":"2024-03-19T00:36:58.341751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:10:36.650725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948779+0000","last_change":"2024-03-19T00:35:59.211769+0000","last_active":"2024-03-19T00:36:57.948779+0000","last_peered":"2024-03-19T00:36:57.948779+0000","last_clean":"2024-03-19T00:36:57.948779+0000","last_became_active":"2024-03-19T00:35:59.211457+0000","last_became_peered":"2024-03-19T00:35:59.211457+0000","last_unstale":"2024-03-19T00:36:57.948779+0000","last_undegraded":"2024-03-19T00:36:57.948779+0000","last_fullsized":"2024-03-19T00:36:57.948779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:28:06.760340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.024437+0000","last_change":"2024-03-19T00:36:00.222727+0000","last_active":"2024-03-19T00:37:56.024437+0000","last_peered":"2024-03-19T00:37:56.024437+0000","last_clean":"2024-03-19T00:37:56.024437+0000","last_became_active":"2024-03-19T00:36:00.222406+0000","last_became_peered":"2024-03-19T00:36:00.222406+0000","last_unstale":"2024-03-19T00:37:56.024437+0000","last_undegraded":"2024-03-19T00:37:56.024437+0000","last_fullsized":"2024-03-19T00:37:56.024437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:14.924871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947891+0000","last_change":"2024-03-19T00:36:04.250532+0000","last_active":"2024-03-19T00:36:57.947891+0000","last_peered":"2024-03-19T00:36:57.947891+0000","last_clean":"2024-03-19T00:36:57.947891+0000","last_became_active":"2024-03-19T00:36:04.250063+0000","last_became_peered":"2024-03-19T00:36:04.250063+0000","last_unstale":"2024-03-19T00:36:57.947891+0000","last_undegraded":"2024-03-19T00:36:57.947891+0000","last_fullsized":"2024-03-19T00:36:57.947891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:03.732394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943536+0000","last_change":"2024-03-19T00:35:59.204383+0000","last_active":"2024-03-19T00:36:57.943536+0000","last_peered":"2024-03-19T00:36:57.943536+0000","last_clean":"2024-03-19T00:36:57.943536+0000","last_became_active":"2024-03-19T00:35:59.204057+0000","last_became_peered":"2024-03-19T00:35:59.204057+0000","last_unstale":"2024-03-19T00:36:57.943536+0000","last_undegraded":"2024-03-19T00:36:57.943536+0000","last_fullsized":"2024-03-19T00:36:57.943536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:53.922523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.037115+0000","last_change":"2024-03-19T00:36:00.211878+0000","last_active":"2024-03-19T00:37:56.037115+0000","last_peered":"2024-03-19T00:37:56.037115+0000","last_clean":"2024-03-19T00:37:56.037115+0000","last_became_active":"2024-03-19T00:36:00.210400+0000","last_became_peered":"2024-03-19T00:36:00.210400+0000","last_unstale":"2024-03-19T00:37:56.037115+0000","last_undegraded":"2024-03-19T00:37:56.037115+0000","last_fullsized":"2024-03-19T00:37:56.037115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:10.982970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018669+0000","last_change":"2024-03-19T00:36:04.246932+0000","last_active":"2024-03-19T00:36:58.018669+0000","last_peered":"2024-03-19T00:36:58.018669+0000","last_clean":"2024-03-19T00:36:58.018669+0000","last_became_active":"2024-03-19T00:36:04.246477+0000","last_became_peered":"2024-03-19T00:36:04.246477+0000","last_unstale":"2024-03-19T00:36:58.018669+0000","last_undegraded":"2024-03-19T00:36:58.018669+0000","last_fullsized":"2024-03-19T00:36:58.018669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:02.125950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092575+0000","last_change":"2024-03-19T00:35:59.210718+0000","last_active":"2024-03-19T00:36:58.092575+0000","last_peered":"2024-03-19T00:36:58.092575+0000","last_clean":"2024-03-19T00:36:58.092575+0000","last_became_active":"2024-03-19T00:35:59.210113+0000","last_became_peered":"2024-03-19T00:35:59.210113+0000","last_unstale":"2024-03-19T00:36:58.092575+0000","last_undegraded":"2024-03-19T00:36:58.092575+0000","last_fullsized":"2024-03-19T00:36:58.092575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:14:49.585833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.034570+0000","last_change":"2024-03-19T00:36:00.217818+0000","last_active":"2024-03-19T00:37:56.034570+0000","last_peered":"2024-03-19T00:37:56.034570+0000","last_clean":"2024-03-19T00:37:56.034570+0000","last_became_active":"2024-03-19T00:36:00.215488+0000","last_became_peered":"2024-03-19T00:36:00.215488+0000","last_unstale":"2024-03-19T00:37:56.034570+0000","last_undegraded":"2024-03-19T00:37:56.034570+0000","last_fullsized":"2024-03-19T00:37:56.034570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:50.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018759+0000","last_change":"2024-03-19T00:36:04.249184+0000","last_active":"2024-03-19T00:36:58.018759+0000","last_peered":"2024-03-19T00:36:58.018759+0000","last_clean":"2024-03-19T00:36:58.018759+0000","last_became_active":"2024-03-19T00:36:04.248751+0000","last_became_peered":"2024-03-19T00:36:04.248751+0000","last_unstale":"2024-03-19T00:36:58.018759+0000","last_undegraded":"2024-03-19T00:36:58.018759+0000","last_fullsized":"2024-03-19T00:36:58.018759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:17.268983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026479+0000","last_change":"2024-03-19T00:36:00.211231+0000","last_active":"2024-03-19T00:37:56.026479+0000","last_peered":"2024-03-19T00:37:56.026479+0000","last_clean":"2024-03-19T00:37:56.026479+0000","last_became_active":"2024-03-19T00:36:00.210862+0000","last_became_peered":"2024-03-19T00:36:00.210862+0000","last_unstale":"2024-03-19T00:37:56.026479+0000","last_undegraded":"2024-03-19T00:37:56.026479+0000","last_fullsized":"2024-03-19T00:37:56.026479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:04.268000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.444526+0000","last_change":"2024-03-19T00:35:59.218505+0000","last_active":"2024-03-19T00:38:23.444526+0000","last_peered":"2024-03-19T00:38:23.444526+0000","last_clean":"2024-03-19T00:38:23.444526+0000","last_became_active":"2024-03-19T00:35:59.218175+0000","last_became_peered":"2024-03-19T00:35:59.218175+0000","last_unstale":"2024-03-19T00:38:23.444526+0000","last_undegraded":"2024-03-19T00:38:23.444526+0000","last_fullsized":"2024-03-19T00:38:23.444526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:30.044815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092869+0000","last_change":"2024-03-19T00:36:04.251446+0000","last_active":"2024-03-19T00:36:58.092869+0000","last_peered":"2024-03-19T00:36:58.092869+0000","last_clean":"2024-03-19T00:36:58.092869+0000","last_became_active":"2024-03-19T00:36:04.250986+0000","last_became_peered":"2024-03-19T00:36:04.250986+0000","last_unstale":"2024-03-19T00:36:58.092869+0000","last_undegraded":"2024-03-19T00:36:58.092869+0000","last_fullsized":"2024-03-19T00:36:58.092869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:34:00.119285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.043291+0000","last_change":"2024-03-19T00:36:00.209427+0000","last_active":"2024-03-19T00:37:56.043291+0000","last_peered":"2024-03-19T00:37:56.043291+0000","last_clean":"2024-03-19T00:37:56.043291+0000","last_became_active":"2024-03-19T00:36:00.209118+0000","last_became_peered":"2024-03-19T00:36:00.209118+0000","last_unstale":"2024-03-19T00:37:56.043291+0000","last_undegraded":"2024-03-19T00:37:56.043291+0000","last_fullsized":"2024-03-19T00:37:56.043291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:09.574806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944026+0000","last_change":"2024-03-19T00:35:59.208696+0000","last_active":"2024-03-19T00:36:57.944026+0000","last_peered":"2024-03-19T00:36:57.944026+0000","last_clean":"2024-03-19T00:36:57.944026+0000","last_became_active":"2024-03-19T00:35:59.207965+0000","last_became_peered":"2024-03-19T00:35:59.207965+0000","last_unstale":"2024-03-19T00:36:57.944026+0000","last_undegraded":"2024-03-19T00:36:57.944026+0000","last_fullsized":"2024-03-19T00:36:57.944026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:26:02.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092415+0000","last_change":"2024-03-19T00:34:46.338395+0000","last_active":"2024-03-19T00:36:58.092415+0000","last_peered":"2024-03-19T00:36:58.092415+0000","last_clean":"2024-03-19T00:36:58.092415+0000","last_became_active":"2024-03-19T00:34:46.337932+0000","last_became_peered":"2024-03-19T00:34:46.337932+0000","last_unstale":"2024-03-19T00:36:58.092415+0000","last_undegraded":"2024-03-19T00:36:58.092415+0000","last_fullsized":"2024-03-19T00:36:58.092415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:31:18.991339+0000","last_clean_scrub_stamp":"2024-03-19T00:31:18.991339+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:53:40.037629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.342005+0000","last_change":"2024-03-19T00:36:04.250153+0000","last_active":"2024-03-19T00:36:58.342005+0000","last_peered":"2024-03-19T00:36:58.342005+0000","last_clean":"2024-03-19T00:36:58.342005+0000","last_became_active":"2024-03-19T00:36:04.249729+0000","last_became_peered":"2024-03-19T00:36:04.249729+0000","last_unstale":"2024-03-19T00:36:58.342005+0000","last_undegraded":"2024-03-19T00:36:58.342005+0000","last_fullsized":"2024-03-19T00:36:58.342005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:51.050528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028043+0000","last_change":"2024-03-19T00:36:00.219063+0000","last_active":"2024-03-19T00:37:56.028043+0000","last_peered":"2024-03-19T00:37:56.028043+0000","last_clean":"2024-03-19T00:37:56.028043+0000","last_became_active":"2024-03-19T00:36:00.218755+0000","last_became_peered":"2024-03-19T00:36:00.218755+0000","last_unstale":"2024-03-19T00:37:56.028043+0000","last_undegraded":"2024-03-19T00:37:56.028043+0000","last_fullsized":"2024-03-19T00:37:56.028043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:03.803971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.092489+0000","last_change":"2024-03-19T00:35:59.210642+0000","last_active":"2024-03-19T00:36:58.092489+0000","last_peered":"2024-03-19T00:36:58.092489+0000","last_clean":"2024-03-19T00:36:58.092489+0000","last_became_active":"2024-03-19T00:35:59.210333+0000","last_became_peered":"2024-03-19T00:35:59.210333+0000","last_unstale":"2024-03-19T00:36:58.092489+0000","last_undegraded":"2024-03-19T00:36:58.092489+0000","last_fullsized":"2024-03-19T00:36:58.092489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:15.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945808+0000","last_change":"2024-03-19T00:36:04.245377+0000","last_active":"2024-03-19T00:36:57.945808+0000","last_peered":"2024-03-19T00:36:57.945808+0000","last_clean":"2024-03-19T00:36:57.945808+0000","last_became_active":"2024-03-19T00:36:04.244126+0000","last_became_peered":"2024-03-19T00:36:04.244126+0000","last_unstale":"2024-03-19T00:36:57.945808+0000","last_undegraded":"2024-03-19T00:36:57.945808+0000","last_fullsized":"2024-03-19T00:36:57.945808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:01:10.941604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.948192+0000","last_change":"2024-03-19T00:35:59.212873+0000","last_active":"2024-03-19T00:36:57.948192+0000","last_peered":"2024-03-19T00:36:57.948192+0000","last_clean":"2024-03-19T00:36:57.948192+0000","last_became_active":"2024-03-19T00:35:59.212561+0000","last_became_peered":"2024-03-19T00:35:59.212561+0000","last_unstale":"2024-03-19T00:36:57.948192+0000","last_undegraded":"2024-03-19T00:36:57.948192+0000","last_fullsized":"2024-03-19T00:36:57.948192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:39:58.478584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026012+0000","last_change":"2024-03-19T00:36:00.218330+0000","last_active":"2024-03-19T00:37:56.026012+0000","last_peered":"2024-03-19T00:37:56.026012+0000","last_clean":"2024-03-19T00:37:56.026012+0000","last_became_active":"2024-03-19T00:36:00.217953+0000","last_became_peered":"2024-03-19T00:36:00.217953+0000","last_unstale":"2024-03-19T00:37:56.026012+0000","last_undegraded":"2024-03-19T00:37:56.026012+0000","last_fullsized":"2024-03-19T00:37:56.026012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:34:38.983810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946095+0000","last_change":"2024-03-19T00:36:04.246355+0000","last_active":"2024-03-19T00:36:57.946095+0000","last_peered":"2024-03-19T00:36:57.946095+0000","last_clean":"2024-03-19T00:36:57.946095+0000","last_became_active":"2024-03-19T00:36:04.245947+0000","last_became_peered":"2024-03-19T00:36:04.245947+0000","last_unstale":"2024-03-19T00:36:57.946095+0000","last_undegraded":"2024-03-19T00:36:57.946095+0000","last_fullsized":"2024-03-19T00:36:57.946095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:29:50.289690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:23.219170+0000","last_change":"2024-03-19T00:36:57.954788+0000","last_active":"2024-03-19T00:38:23.219170+0000","last_peered":"2024-03-19T00:38:23.219170+0000","last_clean":"2024-03-19T00:38:23.219170+0000","last_became_active":"2024-03-19T00:36:57.949916+0000","last_became_peered":"2024-03-19T00:36:57.949916+0000","last_unstale":"2024-03-19T00:38:23.219170+0000","last_undegraded":"2024-03-19T00:38:23.219170+0000","last_fullsized":"2024-03-19T00:38:23.219170+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:09.959915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.033677+0000","last_change":"2024-03-19T00:36:00.214517+0000","last_active":"2024-03-19T00:37:56.033677+0000","last_peered":"2024-03-19T00:37:56.033677+0000","last_clean":"2024-03-19T00:37:56.033677+0000","last_became_active":"2024-03-19T00:36:00.214191+0000","last_became_peered":"2024-03-19T00:36:00.214191+0000","last_unstale":"2024-03-19T00:37:56.033677+0000","last_undegraded":"2024-03-19T00:37:56.033677+0000","last_fullsized":"2024-03-19T00:37:56.033677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:26:23.007225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944256+0000","last_change":"2024-03-19T00:36:04.239680+0000","last_active":"2024-03-19T00:36:57.944256+0000","last_peered":"2024-03-19T00:36:57.944256+0000","last_clean":"2024-03-19T00:36:57.944256+0000","last_became_active":"2024-03-19T00:36:04.239161+0000","last_became_peered":"2024-03-19T00:36:04.239161+0000","last_unstale":"2024-03-19T00:36:57.944256+0000","last_undegraded":"2024-03-19T00:36:57.944256+0000","last_fullsized":"2024-03-19T00:36:57.944256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:46.848857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.945162+0000","last_change":"2024-03-19T00:35:59.212143+0000","last_active":"2024-03-19T00:36:57.945162+0000","last_peered":"2024-03-19T00:36:57.945162+0000","last_clean":"2024-03-19T00:36:57.945162+0000","last_became_active":"2024-03-19T00:35:59.211820+0000","last_became_peered":"2024-03-19T00:35:59.211820+0000","last_unstale":"2024-03-19T00:36:57.945162+0000","last_undegraded":"2024-03-19T00:36:57.945162+0000","last_fullsized":"2024-03-19T00:36:57.945162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:02:19.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044817+0000","last_change":"2024-03-19T00:36:00.221178+0000","last_active":"2024-03-19T00:37:56.044817+0000","last_peered":"2024-03-19T00:37:56.044817+0000","last_clean":"2024-03-19T00:37:56.044817+0000","last_became_active":"2024-03-19T00:36:00.220861+0000","last_became_peered":"2024-03-19T00:36:00.220861+0000","last_unstale":"2024-03-19T00:37:56.044817+0000","last_undegraded":"2024-03-19T00:37:56.044817+0000","last_fullsized":"2024-03-19T00:37:56.044817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:16:34.574922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.343169+0000","last_change":"2024-03-19T00:36:58.342763+0000","last_active":"2024-03-19T00:36:58.343169+0000","last_peered":"2024-03-19T00:36:58.343169+0000","last_clean":"2024-03-19T00:36:58.343169+0000","last_became_active":"2024-03-19T00:36:58.342305+0000","last_became_peered":"2024-03-19T00:36:58.342305+0000","last_unstale":"2024-03-19T00:36:58.343169+0000","last_undegraded":"2024-03-19T00:36:58.343169+0000","last_fullsized":"2024-03-19T00:36:58.343169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:44.312347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.952606+0000","last_change":"2024-03-19T00:36:57.952182+0000","last_active":"2024-03-19T00:36:57.952606+0000","last_peered":"2024-03-19T00:36:57.952606+0000","last_clean":"2024-03-19T00:36:57.952606+0000","last_became_active":"2024-03-19T00:36:57.948822+0000","last_became_peered":"2024-03-19T00:36:57.948822+0000","last_unstale":"2024-03-19T00:36:57.952606+0000","last_undegraded":"2024-03-19T00:36:57.952606+0000","last_fullsized":"2024-03-19T00:36:57.952606+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:21:50.527748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.026905+0000","last_change":"2024-03-19T00:36:00.204989+0000","last_active":"2024-03-19T00:37:56.026905+0000","last_peered":"2024-03-19T00:37:56.026905+0000","last_clean":"2024-03-19T00:37:56.026905+0000","last_became_active":"2024-03-19T00:36:00.204683+0000","last_became_peered":"2024-03-19T00:36:00.204683+0000","last_unstale":"2024-03-19T00:37:56.026905+0000","last_undegraded":"2024-03-19T00:37:56.026905+0000","last_fullsized":"2024-03-19T00:37:56.026905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:38:30.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.943843+0000","last_change":"2024-03-19T00:36:04.243378+0000","last_active":"2024-03-19T00:36:57.943843+0000","last_peered":"2024-03-19T00:36:57.943843+0000","last_clean":"2024-03-19T00:36:57.943843+0000","last_became_active":"2024-03-19T00:36:04.242971+0000","last_became_peered":"2024-03-19T00:36:04.242971+0000","last_unstale":"2024-03-19T00:36:57.943843+0000","last_undegraded":"2024-03-19T00:36:57.943843+0000","last_fullsized":"2024-03-19T00:36:57.943843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:10.957752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946346+0000","last_change":"2024-03-19T00:35:59.202460+0000","last_active":"2024-03-19T00:36:57.946346+0000","last_peered":"2024-03-19T00:36:57.946346+0000","last_clean":"2024-03-19T00:36:57.946346+0000","last_became_active":"2024-03-19T00:35:59.202110+0000","last_became_peered":"2024-03-19T00:35:59.202110+0000","last_unstale":"2024-03-19T00:36:57.946346+0000","last_undegraded":"2024-03-19T00:36:57.946346+0000","last_fullsized":"2024-03-19T00:36:57.946346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:38.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029217+0000","last_change":"2024-03-19T00:36:00.220151+0000","last_active":"2024-03-19T00:37:56.029217+0000","last_peered":"2024-03-19T00:37:56.029217+0000","last_clean":"2024-03-19T00:37:56.029217+0000","last_became_active":"2024-03-19T00:36:00.219842+0000","last_became_peered":"2024-03-19T00:36:00.219842+0000","last_unstale":"2024-03-19T00:37:56.029217+0000","last_undegraded":"2024-03-19T00:37:56.029217+0000","last_fullsized":"2024-03-19T00:37:56.029217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:23.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944721+0000","last_change":"2024-03-19T00:36:04.247326+0000","last_active":"2024-03-19T00:36:57.944721+0000","last_peered":"2024-03-19T00:36:57.944721+0000","last_clean":"2024-03-19T00:36:57.944721+0000","last_became_active":"2024-03-19T00:36:04.246859+0000","last_became_peered":"2024-03-19T00:36:04.246859+0000","last_unstale":"2024-03-19T00:36:57.944721+0000","last_undegraded":"2024-03-19T00:36:57.944721+0000","last_fullsized":"2024-03-19T00:36:57.944721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:50.921669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:38:08.310182+0000","last_change":"2024-03-19T00:35:59.218613+0000","last_active":"2024-03-19T00:38:08.310182+0000","last_peered":"2024-03-19T00:38:08.310182+0000","last_clean":"2024-03-19T00:38:08.310182+0000","last_became_active":"2024-03-19T00:35:59.218265+0000","last_became_peered":"2024-03-19T00:35:59.218265+0000","last_unstale":"2024-03-19T00:38:08.310182+0000","last_undegraded":"2024-03-19T00:38:08.310182+0000","last_fullsized":"2024-03-19T00:38:08.310182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:58:16.274769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041614+0000","last_change":"2024-03-19T00:36:00.220404+0000","last_active":"2024-03-19T00:37:56.041614+0000","last_peered":"2024-03-19T00:37:56.041614+0000","last_clean":"2024-03-19T00:37:56.041614+0000","last_became_active":"2024-03-19T00:36:00.220030+0000","last_became_peered":"2024-03-19T00:36:00.220030+0000","last_unstale":"2024-03-19T00:37:56.041614+0000","last_undegraded":"2024-03-19T00:37:56.041614+0000","last_fullsized":"2024-03-19T00:37:56.041614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:18:27.115836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944067+0000","last_change":"2024-03-19T00:36:04.240936+0000","last_active":"2024-03-19T00:36:57.944067+0000","last_peered":"2024-03-19T00:36:57.944067+0000","last_clean":"2024-03-19T00:36:57.944067+0000","last_became_active":"2024-03-19T00:36:04.240566+0000","last_became_peered":"2024-03-19T00:36:04.240566+0000","last_unstale":"2024-03-19T00:36:57.944067+0000","last_undegraded":"2024-03-19T00:36:57.944067+0000","last_fullsized":"2024-03-19T00:36:57.944067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:19:34.401670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947723+0000","last_change":"2024-03-19T00:35:59.213294+0000","last_active":"2024-03-19T00:36:57.947723+0000","last_peered":"2024-03-19T00:36:57.947723+0000","last_clean":"2024-03-19T00:36:57.947723+0000","last_became_active":"2024-03-19T00:35:59.212974+0000","last_became_peered":"2024-03-19T00:35:59.212974+0000","last_unstale":"2024-03-19T00:36:57.947723+0000","last_undegraded":"2024-03-19T00:36:57.947723+0000","last_fullsized":"2024-03-19T00:36:57.947723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:58.238261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.039981+0000","last_change":"2024-03-19T00:36:00.226876+0000","last_active":"2024-03-19T00:37:56.039981+0000","last_peered":"2024-03-19T00:37:56.039981+0000","last_clean":"2024-03-19T00:37:56.039981+0000","last_became_active":"2024-03-19T00:36:00.226557+0000","last_became_peered":"2024-03-19T00:36:00.226557+0000","last_unstale":"2024-03-19T00:37:56.039981+0000","last_undegraded":"2024-03-19T00:37:56.039981+0000","last_fullsized":"2024-03-19T00:37:56.039981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:09:38.269247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.946223+0000","last_change":"2024-03-19T00:36:04.244568+0000","last_active":"2024-03-19T00:36:57.946223+0000","last_peered":"2024-03-19T00:36:57.946223+0000","last_clean":"2024-03-19T00:36:57.946223+0000","last_became_active":"2024-03-19T00:36:04.244100+0000","last_became_peered":"2024-03-19T00:36:04.244100+0000","last_unstale":"2024-03-19T00:36:57.946223+0000","last_undegraded":"2024-03-19T00:36:57.946223+0000","last_fullsized":"2024-03-19T00:36:57.946223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:06.330494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.944993+0000","last_change":"2024-03-19T00:35:59.214679+0000","last_active":"2024-03-19T00:36:57.944993+0000","last_peered":"2024-03-19T00:36:57.944993+0000","last_clean":"2024-03-19T00:36:57.944993+0000","last_became_active":"2024-03-19T00:35:59.214412+0000","last_became_peered":"2024-03-19T00:35:59.214412+0000","last_unstale":"2024-03-19T00:36:57.944993+0000","last_undegraded":"2024-03-19T00:36:57.944993+0000","last_fullsized":"2024-03-19T00:36:57.944993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:06.670601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040908+0000","last_change":"2024-03-19T00:36:00.214908+0000","last_active":"2024-03-19T00:37:56.040908+0000","last_peered":"2024-03-19T00:37:56.040908+0000","last_clean":"2024-03-19T00:37:56.040908+0000","last_became_active":"2024-03-19T00:36:00.214607+0000","last_became_peered":"2024-03-19T00:36:00.214607+0000","last_unstale":"2024-03-19T00:37:56.040908+0000","last_undegraded":"2024-03-19T00:37:56.040908+0000","last_fullsized":"2024-03-19T00:37:56.040908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:09:18.841570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:57.947224+0000","last_change":"2024-03-19T00:36:04.250155+0000","last_active":"2024-03-19T00:36:57.947224+0000","last_peered":"2024-03-19T00:36:57.947224+0000","last_clean":"2024-03-19T00:36:57.947224+0000","last_became_active":"2024-03-19T00:36:04.249711+0000","last_became_peered":"2024-03-19T00:36:04.249711+0000","last_unstale":"2024-03-19T00:36:57.947224+0000","last_undegraded":"2024-03-19T00:36:57.947224+0000","last_fullsized":"2024-03-19T00:36:57.947224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:01:16.515209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341995+0000","last_change":"2024-03-19T00:35:59.212609+0000","last_active":"2024-03-19T00:36:58.341995+0000","last_peered":"2024-03-19T00:36:58.341995+0000","last_clean":"2024-03-19T00:36:58.341995+0000","last_became_active":"2024-03-19T00:35:59.212273+0000","last_became_peered":"2024-03-19T00:35:59.212273+0000","last_unstale":"2024-03-19T00:36:58.341995+0000","last_undegraded":"2024-03-19T00:36:58.341995+0000","last_fullsized":"2024-03-19T00:36:58.341995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:20:48.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031881+0000","last_change":"2024-03-19T00:36:00.225301+0000","last_active":"2024-03-19T00:37:56.031881+0000","last_peered":"2024-03-19T00:37:56.031881+0000","last_clean":"2024-03-19T00:37:56.031881+0000","last_became_active":"2024-03-19T00:36:00.224969+0000","last_became_peered":"2024-03-19T00:36:00.224969+0000","last_unstale":"2024-03-19T00:37:56.031881+0000","last_undegraded":"2024-03-19T00:37:56.031881+0000","last_fullsized":"2024-03-19T00:37:56.031881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:48:12.430286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.341269+0000","last_change":"2024-03-19T00:36:04.240101+0000","last_active":"2024-03-19T00:36:58.341269+0000","last_peered":"2024-03-19T00:36:58.341269+0000","last_clean":"2024-03-19T00:36:58.341269+0000","last_became_active":"2024-03-19T00:36:04.239690+0000","last_became_peered":"2024-03-19T00:36:04.239690+0000","last_unstale":"2024-03-19T00:36:58.341269+0000","last_undegraded":"2024-03-19T00:36:58.341269+0000","last_fullsized":"2024-03-19T00:36:58.341269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:36:03.213633+0000","last_clean_scrub_stamp":"2024-03-19T00:36:03.213633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:16.711383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:36:58.018797+0000","last_change":"2024-03-19T00:35:59.219747+0000","last_active":"2024-03-19T00:36:58.018797+0000","last_peered":"2024-03-19T00:36:58.018797+0000","last_clean":"2024-03-19T00:36:58.018797+0000","last_became_active":"2024-03-19T00:35:59.218378+0000","last_became_peered":"2024-03-19T00:35:59.218378+0000","last_unstale":"2024-03-19T00:36:58.018797+0000","last_undegraded":"2024-03-19T00:36:58.018797+0000","last_fullsized":"2024-03-19T00:36:58.018797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:58.179354+0000","last_clean_scrub_stamp":"2024-03-19T00:35:58.179354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:09:44.318425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031169+0000","last_change":"2024-03-19T00:36:00.219142+0000","last_active":"2024-03-19T00:37:56.031169+0000","last_peered":"2024-03-19T00:37:56.031169+0000","last_clean":"2024-03-19T00:37:56.031169+0000","last_became_active":"2024-03-19T00:36:00.218737+0000","last_became_peered":"2024-03-19T00:36:00.218737+0000","last_unstale":"2024-03-19T00:37:56.031169+0000","last_undegraded":"2024-03-19T00:37:56.031169+0000","last_fullsized":"2024-03-19T00:37:56.031169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:35:39.467824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.028083+0000","last_change":"2024-03-19T00:36:00.212182+0000","last_active":"2024-03-19T00:37:56.028083+0000","last_peered":"2024-03-19T00:37:56.028083+0000","last_clean":"2024-03-19T00:37:56.028083+0000","last_became_active":"2024-03-19T00:36:00.211844+0000","last_became_peered":"2024-03-19T00:36:00.211844+0000","last_unstale":"2024-03-19T00:37:56.028083+0000","last_undegraded":"2024-03-19T00:37:56.028083+0000","last_fullsized":"2024-03-19T00:37:56.028083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:32:28.677407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.023829+0000","last_change":"2024-03-19T00:36:00.216058+0000","last_active":"2024-03-19T00:37:56.023829+0000","last_peered":"2024-03-19T00:37:56.023829+0000","last_clean":"2024-03-19T00:37:56.023829+0000","last_became_active":"2024-03-19T00:36:00.215752+0000","last_became_peered":"2024-03-19T00:36:00.215752+0000","last_unstale":"2024-03-19T00:37:56.023829+0000","last_undegraded":"2024-03-19T00:37:56.023829+0000","last_fullsized":"2024-03-19T00:37:56.023829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:15.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.040636+0000","last_change":"2024-03-19T00:36:00.220210+0000","last_active":"2024-03-19T00:37:56.040636+0000","last_peered":"2024-03-19T00:37:56.040636+0000","last_clean":"2024-03-19T00:37:56.040636+0000","last_became_active":"2024-03-19T00:36:00.219881+0000","last_became_peered":"2024-03-19T00:36:00.219881+0000","last_unstale":"2024-03-19T00:37:56.040636+0000","last_undegraded":"2024-03-19T00:37:56.040636+0000","last_fullsized":"2024-03-19T00:37:56.040636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:50.531415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.041821+0000","last_change":"2024-03-19T00:36:58.019535+0000","last_active":"2024-03-19T00:37:56.041821+0000","last_peered":"2024-03-19T00:37:56.041821+0000","last_clean":"2024-03-19T00:37:56.041821+0000","last_became_active":"2024-03-19T00:36:58.019220+0000","last_became_peered":"2024-03-19T00:36:58.019220+0000","last_unstale":"2024-03-19T00:37:56.041821+0000","last_undegraded":"2024-03-19T00:37:56.041821+0000","last_fullsized":"2024-03-19T00:37:56.041821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:04:51.797603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.044655+0000","last_change":"2024-03-19T00:36:00.219332+0000","last_active":"2024-03-19T00:37:56.044655+0000","last_peered":"2024-03-19T00:37:56.044655+0000","last_clean":"2024-03-19T00:37:56.044655+0000","last_became_active":"2024-03-19T00:36:00.219005+0000","last_became_peered":"2024-03-19T00:36:00.219005+0000","last_unstale":"2024-03-19T00:37:56.044655+0000","last_undegraded":"2024-03-19T00:37:56.044655+0000","last_fullsized":"2024-03-19T00:37:56.044655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:06.967348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.024108+0000","last_change":"2024-03-19T00:36:00.211275+0000","last_active":"2024-03-19T00:37:56.024108+0000","last_peered":"2024-03-19T00:37:56.024108+0000","last_clean":"2024-03-19T00:37:56.024108+0000","last_became_active":"2024-03-19T00:36:00.210823+0000","last_became_peered":"2024-03-19T00:36:00.210823+0000","last_unstale":"2024-03-19T00:37:56.024108+0000","last_undegraded":"2024-03-19T00:37:56.024108+0000","last_fullsized":"2024-03-19T00:37:56.024108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:04:05.665050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.042382+0000","last_change":"2024-03-19T00:36:00.212003+0000","last_active":"2024-03-19T00:37:56.042382+0000","last_peered":"2024-03-19T00:37:56.042382+0000","last_clean":"2024-03-19T00:37:56.042382+0000","last_became_active":"2024-03-19T00:36:00.211682+0000","last_became_peered":"2024-03-19T00:36:00.211682+0000","last_unstale":"2024-03-19T00:37:56.042382+0000","last_undegraded":"2024-03-19T00:37:56.042382+0000","last_fullsized":"2024-03-19T00:37:56.042382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:47.357235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.031736+0000","last_change":"2024-03-19T00:36:00.216082+0000","last_active":"2024-03-19T00:37:56.031736+0000","last_peered":"2024-03-19T00:37:56.031736+0000","last_clean":"2024-03-19T00:37:56.031736+0000","last_became_active":"2024-03-19T00:36:00.215759+0000","last_became_peered":"2024-03-19T00:36:00.215759+0000","last_unstale":"2024-03-19T00:37:56.031736+0000","last_undegraded":"2024-03-19T00:37:56.031736+0000","last_fullsized":"2024-03-19T00:37:56.031736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:09.877707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.025787+0000","last_change":"2024-03-19T00:36:00.205964+0000","last_active":"2024-03-19T00:37:56.025787+0000","last_peered":"2024-03-19T00:37:56.025787+0000","last_clean":"2024-03-19T00:37:56.025787+0000","last_became_active":"2024-03-19T00:36:00.205660+0000","last_became_peered":"2024-03-19T00:36:00.205660+0000","last_unstale":"2024-03-19T00:37:56.025787+0000","last_undegraded":"2024-03-19T00:37:56.025787+0000","last_fullsized":"2024-03-19T00:37:56.025787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:10.036575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.032629+0000","last_change":"2024-03-19T00:36:00.209326+0000","last_active":"2024-03-19T00:37:56.032629+0000","last_peered":"2024-03-19T00:37:56.032629+0000","last_clean":"2024-03-19T00:37:56.032629+0000","last_became_active":"2024-03-19T00:36:00.208981+0000","last_became_peered":"2024-03-19T00:36:00.208981+0000","last_unstale":"2024-03-19T00:37:56.032629+0000","last_undegraded":"2024-03-19T00:37:56.032629+0000","last_fullsized":"2024-03-19T00:37:56.032629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:31.559118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.043776+0000","last_change":"2024-03-19T00:36:00.216715+0000","last_active":"2024-03-19T00:37:56.043776+0000","last_peered":"2024-03-19T00:37:56.043776+0000","last_clean":"2024-03-19T00:37:56.043776+0000","last_became_active":"2024-03-19T00:36:00.216373+0000","last_became_peered":"2024-03-19T00:36:00.216373+0000","last_unstale":"2024-03-19T00:37:56.043776+0000","last_undegraded":"2024-03-19T00:37:56.043776+0000","last_fullsized":"2024-03-19T00:37:56.043776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:52:58.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.045485+0000","last_change":"2024-03-19T00:36:00.221671+0000","last_active":"2024-03-19T00:37:56.045485+0000","last_peered":"2024-03-19T00:37:56.045485+0000","last_clean":"2024-03-19T00:37:56.045485+0000","last_became_active":"2024-03-19T00:36:00.221314+0000","last_became_peered":"2024-03-19T00:36:00.221314+0000","last_unstale":"2024-03-19T00:37:56.045485+0000","last_undegraded":"2024-03-19T00:37:56.045485+0000","last_fullsized":"2024-03-19T00:37:56.045485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:15:37.331833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T00:37:56.029358+0000","last_change":"2024-03-19T00:36:57.951256+0000","last_active":"2024-03-19T00:37:56.029358+0000","last_peered":"2024-03-19T00:37:56.029358+0000","last_clean":"2024-03-19T00:37:56.029358+0000","last_became_active":"2024-03-19T00:36:57.950879+0000","last_became_peered":"2024-03-19T00:36:57.950879+0000","last_unstale":"2024-03-19T00:37:56.029358+0000","last_undegraded":"2024-03-19T00:37:56.029358+0000","last_fullsized":"2024-03-19T00:37:56.029358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T00:35:59.183348+0000","last_clean_scrub_stamp":"2024-03-19T00:35:59.183348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:46:26.341725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69762,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":446,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":214773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1153209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":500971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1099323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":564942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":561625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1092609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T00:38:28.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.972+0000 7f69349b2640 1 -- 172.21.15.7:0/2040451459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930102300 msgr2=0x7f6930108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.972+0000 7f69349b2640 1 --2- 172.21.15.7:0/2040451459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930102300 0x7f6930108b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6928009ca0 tx=0x7f692802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- 172.21.15.7:0/2040451459 shutdown_connections 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 --2- 172.21.15.7:0/2040451459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930102300 0x7f6930108b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 --2- 172.21.15.7:0/2040451459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6930101940 0x7f6930101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 --2- 172.21.15.7:0/2040451459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 0x7f6930100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- 172.21.15.7:0/2040451459 >> 172.21.15.7:0/2040451459 conn(0x7f69300fbcc0 msgr2=0x7f69300fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- 172.21.15.7:0/2040451459 shutdown_connections 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- 172.21.15.7:0/2040451459 wait complete. 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 Processor -- start 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- start start 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 0x7f69301979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 0x7f6930197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6930102300 0x7f693019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f693010c280 con 0x7f6930101940 2024-03-19T00:38:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f693010c100 con 0x7f6930100750 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.973+0000 7f69349b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f693010c400 con 0x7f6930102300 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 0x7f6930197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 0x7f6930197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42878/0 (socket says 172.21.15.7:42878) 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 -- 172.21.15.7:0/2841108030 learned_addr learned my addr 172.21.15.7:0/2841108030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f692e59c640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 0x7f69301979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f692ed9d640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6930102300 0x7f693019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 -- 172.21.15.7:0/2841108030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6930102300 msgr2=0x7f693019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6930102300 0x7f693019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 -- 172.21.15.7:0/2841108030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 msgr2=0x7f69301979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 0x7f69301979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 -- 172.21.15.7:0/2841108030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693019f600 con 0x7f6930101940 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f692e59c640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 0x7f69301979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f6927fff640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 0x7f6930197f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f691800e9f0 tx=0x7f691800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691800cd80 con 0x7f6930101940 2024-03-19T00:38:28.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69180054d0 con 0x7f6930101940 2024-03-19T00:38:28.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693019f8f0 con 0x7f6930101940 2024-03-19T00:38:28.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918010490 con 0x7f6930101940 2024-03-19T00:38:28.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.974+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693019fdf0 con 0x7f6930101940 2024-03-19T00:38:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.975+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69301a0050 con 0x7f6930101940 2024-03-19T00:38:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.977+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f6918005000 con 0x7f6930101940 2024-03-19T00:38:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.977+0000 7f69277fe640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f691c0a2890 0x7f691c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.978+0000 7f692e59c640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f691c0a2890 0x7f691c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.978+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f69180cf510 con 0x7f6930101940 2024-03-19T00:38:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.978+0000 7f692e59c640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f691c0a2890 0x7f691c0a4d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f69301017a0 tx=0x7f6920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:28.981+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6918053740 con 0x7f6930101940 2024-03-19T00:38:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.164+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69301a0220 con 0x7f6930101940 2024-03-19T00:38:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.166+0000 7f69277fe640 1 -- 172.21.15.7:0/2841108030 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f69180052f0 con 0x7f6930101940 2024-03-19T00:38:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:29.168 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f691c0a2890 msgr2=0x7f691c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f691c0a2890 0x7f691c0a4d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f69301017a0 tx=0x7f6920005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 msgr2=0x7f6930197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 0x7f6930197f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f691800e9f0 tx=0x7f691800edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 shutdown_connections 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f691c0a2890 0x7f691c0a4d50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6930102300 0x7f693019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6930101940 0x7f6930197f20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 --2- 172.21.15.7:0/2841108030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6930100750 0x7f69301979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 >> 172.21.15.7:0/2841108030 conn(0x7f69300fbcc0 msgr2=0x7f6930068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 shutdown_connections 2024-03-19T00:38:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.168+0000 7f69349b2640 1 -- 172.21.15.7:0/2841108030 wait complete. 2024-03-19T00:38:29.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:38:29.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:38:29.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 -- 172.21.15.7:0/2624626270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808100740 msgr2=0x7f1808100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 --2- 172.21.15.7:0/2624626270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808100740 0x7f1808100b60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1804009900 tx=0x7f180402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:29.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 -- 172.21.15.7:0/2624626270 shutdown_connections 2024-03-19T00:38:29.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 --2- 172.21.15.7:0/2624626270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081022f0 0x7f1808108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 --2- 172.21.15.7:0/2624626270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1808101930 0x7f1808101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 --2- 172.21.15.7:0/2624626270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808100740 0x7f1808100b60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 -- 172.21.15.7:0/2624626270 >> 172.21.15.7:0/2624626270 conn(0x7f18080fbcb0 msgr2=0x7f18080fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 -- 172.21.15.7:0/2624626270 shutdown_connections 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 -- 172.21.15.7:0/2624626270 wait complete. 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.286+0000 7f180f968640 1 Processor -- start 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 -- start start 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1808100740 0x7f1808197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 0x7f1808197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081022f0 0x7f180819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f180810d390 con 0x7f1808101930 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f180810d210 con 0x7f1808100740 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f180810d510 con 0x7f18081022f0 2024-03-19T00:38:29.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 0x7f1808197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 0x7f1808197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42900/0 (socket says 172.21.15.7:42900) 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 -- 172.21.15.7:0/1779931201 learned_addr learned my addr 172.21.15.7:0/1779931201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180d704640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1808100740 0x7f1808197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180df05640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081022f0 0x7f180819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 -- 172.21.15.7:0/1779931201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081022f0 msgr2=0x7f180819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081022f0 0x7f180819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 -- 172.21.15.7:0/1779931201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1808100740 msgr2=0x7f1808197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1808100740 0x7f1808197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 -- 172.21.15.7:0/1779931201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180819f6c0 con 0x7f1808101930 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180cf03640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 0x7f1808197fe0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e9f0 tx=0x7f17f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800cd80 con 0x7f1808101930 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17f80108e0 con 0x7f1808101930 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.287+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180819f9b0 con 0x7f1808101930 2024-03-19T00:38:29.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.288+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8018920 con 0x7f1808101930 2024-03-19T00:38:29.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.288+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180819ff50 con 0x7f1808101930 2024-03-19T00:38:29.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.288+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1808107b30 con 0x7f1808101930 2024-03-19T00:38:29.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.289+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f17f8010a90 con 0x7f1808101930 2024-03-19T00:38:29.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.290+0000 7f17fe7fc640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f17f40a2910 0x7f17f40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.290+0000 7f180d704640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f17f40a2910 0x7f17f40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.290+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f17f80cf1e0 con 0x7f1808101930 2024-03-19T00:38:29.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.291+0000 7f180d704640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f17f40a2910 0x7f17f40a4dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1804005ab0 tx=0x7f1804005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:29.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.295+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f17f809b770 con 0x7f1808101930 2024-03-19T00:38:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.477+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f180802d410 con 0x7f1808101930 2024-03-19T00:38:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.478+0000 7f17fe7fc640 1 -- 172.21.15.7:0/1779931201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f17f809f810 con 0x7f1808101930 2024-03-19T00:38:29.480 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f17f40a2910 msgr2=0x7f17f40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f17f40a2910 0x7f17f40a4dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1804005ab0 tx=0x7f1804005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 msgr2=0x7f1808197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 0x7f1808197fe0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e9f0 tx=0x7f17f800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 shutdown_connections 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f17f40a2910 0x7f17f40a4dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18081022f0 0x7f180819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1808101930 0x7f1808197fe0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 --2- 172.21.15.7:0/1779931201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1808100740 0x7f1808197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 >> 172.21.15.7:0/1779931201 conn(0x7f18080fbcb0 msgr2=0x7f1808104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 shutdown_connections 2024-03-19T00:38:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.480+0000 7f180f968640 1 -- 172.21.15.7:0/1779931201 wait complete. 2024-03-19T00:38:29.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:38:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 -- 172.21.15.7:0/4261648845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 msgr2=0x7fb8cc101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/4261648845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 0x7fb8cc101da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009900 tx=0x7fb8b402bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 -- 172.21.15.7:0/4261648845 shutdown_connections 2024-03-19T00:38:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/4261648845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/4261648845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 0x7fb8cc101da0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/4261648845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 0x7fb8cc100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 -- 172.21.15.7:0/4261648845 >> 172.21.15.7:0/4261648845 conn(0x7fb8cc0fbc80 msgr2=0x7fb8cc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 -- 172.21.15.7:0/4261648845 shutdown_connections 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.596+0000 7fb8d1d77640 1 -- 172.21.15.7:0/4261648845 wait complete. 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 Processor -- start 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 -- start start 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 0x7fb8cc196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 0x7fb8cc196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10c240 con 0x7fb8cc1022e0 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10c0c0 con 0x7fb8cc101920 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8d1d77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10c3c0 con 0x7fb8cc100730 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42918/0 (socket says 172.21.15.7:42918) 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8cbfff640 1 -- 172.21.15.7:0/3261235387 learned_addr learned my addr 172.21.15.7:0/3261235387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8cb7fe640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 0x7fb8cc196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.597+0000 7fb8caffd640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 0x7fb8cc196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cbfff640 1 -- 172.21.15.7:0/3261235387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 msgr2=0x7fb8cc196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cbfff640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 0x7fb8cc196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cbfff640 1 -- 172.21.15.7:0/3261235387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 msgr2=0x7fb8cc196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cbfff640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 0x7fb8cc196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cbfff640 1 -- 172.21.15.7:0/3261235387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc19e5a0 con 0x7fb8cc1022e0 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cb7fe640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 0x7fb8cc196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8cbfff640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc19df30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400eca0 tx=0x7fb8c400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4004b20 con 0x7fb8cc1022e0 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc069f40 con 0x7fb8cc1022e0 2024-03-19T00:38:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8c4004cc0 con 0x7fb8cc1022e0 2024-03-19T00:38:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4017670 con 0x7fb8cc1022e0 2024-03-19T00:38:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.598+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc06a410 con 0x7fb8cc1022e0 2024-03-19T00:38:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.599+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc068af0 con 0x7fb8cc1022e0 2024-03-19T00:38:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.600+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fb8c4005020 con 0x7fb8cc1022e0 2024-03-19T00:38:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.601+0000 7fb8c8ff9640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8b80a2890 0x7fb8b80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.601+0000 7fb8cb7fe640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8b80a2890 0x7fb8b80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.601+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb8c40d0a10 con 0x7fb8cc1022e0 2024-03-19T00:38:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.601+0000 7fb8cb7fe640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8b80a2890 0x7fb8b80a4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc101780 tx=0x7fb8bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:29.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.605+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb8c409cf40 con 0x7fb8cc1022e0 2024-03-19T00:38:29.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.788+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8cc107750 con 0x7fb8cc1022e0 2024-03-19T00:38:29.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.788+0000 7fb8c8ff9640 1 -- 172.21.15.7:0/3261235387 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fb8c40a0fe0 con 0x7fb8cc1022e0 2024-03-19T00:38:29.790 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8b80a2890 msgr2=0x7fb8b80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8b80a2890 0x7fb8b80a4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc101780 tx=0x7fb8bc005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 msgr2=0x7fb8cc19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc19df30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400eca0 tx=0x7fb8c400b690 comp rx=0 tx=0).stop 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 shutdown_connections 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8b80a2890 0x7fb8b80a4d50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1022e0 0x7fb8cc19df30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8cc101920 0x7fb8cc196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 --2- 172.21.15.7:0/3261235387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc100730 0x7fb8cc196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 >> 172.21.15.7:0/3261235387 conn(0x7fb8cc0fbc80 msgr2=0x7fb8cc104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 shutdown_connections 2024-03-19T00:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.791+0000 7fb8d1d77640 1 -- 172.21.15.7:0/3261235387 wait complete. 2024-03-19T00:38:29.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:38:29.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 -- 172.21.15.7:0/4125518832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc1002a0 msgr2=0x7fbafc109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 --2- 172.21.15.7:0/4125518832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc1002a0 0x7fbafc109e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009ca0 tx=0x7fbaec02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 -- 172.21.15.7:0/4125518832 shutdown_connections 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 --2- 172.21.15.7:0/4125518832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc1002a0 0x7fbafc109e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 --2- 172.21.15.7:0/4125518832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 0x7fbafc0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 --2- 172.21.15.7:0/4125518832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc102e70 0x7fbafc103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.907+0000 7fbb016dc640 1 -- 172.21.15.7:0/4125518832 >> 172.21.15.7:0/4125518832 conn(0x7fbafc0fb440 msgr2=0x7fbafc0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 -- 172.21.15.7:0/4125518832 shutdown_connections 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 -- 172.21.15.7:0/4125518832 wait complete. 2024-03-19T00:38:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 Processor -- start 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 -- start start 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 0x7fbafc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 0x7fbafc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc102e70 0x7fbafc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc10d380 con 0x7fbafc1002a0 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc10d200 con 0x7fbafc0ff8e0 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbb016dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc10d500 con 0x7fbafc102e70 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.908+0000 7fbafa7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 0x7fbafc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 0x7fbafc197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42926/0 (socket says 172.21.15.7:42926) 2024-03-19T00:38:29.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 -- 172.21.15.7:0/2641822746 learned_addr learned my addr 172.21.15.7:0/2641822746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafb7fe640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc102e70 0x7fbafc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafaffd640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 0x7fbafc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 -- 172.21.15.7:0/2641822746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc102e70 msgr2=0x7fbafc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc102e70 0x7fbafc19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 -- 172.21.15.7:0/2641822746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 msgr2=0x7fbafc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 0x7fbafc197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 -- 172.21.15.7:0/2641822746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc19f640 con 0x7fbafc1002a0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafaffd640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 0x7fbafc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbafa7fc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 0x7fbafc197f60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbae400e9f0 tx=0x7fbae400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae400cd80 con 0x7fbafc1002a0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc19f930 con 0x7fbafc1002a0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbae40054d0 con 0x7fbafc1002a0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4010490 con 0x7fbafc1002a0 2024-03-19T00:38:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.909+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc19fe30 con 0x7fbafc1002a0 2024-03-19T00:38:29.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.910+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc108d30 con 0x7fbafc1002a0 2024-03-19T00:38:29.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.911+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fbae4005000 con 0x7fbafc1002a0 2024-03-19T00:38:29.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.912+0000 7fbaf3fff640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbae80a2890 0x7fbae80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.912+0000 7fbafaffd640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbae80a2890 0x7fbae80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.912+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbae40cf4b0 con 0x7fbafc1002a0 2024-03-19T00:38:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.912+0000 7fbafaffd640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbae80a2890 0x7fbae80a4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbafc069ba0 tx=0x7fbaf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:29.916+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbae4053740 con 0x7fbafc1002a0 2024-03-19T00:38:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.100+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbafc02d380 con 0x7fbafc1002a0 2024-03-19T00:38:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.100+0000 7fbaf3fff640 1 -- 172.21.15.7:0/2641822746 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fbae40052f0 con 0x7fbafc1002a0 2024-03-19T00:38:30.102 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:38:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbae80a2890 msgr2=0x7fbae80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbae80a2890 0x7fbae80a4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbafc069ba0 tx=0x7fbaf4005f90 comp rx=0 tx=0).stop 2024-03-19T00:38:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 msgr2=0x7fbafc197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 0x7fbafc197f60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbae400e9f0 tx=0x7fbae400edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 shutdown_connections 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbae80a2890 0x7fbae80a4d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbafc102e70 0x7fbafc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbafc1002a0 0x7fbafc197f60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 --2- 172.21.15.7:0/2641822746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbafc0ff8e0 0x7fbafc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 >> 172.21.15.7:0/2641822746 conn(0x7fbafc0fb440 msgr2=0x7fbafc105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 shutdown_connections 2024-03-19T00:38:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.102+0000 7fbb016dc640 1 -- 172.21.15.7:0/2641822746 wait complete. 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2841108030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1779931201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:30.114 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:29 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3261235387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:30.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:38:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 -- 172.21.15.7:0/3332168805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 msgr2=0x7f42e0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 --2- 172.21.15.7:0/3332168805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0100b60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009860 tx=0x7f42d802bb70 comp rx=0 tx=0).stop 2024-03-19T00:38:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 -- 172.21.15.7:0/3332168805 shutdown_connections 2024-03-19T00:38:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 --2- 172.21.15.7:0/3332168805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01022f0 0x7f42e0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 --2- 172.21.15.7:0/3332168805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e0101930 0x7f42e0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 --2- 172.21.15.7:0/3332168805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0100b60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.219+0000 7f42e826c640 1 -- 172.21.15.7:0/3332168805 >> 172.21.15.7:0/3332168805 conn(0x7f42e00fbcb0 msgr2=0x7f42e00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 -- 172.21.15.7:0/3332168805 shutdown_connections 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 -- 172.21.15.7:0/3332168805 wait complete. 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 Processor -- start 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 -- start start 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 0x7f42e0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e01022f0 0x7f42e019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c260 con 0x7f42e0101930 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c0e0 con 0x7f42e0100740 2024-03-19T00:38:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e826c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c3e0 con 0x7f42e01022f0 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e5807640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 0x7f42e0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e5807640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 0x7f42e0198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42944/0 (socket says 172.21.15.7:42944) 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e6008640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e5807640 1 -- 172.21.15.7:0/2573985295 learned_addr learned my addr 172.21.15.7:0/2573985295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.220+0000 7f42e6809640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e01022f0 0x7f42e019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e5807640 1 -- 172.21.15.7:0/2573985295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e01022f0 msgr2=0x7f42e019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e5807640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e01022f0 0x7f42e019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e5807640 1 -- 172.21.15.7:0/2573985295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 msgr2=0x7f42e0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e5807640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e5807640 1 -- 172.21.15.7:0/2573985295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e019f740 con 0x7f42e0101930 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e6008640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e5807640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 0x7f42e0198060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f42d400e9f0 tx=0x7f42d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400cd80 con 0x7f42e0101930 2024-03-19T00:38:30.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e019fa30 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d40054d0 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4010490 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.221+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e019ff80 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.222+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e01a0210 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.223+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f42d4005020 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.223+0000 7f42df7fe640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f42c80a2890 0x7f42c80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.224+0000 7f42e6008640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f42c80a2890 0x7f42c80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.224+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f42d40cf2d0 con 0x7f42e0101930 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.224+0000 7f42e6008640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f42c80a2890 0x7f42c80a4d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42d802c180 tx=0x7f42d80349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.228+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f42d409beb0 con 0x7f42e0101930 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2841108030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1779931201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:29 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3261235387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2841108030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1779931201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:38:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:29 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3261235387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:38:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.410+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f42e01a03e0 con 0x7f42e0101930 2024-03-19T00:38:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.410+0000 7f42df7fe640 1 -- 172.21.15.7:0/2573985295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f42d409b7b0 con 0x7f42e0101930 2024-03-19T00:38:30.412 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:38:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f42c80a2890 msgr2=0x7f42c80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f42c80a2890 0x7f42c80a4d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42d802c180 tx=0x7f42d80349a0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 msgr2=0x7f42e0198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 0x7f42e0198060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f42d400e9f0 tx=0x7f42d400edf0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 shutdown_connections 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f42c80a2890 0x7f42c80a4d50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e01022f0 0x7f42e019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0101930 0x7f42e0198060 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 --2- 172.21.15.7:0/2573985295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0100740 0x7f42e0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 >> 172.21.15.7:0/2573985295 conn(0x7f42e00fbcb0 msgr2=0x7f42e0068ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 shutdown_connections 2024-03-19T00:38:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.412+0000 7f42e826c640 1 -- 172.21.15.7:0/2573985295 wait complete. 2024-03-19T00:38:30.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 -- 172.21.15.7:0/756333221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c102300 msgr2=0x7f6a8c108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 --2- 172.21.15.7:0/756333221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c102300 0x7f6a8c108b90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009be0 tx=0x7f6a8402bef0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 -- 172.21.15.7:0/756333221 shutdown_connections 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 --2- 172.21.15.7:0/756333221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c102300 0x7f6a8c108b90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 --2- 172.21.15.7:0/756333221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c101940 0x7f6a8c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 --2- 172.21.15.7:0/756333221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8c100750 0x7f6a8c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 -- 172.21.15.7:0/756333221 >> 172.21.15.7:0/756333221 conn(0x7f6a8c0fbce0 msgr2=0x7f6a8c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 -- 172.21.15.7:0/756333221 shutdown_connections 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.526+0000 7f6a90b77640 1 -- 172.21.15.7:0/756333221 wait complete. 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 Processor -- start 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 -- start start 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c100750 0x7f6a8c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8c101940 0x7f6a8c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 0x7f6a8c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c10c280 con 0x7f6a8c102300 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c10c100 con 0x7f6a8c100750 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a90b77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c10c400 con 0x7f6a8c101940 2024-03-19T00:38:30.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a8ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 0x7f6a8c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a8ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 0x7f6a8c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42956/0 (socket says 172.21.15.7:42956) 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a8a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c100750 0x7f6a8c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a8ad9d640 1 -- 172.21.15.7:0/1113715585 learned_addr learned my addr 172.21.15.7:0/1113715585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.527+0000 7f6a83fff640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8c101940 0x7f6a8c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8a59c640 1 -- 172.21.15.7:0/1113715585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8c101940 msgr2=0x7f6a8c19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8a59c640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8c101940 0x7f6a8c19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8a59c640 1 -- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 msgr2=0x7f6a8c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8a59c640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 0x7f6a8c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8a59c640 1 -- 172.21.15.7:0/1113715585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c19f640 con 0x7f6a8c100750 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8ad9d640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 0x7f6a8c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a8a59c640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c100750 0x7f6a8c06aad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00e9c0 tx=0x7f6a7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00ccc0 con 0x7f6a8c100750 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a7c0035f0 con 0x7f6a8c100750 2024-03-19T00:38:30.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c19f930 con 0x7f6a8c100750 2024-03-19T00:38:30.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00ce60 con 0x7f6a8c100750 2024-03-19T00:38:30.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.528+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c19fe80 con 0x7f6a8c100750 2024-03-19T00:38:30.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.529+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8c107a70 con 0x7f6a8c100750 2024-03-19T00:38:30.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.530+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f6a7c0037a0 con 0x7f6a8c100750 2024-03-19T00:38:30.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.531+0000 7f6a837fe640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2910 0x7f6a780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.531+0000 7f6a83fff640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2910 0x7f6a780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.531+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6a7c0cf280 con 0x7f6a8c100750 2024-03-19T00:38:30.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.531+0000 7f6a83fff640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2910 0x7f6a780a4dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c06a840 tx=0x7f6a74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:30.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.538+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6a7c09be00 con 0x7f6a8c100750 2024-03-19T00:38:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.760+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a8c02d480 con 0x7f6a8c100750 2024-03-19T00:38:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.761+0000 7f6a837fe640 1 -- 172.21.15.7:0/1113715585 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7f6a7c09b700 con 0x7f6a8c100750 2024-03-19T00:38:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:30.763 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:38:23.871158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:38:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-19T00:38:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2910 msgr2=0x7f6a780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2910 0x7f6a780a4dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c06a840 tx=0x7f6a74009260 comp rx=0 tx=0).stop 2024-03-19T00:38:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c100750 msgr2=0x7f6a8c06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c100750 0x7f6a8c06aad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00e9c0 tx=0x7f6a7c00ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 shutdown_connections 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6a780a2910 0x7f6a780a4dd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c102300 0x7f6a8c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8c101940 0x7f6a8c19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 --2- 172.21.15.7:0/1113715585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a8c100750 0x7f6a8c06aad0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 >> 172.21.15.7:0/1113715585 conn(0x7f6a8c0fbce0 msgr2=0x7f6a8c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 shutdown_connections 2024-03-19T00:38:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.763+0000 7f6a90b77640 1 -- 172.21.15.7:0/1113715585 wait complete. 2024-03-19T00:38:30.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:38:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 -- 172.21.15.7:0/4019417692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078101940 msgr2=0x7f2078101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 --2- 172.21.15.7:0/4019417692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078101940 0x7f2078101dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2068009900 tx=0x7f206802bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 -- 172.21.15.7:0/4019417692 shutdown_connections 2024-03-19T00:38:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 --2- 172.21.15.7:0/4019417692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2078102300 0x7f2078108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 --2- 172.21.15.7:0/4019417692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078101940 0x7f2078101dc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 --2- 172.21.15.7:0/4019417692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2078100750 0x7f2078100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 -- 172.21.15.7:0/4019417692 >> 172.21.15.7:0/4019417692 conn(0x7f20780fbcc0 msgr2=0x7f20780fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 -- 172.21.15.7:0/4019417692 shutdown_connections 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.877+0000 7f20804ea640 1 -- 172.21.15.7:0/4019417692 wait complete. 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 Processor -- start 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 -- start start 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 0x7f2078197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2078101940 0x7f2078197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2078102300 0x7f207819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f207810c260 con 0x7f2078100750 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f207810c0e0 con 0x7f2078101940 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f20804ea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f207810c3e0 con 0x7f2078102300 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 0x7f2078197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 0x7f2078197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42968/0 (socket says 172.21.15.7:42968) 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 -- 172.21.15.7:0/2493916164 learned_addr learned my addr 172.21.15.7:0/2493916164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207da85640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2078101940 0x7f2078197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207ea87640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2078102300 0x7f207819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 -- 172.21.15.7:0/2493916164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2078102300 msgr2=0x7f207819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2078102300 0x7f207819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 -- 172.21.15.7:0/2493916164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2078101940 msgr2=0x7f2078197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2078101940 0x7f2078197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 -- 172.21.15.7:0/2493916164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207819f6d0 con 0x7f2078100750 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.878+0000 7f207e286640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 0x7f2078197ab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f207400e9c0 tx=0x7f207400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.879+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207400cd80 con 0x7f2078100750 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.879+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20740035f0 con 0x7f2078100750 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.879+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207819f9c0 con 0x7f2078100750 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.879+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074010690 con 0x7f2078100750 2024-03-19T00:38:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.879+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207819fee0 con 0x7f2078100750 2024-03-19T00:38:30.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.880+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20781a0170 con 0x7f2078100750 2024-03-19T00:38:30.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.881+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f20740037a0 con 0x7f2078100750 2024-03-19T00:38:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.881+0000 7f206f7fe640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f20640a2890 0x7f20640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.881+0000 7f207da85640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f20640a2890 0x7f20640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.881+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f20740cf2e0 con 0x7f2078100750 2024-03-19T00:38:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.882+0000 7f207da85640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f20640a2890 0x7f20640a4d50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2068009230 tx=0x7f2068005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:30.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:30.886+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f207409bec0 con 0x7f2078100750 2024-03-19T00:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.071+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20781a03e0 con 0x7f2078100750 2024-03-19T00:38:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.073+0000 7f206f7fe640 1 -- 172.21.15.7:0/2493916164 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f207409b7c0 con 0x7f2078100750 2024-03-19T00:38:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:38:31.075 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:38:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f20640a2890 msgr2=0x7f20640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f20640a2890 0x7f20640a4d50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2068009230 tx=0x7f2068005fd0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 msgr2=0x7f2078197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 0x7f2078197ab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f207400e9c0 tx=0x7f207400ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 shutdown_connections 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f20640a2890 0x7f20640a4d50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2078102300 0x7f207819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2078101940 0x7f2078197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 --2- 172.21.15.7:0/2493916164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2078100750 0x7f2078197ab0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 >> 172.21.15.7:0/2493916164 conn(0x7f20780fbcc0 msgr2=0x7f20780fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 shutdown_connections 2024-03-19T00:38:31.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.075+0000 7f20804ea640 1 -- 172.21.15.7:0/2493916164 wait complete. 2024-03-19T00:38:31.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:30 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2641822746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:31.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:30 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2573985295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:31.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:30 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1113715585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:31.089 DEBUG:tasks.fwd_scrub:Yielding 2024-03-19T00:38:31.089 INFO:teuthology.run_tasks:Running task workunit... 2024-03-19T00:38:31.101 INFO:tasks.workunit:Pulling workunits from ref e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-19T00:38:31.101 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-19T00:38:31.101 INFO:tasks.workunit:timeout=3h 2024-03-19T00:38:31.101 INFO:tasks.workunit:cleanup=True 2024-03-19T00:38:31.102 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:31.102 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-19T00:38:31.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:38:31.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:38:31.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:38:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-19 00:36:27.006329336 +0000 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-19 00:36:27.006329336 +0000 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-19 00:38:23.489437575 +0000 2024-03-19T00:38:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-03-19T00:38:31.118 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-19T00:38:31.118 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-19T00:38:31.191 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-19T00:38:31.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- 172.21.15.7:0/2324414071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c101830 msgr2=0x7fc84c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- 172.21.15.7:0/2324414071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c101830 0x7fc84c101c50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc840009900 tx=0x7fc84002bc30 comp rx=0 tx=0).stop 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- 172.21.15.7:0/2324414071 shutdown_connections 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- 172.21.15.7:0/2324414071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc84c1033e0 0x7fc84c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- 172.21.15.7:0/2324414071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c102a20 0x7fc84c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- 172.21.15.7:0/2324414071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c101830 0x7fc84c101c50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- 172.21.15.7:0/2324414071 >> 172.21.15.7:0/2324414071 conn(0x7fc84c0fcda0 msgr2=0x7fc84c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- 172.21.15.7:0/2324414071 shutdown_connections 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- 172.21.15.7:0/2324414071 wait complete. 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 Processor -- start 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- start start 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c101830 0x7fc84c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc84c102a20 0x7fc84c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 0x7fc84c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c10d360 con 0x7fc84c1033e0 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c10d1e0 con 0x7fc84c101830 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc852f37640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c10d4e0 con 0x7fc84c102a20 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.210+0000 7fc8514d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 0x7fc84c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 0x7fc84c1a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42996/0 (socket says 172.21.15.7:42996) 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 -- 172.21.15.7:0/2613777952 learned_addr learned my addr 172.21.15.7:0/2613777952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc84bfff640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc84c102a20 0x7fc84c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc850cd3640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c101830 0x7fc84c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 -- 172.21.15.7:0/2613777952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc84c102a20 msgr2=0x7fc84c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc84c102a20 0x7fc84c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 -- 172.21.15.7:0/2613777952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c101830 msgr2=0x7fc84c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c101830 0x7fc84c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 -- 172.21.15.7:0/2613777952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c1a1900 con 0x7fc84c1033e0 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc8514d4640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 0x7fc84c1a1260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc84400eb10 tx=0x7fc84400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc849ffb640 1 -- 172.21.15.7:0/2613777952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844004b20 con 0x7fc84c1033e0 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc849ffb640 1 -- 172.21.15.7:0/2613777952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc844004cc0 con 0x7fc84c1033e0 2024-03-19T00:38:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc849ffb640 1 -- 172.21.15.7:0/2613777952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844010690 con 0x7fc84c1033e0 2024-03-19T00:38:31.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.211+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c1a1bf0 con 0x7fc84c1033e0 2024-03-19T00:38:31.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.212+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c1a20c0 con 0x7fc84c1033e0 2024-03-19T00:38:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.214+0000 7fc849ffb640 1 -- 172.21.15.7:0/2613777952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fc844005020 con 0x7fc84c1033e0 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.214+0000 7fc855bfa740 1 Processor -- start 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.214+0000 7fc855bfa740 1 -- start start 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484e7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc855bfa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ba48444160 con 0x55ba484e4f00 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc855bfa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba483ece10 con 0x55ba484dfd90 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc855bfa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ba48440200 con 0x55ba484df3a0 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484e4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54724/0 (socket says 172.21.15.7:54724) 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 -- 172.21.15.7:0/3797051679 learned_addr learned my addr 172.21.15.7:0/3797051679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc849ffb640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc83c0a2910 0x7fc83c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc84bfff640 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc8514d4640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc83c0a2910 0x7fc83c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 -- 172.21.15.7:0/3797051679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 msgr2=0x55ba484df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 -- 172.21.15.7:0/3797051679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 msgr2=0x55ba484e7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484e7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 -- 172.21.15.7:0/3797051679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba482207e0 con 0x55ba484dfd90 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc84bfff640 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484e7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc849ffb640 1 -- 172.21.15.7:0/2613777952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc8440cf620 con 0x7fc84c1033e0 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.215+0000 7fc850cd3640 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484e4890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc840009900 tx=0x7fc8400095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a9fadc9c84c7f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc8514d4640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc83c0a2910 0x7fc83c0a4dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc844099b80 tx=0x7fc84407f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc836ffd640 1 -- 172.21.15.7:0/3797051679 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840045020 con 0x55ba484dfd90 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc836ffd640 1 -- 172.21.15.7:0/3797051679 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc840004040 con 0x55ba484dfd90 2024-03-19T00:38:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc836ffd640 1 -- 172.21.15.7:0/3797051679 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840005580 con 0x55ba484dfd90 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc855bfa740 1 -- 172.21.15.7:0/3797051679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 msgr2=0x55ba484e4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc855bfa740 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484e4890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc840009900 tx=0x7fc8400095f0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc855bfa740 1 -- 172.21.15.7:0/3797051679 shutdown_connections 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc855bfa740 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484e7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc855bfa740 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484e4890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.216+0000 7fc855bfa740 1 --2- 172.21.15.7:0/3797051679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.217+0000 7fc855bfa740 1 -- 172.21.15.7:0/3797051679 >> 172.21.15.7:0/3797051679 conn(0x55ba484da910 msgr2=0x55ba484dad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.217+0000 7fc855bfa740 1 -- 172.21.15.7:0/3797051679 shutdown_connections 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.217+0000 7fc855bfa740 1 -- 172.21.15.7:0/3797051679 wait complete. 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.217+0000 7fc855bfa740 1 -- start start 2024-03-19T00:38:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.217+0000 7fc855bfa740 1 Processor -- start 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 0x55ba484e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484dfd90 0x55ba484e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ba484e7e50 con 0x55ba484e4f00 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba484001a0 con 0x55ba484df3a0 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ba484e7fd0 con 0x55ba484dfd90 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc855bfa740 10 client.0 _ll_register_callbacks cb 0x55ba48157c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 0x55ba484e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 0x55ba484e9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54728/0 (socket says 172.21.15.7:54728) 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 -- 172.21.15.7:0/27379614 learned_addr learned my addr 172.21.15.7:0/27379614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc850cd3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484dfd90 0x55ba484e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 -- 172.21.15.7:0/27379614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484dfd90 msgr2=0x55ba484e9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484dfd90 0x55ba484e9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 -- 172.21.15.7:0/27379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 msgr2=0x55ba484f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 -- 172.21.15.7:0/27379614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba48280f70 con 0x55ba484df3a0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc850cd3640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484dfd90 0x55ba484e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc84bfff640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 0x55ba484e9790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838011f40 tx=0x7fc838004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521d7201cddb5568 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc82bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc82bfff640 1 -- 172.21.15.7:0/27379614 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838005020 con 0x55ba484df3a0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.218+0000 7fc82bfff640 1 -- 172.21.15.7:0/27379614 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc838009040 con 0x55ba484df3a0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc840004040 con 0x55ba484df3a0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc82bfff640 1 -- 172.21.15.7:0/27379614 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc83802b050 con 0x55ba484df3a0 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 10 client.25475 fetch_fsmap learned FSMap version 25 2024-03-19T00:38:31.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba48118170 con 0x55ba484df3a0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc82bfff640 1 -- 172.21.15.7:0/27379614 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc840004040 con 0x55ba484df3a0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc82bfff640 1 -- 172.21.15.7:0/27379614 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc838020040 con 0x55ba484df3a0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 10 client.25475 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 20 client.25475 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba484f55c0 0x55ba484f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 4 client.25475 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba484516a0 con 0x55ba484f55c0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.219+0000 7fc850cd3640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba484f55c0 0x55ba484f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.220+0000 7fc82bfff640 10 client.25475 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.220+0000 7fc850cd3640 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba484f55c0 0x55ba484f7a60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff29d051a7b40593 server_cookie=b60f0930d20f831a in_seq=0 out_seq=0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc82bfff640 1 -- 172.21.15.7:0/27379614 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ba484516a0 con 0x55ba484f55c0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc82bfff640 10 client.25475 handle_command_reply: tid=0 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 client.25475 shutdown 2024-03-19T00:38:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 msgr2=0x55ba484e9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 0x55ba484e9790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838011f40 tx=0x7fc838004730 comp rx=0 tx=0).stop 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 shutdown_connections 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba484f55c0 0x55ba484f7a60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484dfd90 0x55ba484e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc82bfff640 0 client.25475 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 --2- 172.21.15.7:0/27379614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484df3a0 0x55ba484e9790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 >> 172.21.15.7:0/27379614 conn(0x55ba484da910 msgr2=0x55ba484d9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 shutdown_connections 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 1 -- 172.21.15.7:0/27379614 wait complete. 2024-03-19T00:38:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.221+0000 7fc855bfa740 20 client.25475 trim_cache size 0 max 16384 2024-03-19T00:38:31.229 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 Processor -- start 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 -- start start 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 0x55ba484f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484e4f00 0x55ba484de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1aed20 con 0x55ba484dfd90 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1ac5f0 con 0x55ba484e4f00 2024-03-19T00:38:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc855bfa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1aa440 con 0x55ba484df3a0 2024-03-19T00:38:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 0x55ba484f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 0x55ba484f0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43016/0 (socket says 172.21.15.7:43016) 2024-03-19T00:38:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 -- 172.21.15.7:0/2677175691 learned_addr learned my addr 172.21.15.7:0/2677175691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc84bfff640 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484e4f00 0x55ba484de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 -- 172.21.15.7:0/2677175691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 msgr2=0x55ba484e9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484e9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 -- 172.21.15.7:0/2677175691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484e4f00 msgr2=0x55ba484de280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484e4f00 0x55ba484de280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 -- 172.21.15.7:0/2677175691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba484da490 con 0x55ba484dfd90 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc84bfff640 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484e4f00 0x55ba484de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.241+0000 7fc850cd3640 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 0x55ba484f0cd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc840005ab0 tx=0x7fc840005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00117024f195de4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc8367fc640 1 -- 172.21.15.7:0/2677175691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840045040 con 0x55ba484dfd90 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc8367fc640 1 -- 172.21.15.7:0/2677175691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc84002cae0 con 0x55ba484dfd90 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc8367fc640 1 -- 172.21.15.7:0/2677175691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840041d50 con 0x55ba484dfd90 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 -- 172.21.15.7:0/2677175691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 msgr2=0x55ba484f0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 0x55ba484f0cd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc840005ab0 tx=0x7fc840005b50 comp rx=0 tx=0).stop 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 -- 172.21.15.7:0/2677175691 shutdown_connections 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484e4f00 0x55ba484de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484dfd90 0x55ba484f0cd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 --2- 172.21.15.7:0/2677175691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484e9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 -- 172.21.15.7:0/2677175691 >> 172.21.15.7:0/2677175691 conn(0x55ba484da910 msgr2=0x55ba484ddb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.242+0000 7fc855bfa740 1 -- 172.21.15.7:0/2677175691 shutdown_connections 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 -- 172.21.15.7:0/2677175691 wait complete. 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 -- start start 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 Processor -- start 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484ef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1aaab0 con 0x55ba484e4f00 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1abf20 con 0x55ba484dfd90 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1ab180 con 0x55ba484df3a0 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc855bfa740 10 client.0 _ll_register_callbacks cb 0x7fc838009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc850cd3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43018/0 (socket says 172.21.15.7:43018) 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc84bfff640 1 -- 172.21.15.7:0/1177560706 learned_addr learned my addr 172.21.15.7:0/1177560706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.243+0000 7fc84bfff640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc84bfff640 1 -- 172.21.15.7:0/1177560706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 msgr2=0x55ba484eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc84bfff640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc84bfff640 1 -- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 msgr2=0x55ba484ef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc84bfff640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484ef080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc84bfff640 1 -- 172.21.15.7:0/1177560706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba484f0630 con 0x55ba484e4f00 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc850cd3640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484ef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc84bfff640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc838004ec0 tx=0x7fc838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac1e00811b196a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc82bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc82bfff640 1 -- 172.21.15.7:0/1177560706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380049b0 con 0x55ba484e4f00 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc82bfff640 1 -- 172.21.15.7:0/1177560706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc83800e990 con 0x55ba484e4f00 2024-03-19T00:38:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84002cae0 con 0x55ba484e4f00 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc82bfff640 1 -- 172.21.15.7:0/1177560706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc838040050 con 0x55ba484e4f00 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 10 client.15630 fetch_fsmap learned FSMap version 25 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba4855c050 con 0x55ba484e4f00 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc82bfff640 1 -- 172.21.15.7:0/1177560706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc84002cae0 con 0x55ba484e4f00 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc82bfff640 1 -- 172.21.15.7:0/1177560706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc83803b030 con 0x55ba484e4f00 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 10 client.15630 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 20 client.15630 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.244+0000 7fc855bfa740 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba4855ca10 0x55ba4855ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.245+0000 7fc855bfa740 4 client.15630 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.245+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba4844b9f0 con 0x55ba4855ca10 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.245+0000 7fc850cd3640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba4855ca10 0x55ba4855ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.245+0000 7fc82bfff640 10 client.15630 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.245+0000 7fc850cd3640 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba4855ca10 0x55ba4855ee30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a1e73e6623e47c server_cookie=eb61f026a836e235 in_seq=0 out_seq=0 2024-03-19T00:38:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc82bfff640 1 -- 172.21.15.7:0/1177560706 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba4844b9f0 con 0x55ba4855ca10 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc82bfff640 10 client.15630 handle_command_reply: tid=0 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc855bfa740 1 client.15630 shutdown 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 msgr2=0x55ba484f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc855bfa740 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc838004ec0 tx=0x7fc838004730 comp rx=0 tx=0).stop 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 shutdown_connections 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.246+0000 7fc855bfa740 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba4855ca10 0x55ba4855ee30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba484e4f00 0x55ba484f0040 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba484dfd90 0x55ba484ef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 1 --2- 172.21.15.7:0/1177560706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba484df3a0 0x55ba484eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 >> 172.21.15.7:0/1177560706 conn(0x55ba484da910 msgr2=0x55ba484de400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc82bfff640 0 client.15630 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 shutdown_connections 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 1 -- 172.21.15.7:0/1177560706 wait complete. 2024-03-19T00:38:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc855bfa740 20 client.15630 trim_cache size 0 max 16384 2024-03-19T00:38:31.257 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:38:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:38:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "212c98fc-8249-45e0-b31f-ca5121fa85a7", 2024-03-19T00:38:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:38:31.257 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc83c0a2910 msgr2=0x7fc83c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc83c0a2910 0x7fc83c0a4dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc844099b80 tx=0x7fc84407f040 comp rx=0 tx=0).stop 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 msgr2=0x7fc84c1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 0x7fc84c1a1260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc84400eb10 tx=0x7fc84400c2d0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 shutdown_connections 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc83c0a2910 0x7fc83c0a4dd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c1033e0 0x7fc84c1a1260 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc84c102a20 0x7fc84c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 --2- 172.21.15.7:0/2613777952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c101830 0x7fc84c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 >> 172.21.15.7:0/2613777952 conn(0x7fc84c0fcda0 msgr2=0x7fc84c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 shutdown_connections 2024-03-19T00:38:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.247+0000 7fc852f37640 1 -- 172.21.15.7:0/2613777952 wait complete. 2024-03-19T00:38:31.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:38:31.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:38:31.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:30 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2641822746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:30 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2573985295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:31.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:30 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1113715585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:31.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:30 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2641822746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:38:31.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:30 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2573985295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:38:31.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:30 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1113715585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:38:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 -- 172.21.15.7:0/1443936683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90401034a0 msgr2=0x7f9040109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 --2- 172.21.15.7:0/1443936683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90401034a0 0x7f9040109d30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9038009cd0 tx=0x7f903802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 -- 172.21.15.7:0/1443936683 shutdown_connections 2024-03-19T00:38:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 --2- 172.21.15.7:0/1443936683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90401034a0 0x7f9040109d30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 --2- 172.21.15.7:0/1443936683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102ae0 0x7f9040102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 --2- 172.21.15.7:0/1443936683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90400ffb10 0x7f90400fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 -- 172.21.15.7:0/1443936683 >> 172.21.15.7:0/1443936683 conn(0x7f90400fb440 msgr2=0x7f90400fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 -- 172.21.15.7:0/1443936683 shutdown_connections 2024-03-19T00:38:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.406+0000 7f9045916640 1 -- 172.21.15.7:0/1443936683 wait complete. 2024-03-19T00:38:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 Processor -- start 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 -- start start 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 0x7f9040197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102ae0 0x7f9040197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90401034a0 0x7f904019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f904010d3a0 con 0x7f90400ffb10 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f904010d220 con 0x7f90401034a0 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f9045916640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f904010d520 con 0x7f9040102ae0 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 0x7f9040197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 0x7f9040197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43046/0 (socket says 172.21.15.7:43046) 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 -- 172.21.15.7:0/640391014 learned_addr learned my addr 172.21.15.7:0/640391014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903f7fe640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90401034a0 0x7f904019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903e7fc640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102ae0 0x7f9040197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 -- 172.21.15.7:0/640391014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102ae0 msgr2=0x7f9040197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102ae0 0x7f9040197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 -- 172.21.15.7:0/640391014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90401034a0 msgr2=0x7f904019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90401034a0 0x7f904019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 -- 172.21.15.7:0/640391014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904019f670 con 0x7f90400ffb10 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.407+0000 7f903effd640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 0x7f9040197a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f903000e9c0 tx=0x7f903000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.408+0000 7f9044914640 1 -- 172.21.15.7:0/640391014 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903000cd80 con 0x7f90400ffb10 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.408+0000 7f9044914640 1 -- 172.21.15.7:0/640391014 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90300035f0 con 0x7f90400ffb10 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.408+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904019f960 con 0x7f90400ffb10 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.408+0000 7f9044914640 1 -- 172.21.15.7:0/640391014 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030010690 con 0x7f90400ffb10 2024-03-19T00:38:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.408+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904019ff00 con 0x7f90400ffb10 2024-03-19T00:38:31.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f9044914640 1 -- 172.21.15.7:0/640391014 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f90300037a0 con 0x7f90400ffb10 2024-03-19T00:38:31.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 Processor -- start 2024-03-19T00:38:31.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 -- start start 2024-03-19T00:38:31.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f9044914640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f90280a2910 0x7f90280a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f65e050500 con 0x55f65e0ec440 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f65dff8be0 con 0x55f65e0f15b0 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f90485d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f65e04c5a0 con 0x55f65e0eba50 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f903e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f903f7fe640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f90280a2910 0x7f90280a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f903e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43062/0 (socket says 172.21.15.7:43062) 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f903e7fc640 1 -- 172.21.15.7:0/3492527548 learned_addr learned my addr 172.21.15.7:0/3492527548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f903effd640 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f9044914640 1 -- 172.21.15.7:0/640391014 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f90300cf260 con 0x7f90400ffb10 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.410+0000 7f903f7fe640 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903e7fc640 1 -- 172.21.15.7:0/3492527548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 msgr2=0x55f65e0ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903e7fc640 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0ebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903e7fc640 1 -- 172.21.15.7:0/3492527548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 msgr2=0x55f65e0f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903e7fc640 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903e7fc640 1 -- 172.21.15.7:0/3492527548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65e01a000 con 0x55f65e0ec440 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903effd640 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903e7fc640 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0f0f40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9034009ca0 tx=0x7f9034011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b55bdac86d9e01e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f903f7fe640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f90280a2910 0x7f90280a4dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9038009ca0 tx=0x7f9038009970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f9025ffb640 1 -- 172.21.15.7:0/3492527548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903401e070 con 0x55f65e0ec440 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f9025ffb640 1 -- 172.21.15.7:0/3492527548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9034019ae0 con 0x55f65e0ec440 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.411+0000 7f9025ffb640 1 -- 172.21.15.7:0/3492527548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903400e900 con 0x55f65e0ec440 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- 172.21.15.7:0/3492527548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 msgr2=0x55f65e0f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0f0f40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9034009ca0 tx=0x7f9034011810 comp rx=0 tx=0).stop 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- 172.21.15.7:0/3492527548 shutdown_connections 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0f0f40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- 172.21.15.7:0/3492527548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- 172.21.15.7:0/3492527548 >> 172.21.15.7:0/3492527548 conn(0x55f65e0e6fc0 msgr2=0x55f65e0e73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- 172.21.15.7:0/3492527548 shutdown_connections 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- 172.21.15.7:0/3492527548 wait complete. 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- start start 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 Processor -- start 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0ec440 0x55f65e0f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f65e0f4560 con 0x55f65e0f15b0 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.412+0000 7f90485d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f65e00c160 con 0x55f65e0ec440 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f90485d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f65e0f46e0 con 0x55f65e0eba50 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f90485d6740 10 client.0 _ll_register_callbacks cb 0x55f65e048a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43074/0 (socket says 172.21.15.7:43074) 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 -- 172.21.15.7:0/1439584426 learned_addr learned my addr 172.21.15.7:0/1439584426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0ec440 0x55f65e0f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 -- 172.21.15.7:0/1439584426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 msgr2=0x55f65e0f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 -- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0ec440 msgr2=0x55f65e0f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0ec440 0x55f65e0f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 -- 172.21.15.7:0/1439584426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65de2e7d0 con 0x55f65e0f15b0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f903f7fe640 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0fd420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9038009cd0 tx=0x7f903802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6e7468c982e24f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f901effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f901effd640 1 -- 172.21.15.7:0/1439584426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038039040 con 0x55f65e0f15b0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f901effd640 1 -- 172.21.15.7:0/1439584426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9038004590 con 0x55f65e0f15b0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9034019ae0 con 0x55f65e0f15b0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f901effd640 1 -- 172.21.15.7:0/1439584426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9038059050 con 0x55f65e0f15b0 2024-03-19T00:38:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f90485d6740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.413+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f65dd643f0 con 0x55f65e0f15b0 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f901effd640 1 -- 172.21.15.7:0/1439584426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9034019ae0 con 0x55f65e0f15b0 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f901effd640 1 -- 172.21.15.7:0/1439584426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f903803f030 con 0x55f65e0f15b0 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f90485d6740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f90485d6740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f90485d6740 20 client.15642 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f90485d6740 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e101d40 0x55f65e104200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f90485d6740 4 client.15642 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f65e05da40 con 0x55f65e101d40 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f903effd640 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e101d40 0x55f65e104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f901effd640 10 client.15642 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.414+0000 7f903effd640 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e101d40 0x55f65e104200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b680fa36c16f697 server_cookie=f4b15d46fe0418a7 in_seq=0 out_seq=0 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f901effd640 1 -- 172.21.15.7:0/1439584426 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f65e05da40 con 0x55f65e101d40 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f901effd640 10 client.15642 handle_command_reply: tid=0 2024-03-19T00:38:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 client.15642 shutdown 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 msgr2=0x55f65e0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0fd420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9038009cd0 tx=0x7f903802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 shutdown_connections 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e101d40 0x55f65e104200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0fd420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0ec440 0x55f65e0f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f901effd640 0 client.15642 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 --2- 172.21.15.7:0/1439584426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 >> 172.21.15.7:0/1439584426 conn(0x55f65e0e6fc0 msgr2=0x55f65e0e61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 shutdown_connections 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 1 -- 172.21.15.7:0/1439584426 wait complete. 2024-03-19T00:38:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.415+0000 7f90485d6740 20 client.15642 trim_cache size 0 max 16384 2024-03-19T00:38:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 Processor -- start 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 -- start start 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 0x55f65e0e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0ec440 0x55f65e0fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90401acb90 con 0x55f65e0f15b0 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90401aa460 con 0x55f65e0eba50 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f90485d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90401a82b0 con 0x55f65e0ec440 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 0x55f65e0e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0ea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43090/0 (socket says 172.21.15.7:43090) 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 0x55f65e0e21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54802/0 (socket says 172.21.15.7:54802) 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0ec440 0x55f65e0fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 -- 172.21.15.7:0/3652015284 learned_addr learned my addr 172.21.15.7:0/3652015284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 -- 172.21.15.7:0/3652015284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0ec440 msgr2=0x55f65e0fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0ec440 0x55f65e0fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 -- 172.21.15.7:0/3652015284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 msgr2=0x55f65e0e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 0x55f65e0e21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 -- 172.21.15.7:0/3652015284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65dd240f0 con 0x55f65e0f15b0 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903effd640 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 0x55f65e0e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.433+0000 7f903f7fe640 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0ea930 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9038009ba0 tx=0x7f90380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f502ee0852aaf56a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90257fa640 1 -- 172.21.15.7:0/3652015284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038039070 con 0x55f65e0f15b0 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90257fa640 1 -- 172.21.15.7:0/3652015284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9038043450 con 0x55f65e0f15b0 2024-03-19T00:38:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90257fa640 1 -- 172.21.15.7:0/3652015284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038034440 con 0x55f65e0f15b0 2024-03-19T00:38:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 -- 172.21.15.7:0/3652015284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 msgr2=0x55f65e0ea930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0ea930 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9038009ba0 tx=0x7f90380095c0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 -- 172.21.15.7:0/3652015284 shutdown_connections 2024-03-19T00:38:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0f15b0 0x55f65e0ea930 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0ec440 0x55f65e0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 --2- 172.21.15.7:0/3652015284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0eba50 0x55f65e0e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 -- 172.21.15.7:0/3652015284 >> 172.21.15.7:0/3652015284 conn(0x55f65e0e6fc0 msgr2=0x55f65e0ea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 -- 172.21.15.7:0/3652015284 shutdown_connections 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.434+0000 7f90485d6740 1 -- 172.21.15.7:0/3652015284 wait complete. 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 -- start start 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 Processor -- start 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0faf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90401a8920 con 0x55f65e0ec440 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90401a9d90 con 0x55f65e0f15b0 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90401a8ff0 con 0x55f65e0eba50 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f90485d6740 10 client.0 _ll_register_callbacks cb 0x7f9034005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f903e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f903e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43102/0 (socket says 172.21.15.7:43102) 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f903e7fc640 1 -- 172.21.15.7:0/2755571984 learned_addr learned my addr 172.21.15.7:0/2755571984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f903effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.435+0000 7f903f7fe640 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f903e7fc640 1 -- 172.21.15.7:0/2755571984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 msgr2=0x55f65e0faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f903e7fc640 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0faf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f903e7fc640 1 -- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 msgr2=0x55f65e0fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f903e7fc640 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f903e7fc640 1 -- 172.21.15.7:0/2755571984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65e0fc9d0 con 0x55f65e0ec440 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f903e7fc640 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0fb4b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9034009ca0 tx=0x7f9034005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ffd261309e1ea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f901effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f901effd640 1 -- 172.21.15.7:0/2755571984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034029070 con 0x55f65e0ec440 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f901effd640 1 -- 172.21.15.7:0/2755571984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f90340055c0 con 0x55f65e0ec440 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9038043450 con 0x55f65e0ec440 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f901effd640 1 -- 172.21.15.7:0/2755571984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9034035050 con 0x55f65e0ec440 2024-03-19T00:38:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 10 client.15648 fetch_fsmap learned FSMap version 25 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f65e168530 con 0x55f65e0ec440 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f901effd640 1 -- 172.21.15.7:0/2755571984 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9038043450 con 0x55f65e0ec440 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f901effd640 1 -- 172.21.15.7:0/2755571984 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9034004ad0 con 0x55f65e0ec440 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 20 client.15648 populate_metadata read hostname 'smithi007' 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e168cc0 0x55f65e16b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 4 client.15648 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.436+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f65e057d90 con 0x55f65e168cc0 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f903effd640 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e168cc0 0x55f65e16b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f903effd640 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e168cc0 0x55f65e16b0e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d981f20dde7091a5 server_cookie=ef9b21cca5273286 in_seq=0 out_seq=0 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f901effd640 10 client.15648 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f901effd640 1 -- 172.21.15.7:0/2755571984 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f65e057d90 con 0x55f65e168cc0 2024-03-19T00:38:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f901effd640 10 client.15648 handle_command_reply: tid=0 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 client.15648 shutdown 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 msgr2=0x55f65e0fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0fb4b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9034009ca0 tx=0x7f9034005040 comp rx=0 tx=0).stop 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 shutdown_connections 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f65e0f15b0 0x55f65e0fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f65e168cc0 0x55f65e16b0e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f65e0ec440 0x55f65e0fb4b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 --2- 172.21.15.7:0/2755571984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f65e0eba50 0x55f65e0faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f901effd640 0 client.15648 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 >> 172.21.15.7:0/2755571984 conn(0x55f65e0e6fc0 msgr2=0x55f65e0eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.437+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 shutdown_connections 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f90485d6740 1 -- 172.21.15.7:0/2755571984 wait complete. 2024-03-19T00:38:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f90485d6740 20 client.15648 trim_cache size 0 max 16384 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: "212c98fc-8249-45e0-b31f-ca5121fa85a7": { 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "212c98fc-8249-45e0-b31f-ca5121fa85a7", 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:38:31.446 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:38:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f90280a2910 msgr2=0x7f90280a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f90280a2910 0x7f90280a4dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9038009ca0 tx=0x7f9038009970 comp rx=0 tx=0).stop 2024-03-19T00:38:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 msgr2=0x7f9040197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:38:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 0x7f9040197a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f903000e9c0 tx=0x7f903000ee80 comp rx=0 tx=0).stop 2024-03-19T00:38:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 shutdown_connections 2024-03-19T00:38:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f90280a2910 0x7f90280a4dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90401034a0 0x7f904019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9040102ae0 0x7f9040197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 --2- 172.21.15.7:0/640391014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90400ffb10 0x7f9040197a50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:38:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 >> 172.21.15.7:0/640391014 conn(0x7f90400fb440 msgr2=0x7f9040105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:38:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 shutdown_connections 2024-03-19T00:38:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:38:31.438+0000 7f9045916640 1 -- 172.21.15.7:0/640391014 wait complete. 2024-03-19T00:38:31.451 INFO:tasks.cephfs.filesystem:scrub status for tag:212c98fc-8249-45e0-b31f-ca5121fa85a7 - {'path': '/', 'tag': '212c98fc-8249-45e0-b31f-ca5121fa85a7', 'options': 'recursive,force'} 2024-03-19T00:38:32.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:31 smithi007 ceph-mon[62854]: pgmap v368: 129 pgs: 129 active+clean; 652 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-19T00:38:32.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:31 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2493916164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:31 smithi204 ceph-mon[66572]: pgmap v368: 129 pgs: 129 active+clean; 652 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-19T00:38:32.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:31 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2493916164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:32.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:31 smithi044 ceph-mon[66366]: pgmap v368: 129 pgs: 129 active+clean; 652 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-19T00:38:32.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:31 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2493916164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:38:33.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:32 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:38:33.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:32 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:38:33.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:32 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:38:33.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:32 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:38:33.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:32 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:38:33.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:32 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:38:33.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:32 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:38:33.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:32 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:38:33.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:32 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:38:33.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:32 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:38:33.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:32 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:38:33.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:32 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:38:34.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:33 smithi007 ceph-mon[62854]: pgmap v369: 129 pgs: 129 active+clean; 652 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-03-19T00:38:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:33 smithi204 ceph-mon[66572]: pgmap v369: 129 pgs: 129 active+clean; 652 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-03-19T00:38:34.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:33 smithi044 ceph-mon[66366]: pgmap v369: 129 pgs: 129 active+clean; 652 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-03-19T00:38:36.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:35 smithi007 ceph-mon[62854]: pgmap v370: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-19T00:38:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:35 smithi204 ceph-mon[66572]: pgmap v370: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-19T00:38:36.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:35 smithi044 ceph-mon[66366]: pgmap v370: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-19T00:38:38.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:37 smithi007 ceph-mon[62854]: pgmap v371: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-19T00:38:38.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:37 smithi204 ceph-mon[66572]: pgmap v371: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-19T00:38:38.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:37 smithi044 ceph-mon[66366]: pgmap v371: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-19T00:38:39.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:38 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:38:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:38 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:38:39.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:38 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:38:40.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:39 smithi007 ceph-mon[62854]: pgmap v372: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-19T00:38:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:39 smithi204 ceph-mon[66572]: pgmap v372: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-19T00:38:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:39 smithi044 ceph-mon[66366]: pgmap v372: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-19T00:38:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: pgmap v373: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-19T00:38:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:38:41.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:38:41.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:41 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: pgmap v373: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-19T00:38:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:38:41.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:38:41.717 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:41 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: pgmap v373: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-19T00:38:41.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:38:41.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:38:41.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:41 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:38:43.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:43 smithi044 ceph-mon[66366]: pgmap v374: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-19T00:38:43.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:43 smithi007 ceph-mon[62854]: pgmap v374: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-19T00:38:43.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:43 smithi204 ceph-mon[66572]: pgmap v374: 129 pgs: 129 active+clean; 659 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-19T00:38:45.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:45 smithi044 ceph-mon[66366]: pgmap v375: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-19T00:38:45.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:45 smithi007 ceph-mon[62854]: pgmap v375: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-19T00:38:45.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:45 smithi204 ceph-mon[66572]: pgmap v375: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-19T00:38:46.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:38:46.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:38:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:38:47.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:47 smithi044 ceph-mon[66366]: pgmap v376: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-19T00:38:47.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:47 smithi007 ceph-mon[62854]: pgmap v376: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-19T00:38:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:47 smithi204 ceph-mon[66572]: pgmap v376: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-19T00:38:49.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:49 smithi044 ceph-mon[66366]: pgmap v377: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-19T00:38:49.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:49 smithi007 ceph-mon[62854]: pgmap v377: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-19T00:38:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:49 smithi204 ceph-mon[66572]: pgmap v377: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-19T00:38:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:51 smithi007 ceph-mon[62854]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-19T00:38:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:51 smithi204 ceph-mon[66572]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-19T00:38:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:51 smithi044 ceph-mon[66366]: pgmap v378: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-19T00:38:53.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:53 smithi204 ceph-mon[66572]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:53.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:53 smithi007 ceph-mon[62854]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:53 smithi044 ceph-mon[66366]: pgmap v379: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-19T00:38:55.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:55 smithi007 ceph-mon[62854]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-19T00:38:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:55 smithi204 ceph-mon[66572]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-19T00:38:55.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:55 smithi044 ceph-mon[66366]: pgmap v380: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-19T00:38:57.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:57 smithi007 ceph-mon[62854]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:38:57.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:57 smithi204 ceph-mon[66572]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:38:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:57 smithi044 ceph-mon[66366]: pgmap v381: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:38:59.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:38:59 smithi007 ceph-mon[62854]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:38:59 smithi204 ceph-mon[66572]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:38:59.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:38:59 smithi044 ceph-mon[66366]: pgmap v382: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:39:01.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:39:01.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:39:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.615+0000 7f181e1f1640 1 -- 172.21.15.7:0/4027205358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 msgr2=0x7f18180fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.615+0000 7f181e1f1640 1 --2- 172.21.15.7:0/4027205358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f18180fff30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1808009900 tx=0x7f180802bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 -- 172.21.15.7:0/4027205358 shutdown_connections 2024-03-19T00:39:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 --2- 172.21.15.7:0/4027205358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18181034a0 0x7f1818109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 --2- 172.21.15.7:0/4027205358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1818102ae0 0x7f1818102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 --2- 172.21.15.7:0/4027205358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f18180fff30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 -- 172.21.15.7:0/4027205358 >> 172.21.15.7:0/4027205358 conn(0x7f18180fb440 msgr2=0x7f18180fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 -- 172.21.15.7:0/4027205358 shutdown_connections 2024-03-19T00:39:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.616+0000 7f181e1f1640 1 -- 172.21.15.7:0/4027205358 wait complete. 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.617+0000 7f181e1f1640 1 Processor -- start 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.617+0000 7f181e1f1640 1 -- start start 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.617+0000 7f181e1f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f1818197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.617+0000 7f181e1f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 0x7f1818197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f181e1f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18181034a0 0x7f181819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f181e1f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f181810d3c0 con 0x7f18181034a0 2024-03-19T00:39:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 0x7f1818197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f1818197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 0x7f1818197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:32954/0 (socket says 172.21.15.7:32954) 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f1818197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41894/0 (socket says 172.21.15.7:41894) 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f1816ffd640 1 -- 172.21.15.7:0/1178689662 learned_addr learned my addr 172.21.15.7:0/1178689662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f1817fff640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18181034a0 0x7f181819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f181810d240 con 0x7f18180ffb10 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f181810d540 con 0x7f1818102ae0 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 -- 172.21.15.7:0/1178689662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 msgr2=0x7f1818197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 0x7f1818197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 -- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18181034a0 msgr2=0x7f181819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18181034a0 0x7f181819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f18177fe640 1 -- 172.21.15.7:0/1178689662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181819f6a0 con 0x7f18180ffb10 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f1816ffd640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 0x7f1818197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.618+0000 7f1817fff640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18181034a0 0x7f181819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.619+0000 7f18177fe640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f1818197a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1808009230 tx=0x7f18080040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.619+0000 7f1814ff9640 1 -- 172.21.15.7:0/1178689662 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808043020 con 0x7f18180ffb10 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.619+0000 7f1814ff9640 1 -- 172.21.15.7:0/1178689662 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18080044c0 con 0x7f18180ffb10 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.619+0000 7f1814ff9640 1 -- 172.21.15.7:0/1178689662 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808003a90 con 0x7f18180ffb10 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.619+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181819f930 con 0x7f18180ffb10 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.619+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181819fea0 con 0x7f18180ffb10 2024-03-19T00:39:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.621+0000 7f1814ff9640 1 -- 172.21.15.7:0/1178689662 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f1808046020 con 0x7f18180ffb10 2024-03-19T00:39:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.622+0000 7f1820dd8740 1 Processor -- start 2024-03-19T00:39:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.622+0000 7f1820dd8740 1 -- start start 2024-03-19T00:39:01.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.622+0000 7f1814ff9640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18000a2910 0x7f18000a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059d15b0 0x55fa059d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1820dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa05930500 con 0x55fa059d15b0 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1820dd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fa058d8be0 con 0x55fa059cc440 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1820dd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fa0592c5a0 con 0x55fa059cba50 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1817fff640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18000a2910 0x7f18000a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1814ff9640 1 -- 172.21.15.7:0/1178689662 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f18080f0ec0 con 0x7f18180ffb10 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41902/0 (socket says 172.21.15.7:41902) 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1816ffd640 1 -- 172.21.15.7:0/2908794175 learned_addr learned my addr 172.21.15.7:0/2908794175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f18177fe640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1817fff640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059d15b0 0x55fa059d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f18177fe640 1 -- 172.21.15.7:0/2908794175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 msgr2=0x55fa059d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f18177fe640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f18177fe640 1 -- 172.21.15.7:0/2908794175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059d15b0 msgr2=0x55fa059d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f18177fe640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059d15b0 0x55fa059d3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f18177fe640 1 -- 172.21.15.7:0/2908794175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa058fa000 con 0x55fa059cba50 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1817fff640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059d15b0 0x55fa059d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.623+0000 7f1816ffd640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.624+0000 7f18177fe640 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059cbe70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18080ee620 tx=0x7f18080bd0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9c41c996d558c33 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.624+0000 7f17fdffb640 1 -- 172.21.15.7:0/2908794175 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18080c1600 con 0x55fa059cba50 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.624+0000 7f1817fff640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18000a2910 0x7f18000a4dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1810005fd0 tx=0x7f1810013460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.624+0000 7f17fdffb640 1 -- 172.21.15.7:0/2908794175 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18080c5c50 con 0x55fa059cba50 2024-03-19T00:39:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.624+0000 7f17fdffb640 1 -- 172.21.15.7:0/2908794175 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18080c1170 con 0x55fa059cba50 2024-03-19T00:39:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 -- 172.21.15.7:0/2908794175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 msgr2=0x55fa059cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059cbe70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18080ee620 tx=0x7f18080bd0b0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 -- 172.21.15.7:0/2908794175 shutdown_connections 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059d15b0 0x55fa059d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 --2- 172.21.15.7:0/2908794175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059cbe70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 -- 172.21.15.7:0/2908794175 >> 172.21.15.7:0/2908794175 conn(0x55fa059c6fc0 msgr2=0x55fa059c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 -- 172.21.15.7:0/2908794175 shutdown_connections 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 -- 172.21.15.7:0/2908794175 wait complete. 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.625+0000 7f1820dd8740 1 -- start start 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.626+0000 7f1820dd8740 1 Processor -- start 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.626+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 0x55fa059d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.626+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.626+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059d15b0 0x55fa059e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f18177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 0x55fa059d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059da450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41908/0 (socket says 172.21.15.7:41908) 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f18177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 0x55fa059d9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46882/0 (socket says 172.21.15.7:46882) 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 -- 172.21.15.7:0/3916690947 learned_addr learned my addr 172.21.15.7:0/3916690947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa059d85c0 con 0x55fa059cba50 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fa058ec160 con 0x55fa059cc440 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fa059d8740 con 0x55fa059d15b0 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1820dd8740 10 client.0 _ll_register_callbacks cb 0x55fa05928a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 -- 172.21.15.7:0/3916690947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059d15b0 msgr2=0x55fa059e1490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059d15b0 0x55fa059e1490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 -- 172.21.15.7:0/3916690947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 msgr2=0x55fa059d9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 0x55fa059d9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f1816ffd640 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0570e7d0 con 0x55fa059cc440 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.627+0000 7f18177fe640 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 0x55fa059d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f1816ffd640 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059da450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f180c012d20 tx=0x7f180c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610072a344cc37dc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f17f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f17f6ffd640 1 -- 172.21.15.7:0/3916690947 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c005020 con 0x55fa059cc440 2024-03-19T00:39:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f17f6ffd640 1 -- 172.21.15.7:0/3916690947 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f180c0051c0 con 0x55fa059cc440 2024-03-19T00:39:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18080c5c50 con 0x55fa059cc440 2024-03-19T00:39:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f17f6ffd640 1 -- 172.21.15.7:0/3916690947 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f180c0053c0 con 0x55fa059cc440 2024-03-19T00:39:01.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f1820dd8740 10 client.25511 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa056443f0 con 0x55fa059cc440 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.628+0000 7f17f6ffd640 1 -- 172.21.15.7:0/3916690947 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f180c005560 con 0x55fa059cc440 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f17f6ffd640 1 -- 172.21.15.7:0/3916690947 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f180c01c600 con 0x55fa059cc440 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f1820dd8740 10 client.25511 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f1820dd8740 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f1820dd8740 20 client.25511 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f1820dd8740 4 client.25511 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18080c5c50 con 0x55fa059e5d90 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.629+0000 7f18177fe640 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.630+0000 7f17f6ffd640 10 client.25511 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.630+0000 7f18177fe640 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059e81b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7177a7e909460b server_cookie=12cdc581bb8cf285 in_seq=0 out_seq=0 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.630+0000 7f17f6ffd640 1 -- 172.21.15.7:0/3916690947 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f18080c5c50 con 0x55fa059e5d90 2024-03-19T00:39:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.630+0000 7f17f6ffd640 10 client.25511 handle_command_reply: tid=0 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.630+0000 7f1820dd8740 1 client.25511 shutdown 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 msgr2=0x55fa059da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059da450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f180c012d20 tx=0x7f180c00f790 comp rx=0 tx=0).stop 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 shutdown_connections 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059d15b0 0x55fa059e1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059cc440 0x55fa059da450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059e81b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3916690947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cba50 0x55fa059d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f17f6ffd640 0 client.25511 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 >> 172.21.15.7:0/3916690947 conn(0x55fa059c6fc0 msgr2=0x55fa059c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 shutdown_connections 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 1 -- 172.21.15.7:0/3916690947 wait complete. 2024-03-19T00:39:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.631+0000 7f1820dd8740 20 client.25511 trim_cache size 0 max 16384 2024-03-19T00:39:01.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:01 smithi007 ceph-mon[62854]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:39:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1820dd8740 1 Processor -- start 2024-03-19T00:39:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1820dd8740 1 -- start start 2024-03-19T00:39:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1820dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18181aa8f0 con 0x55fa059cc440 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f18177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f18177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059d91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:32986/0 (socket says 172.21.15.7:32986) 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f18177fe640 1 -- 172.21.15.7:0/3987060349 learned_addr learned my addr 172.21.15.7:0/3987060349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1817fff640 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.667+0000 7f1816ffd640 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18181a81c0 con 0x55fa059d15b0 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18181a6010 con 0x55fa059cba50 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f18177fe640 1 -- 172.21.15.7:0/3987060349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 msgr2=0x55fa059d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f18177fe640 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059d9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f18177fe640 1 -- 172.21.15.7:0/3987060349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 msgr2=0x55fa059e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f18177fe640 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f18177fe640 1 -- 172.21.15.7:0/3987060349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa056040f0 con 0x55fa059cba50 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f1816ffd640 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.668+0000 7f18177fe640 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059d91e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1808005ab0 tx=0x7f18080bd330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644a4f0a83eee7a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.669+0000 7f17fd7fa640 1 -- 172.21.15.7:0/3987060349 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18080c5c50 con 0x55fa059cba50 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.669+0000 7f17fd7fa640 1 -- 172.21.15.7:0/3987060349 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18080ee6d0 con 0x55fa059cba50 2024-03-19T00:39:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.669+0000 7f17fd7fa640 1 -- 172.21.15.7:0/3987060349 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18080c1170 con 0x55fa059cba50 2024-03-19T00:39:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.669+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 msgr2=0x55fa059d91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.669+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059d91e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1808005ab0 tx=0x7f18080bd330 comp rx=0 tx=0).stop 2024-03-19T00:39:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 shutdown_connections 2024-03-19T00:39:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 --2- 172.21.15.7:0/3987060349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059d91e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 >> 172.21.15.7:0/3987060349 conn(0x55fa059c6fc0 msgr2=0x55fa059c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 shutdown_connections 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 -- 172.21.15.7:0/3987060349 wait complete. 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 -- start start 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 Processor -- start 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059deec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059df400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.670+0000 7f1820dd8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1820dd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18181a6680 con 0x55fa059cc440 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1820dd8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18181a7af0 con 0x55fa059d15b0 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1820dd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f18181a6d50 con 0x55fa059cba50 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1820dd8740 10 client.0 _ll_register_callbacks cb 0x7f180c009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:01.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f18177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059deec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f18177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059deec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33002/0 (socket says 172.21.15.7:33002) 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f18177fe640 1 -- 172.21.15.7:0/752415273 learned_addr learned my addr 172.21.15.7:0/752415273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1816ffd640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 -- 172.21.15.7:0/752415273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 msgr2=0x55fa059deec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059deec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 -- 172.21.15.7:0/752415273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 msgr2=0x55fa059df400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059df400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 -- 172.21.15.7:0/752415273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa059e0a00 con 0x55fa059d15b0 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f18177fe640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059deec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1817fff640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059e0410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f181000f960 tx=0x7f181001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c75d1c1e46743b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f17f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f17f6ffd640 1 -- 172.21.15.7:0/752415273 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810004b20 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18080ee6d0 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.671+0000 7f17f6ffd640 1 -- 172.21.15.7:0/752415273 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1810004cc0 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f17f6ffd640 1 -- 172.21.15.7:0/752415273 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1810004510 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 10 client.25517 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa059d4360 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f17f6ffd640 1 -- 172.21.15.7:0/752415273 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18080ee6d0 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f17f6ffd640 1 -- 172.21.15.7:0/752415273 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f181001f070 con 0x55fa059d15b0 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 10 client.25517 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 20 client.25517 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 4 client.25517 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:39:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.672+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa0593da40 con 0x55fa059e5d90 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f18177fe640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f17f6ffd640 10 client.25517 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f18177fe640 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059d6d70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da27456727fad9e server_cookie=5f9ee95275561207 in_seq=0 out_seq=0 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f17f6ffd640 1 -- 172.21.15.7:0/752415273 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa0593da40 con 0x55fa059e5d90 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f17f6ffd640 10 client.25517 handle_command_reply: tid=0 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 client.25517 shutdown 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 msgr2=0x55fa059e0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059e0410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f181000f960 tx=0x7f181001b000 comp rx=0 tx=0).stop 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 shutdown_connections 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fa059d15b0 0x55fa059e0410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fa059e5d90 0x55fa059d6d70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f17f6ffd640 0 client.25517 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa059cc440 0x55fa059df400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 --2- 172.21.15.7:0/752415273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa059cba50 0x55fa059deec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.673+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 >> 172.21.15.7:0/752415273 conn(0x55fa059c6fc0 msgr2=0x55fa059d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.674+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 shutdown_connections 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.674+0000 7f1820dd8740 1 -- 172.21.15.7:0/752415273 wait complete. 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.674+0000 7f1820dd8740 20 client.25517 trim_cache size 0 max 16384 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:39:01.680 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:39:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18000a2910 msgr2=0x7f18000a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18000a2910 0x7f18000a4dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1810005fd0 tx=0x7f1810013460 comp rx=0 tx=0).stop 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 msgr2=0x7f1818197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f1818197a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1808009230 tx=0x7f18080040c0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 shutdown_connections 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f18000a2910 0x7f18000a4dd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18181034a0 0x7f181819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818102ae0 0x7f1818197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 --2- 172.21.15.7:0/1178689662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18180ffb10 0x7f1818197a80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 >> 172.21.15.7:0/1178689662 conn(0x7f18180fb440 msgr2=0x7f1818105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 shutdown_connections 2024-03-19T00:39:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.675+0000 7f181e1f1640 1 -- 172.21.15.7:0/1178689662 wait complete. 2024-03-19T00:39:01.687 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:39:01.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:39:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:01 smithi204 ceph-mon[66572]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:39:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 -- 172.21.15.7:0/3505735098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 msgr2=0x7fe3fc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 --2- 172.21.15.7:0/3505735098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc101dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009900 tx=0x7fe3ec02bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 -- 172.21.15.7:0/3505735098 shutdown_connections 2024-03-19T00:39:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 --2- 172.21.15.7:0/3505735098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3fc102300 0x7fe3fc108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 --2- 172.21.15.7:0/3505735098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc101dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 --2- 172.21.15.7:0/3505735098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc100750 0x7fe3fc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 -- 172.21.15.7:0/3505735098 >> 172.21.15.7:0/3505735098 conn(0x7fe3fc0fbce0 msgr2=0x7fe3fc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 -- 172.21.15.7:0/3505735098 shutdown_connections 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.791+0000 7fe403d80640 1 -- 172.21.15.7:0/3505735098 wait complete. 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 Processor -- start 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 -- start start 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3fc100750 0x7fe3fc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102300 0x7fe3fc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc10c280 con 0x7fe3fc102300 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc10c100 con 0x7fe3fc101940 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe403d80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc10c400 con 0x7fe3fc100750 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe40131b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe40131b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41946/0 (socket says 172.21.15.7:41946) 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe40131b640 1 -- 172.21.15.7:0/4269005120 learned_addr learned my addr 172.21.15.7:0/4269005120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe401b1c640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3fc100750 0x7fe3fc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.792+0000 7fe40231d640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102300 0x7fe3fc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe40131b640 1 -- 172.21.15.7:0/4269005120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3fc100750 msgr2=0x7fe3fc197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe40131b640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3fc100750 0x7fe3fc197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe40131b640 1 -- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102300 msgr2=0x7fe3fc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe40131b640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102300 0x7fe3fc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe40131b640 1 -- 172.21.15.7:0/4269005120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc19f680 con 0x7fe3fc101940 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe40131b640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc197fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006010 tx=0x7fe3ec005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec043020 con 0x7fe3fc101940 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3ec0043d0 con 0x7fe3fc101940 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc19f910 con 0x7fe3fc101940 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec003b80 con 0x7fe3fc101940 2024-03-19T00:39:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.793+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc19feb0 con 0x7fe3fc101940 2024-03-19T00:39:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.794+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc1a0140 con 0x7fe3fc101940 2024-03-19T00:39:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.795+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7fe3ec046020 con 0x7fe3fc101940 2024-03-19T00:39:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.796+0000 7fe3f2ffd640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe3e80a2910 0x7fe3e80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.796+0000 7fe401b1c640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe3e80a2910 0x7fe3e80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.796+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe3ec0f1670 con 0x7fe3fc101940 2024-03-19T00:39:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.797+0000 7fe401b1c640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe3e80a2910 0x7fe3e80a4dd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc1017a0 tx=0x7fe3f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:01.800+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe3ec04d020 con 0x7fe3fc101940 2024-03-19T00:39:01.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:01 smithi044 ceph-mon[66366]: pgmap v383: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:39:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.024+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3fc1a0310 con 0x7fe3fc101940 2024-03-19T00:39:02.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.024+0000 7fe3f2ffd640 1 -- 172.21.15.7:0/4269005120 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7fe3ec0be300 con 0x7fe3fc101940 2024-03-19T00:39:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:39:02.026 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:38:23.871158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:39:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe3e80a2910 msgr2=0x7fe3e80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe3e80a2910 0x7fe3e80a4dd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc1017a0 tx=0x7fe3f8005f90 comp rx=0 tx=0).stop 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 msgr2=0x7fe3fc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc197fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006010 tx=0x7fe3ec005e40 comp rx=0 tx=0).stop 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 shutdown_connections 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe3e80a2910 0x7fe3e80a4dd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102300 0x7fe3fc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc101940 0x7fe3fc197fa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 --2- 172.21.15.7:0/4269005120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3fc100750 0x7fe3fc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 >> 172.21.15.7:0/4269005120 conn(0x7fe3fc0fbce0 msgr2=0x7fe3fc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 shutdown_connections 2024-03-19T00:39:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.027+0000 7fe403d80640 1 -- 172.21.15.7:0/4269005120 wait complete. 2024-03-19T00:39:02.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:39:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 -- 172.21.15.7:0/1709159500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 msgr2=0x7f2908109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 --2- 172.21.15.7:0/1709159500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 0x7f2908109950 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ca0 tx=0x7f290002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 -- 172.21.15.7:0/1709159500 shutdown_connections 2024-03-19T00:39:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 --2- 172.21.15.7:0/1709159500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 0x7f2908109950 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 --2- 172.21.15.7:0/1709159500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 0x7f2908107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 --2- 172.21.15.7:0/1709159500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f29081046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 -- 172.21.15.7:0/1709159500 >> 172.21.15.7:0/1709159500 conn(0x7f29080fbc50 msgr2=0x7f29080fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 -- 172.21.15.7:0/1709159500 shutdown_connections 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 -- 172.21.15.7:0/1709159500 wait complete. 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.148+0000 7f290e0c0640 1 Processor -- start 2024-03-19T00:39:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 -- start start 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f2908101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 0x7f2908100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 0x7f29081007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f2906ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 0x7f2908100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f29077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f2908101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f2906ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 0x7f2908100290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46936/0 (socket says 172.21.15.7:46936) 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f29077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f2908101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33050/0 (socket says 172.21.15.7:33050) 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f2906ffd640 1 -- 172.21.15.7:0/314756602 learned_addr learned my addr 172.21.15.7:0/314756602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f290810d350 con 0x7f2908104c10 2024-03-19T00:39:02.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f290810d1d0 con 0x7f2908107540 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f290810d4d0 con 0x7f29081022e0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.149+0000 7f2907fff640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 0x7f29081007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f29077fe640 1 -- 172.21.15.7:0/314756602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 msgr2=0x7f29081007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f29077fe640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 0x7f29081007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f29077fe640 1 -- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 msgr2=0x7f2908100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f29077fe640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 0x7f2908100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f29077fe640 1 -- 172.21.15.7:0/314756602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2908101060 con 0x7f29081022e0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f29077fe640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f2908101c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9f0 tx=0x7f28f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f2904ff9640 1 -- 172.21.15.7:0/314756602 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f800cd80 con 0x7f29081022e0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f2904ff9640 1 -- 172.21.15.7:0/314756602 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f80035f0 con 0x7f29081022e0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f2904ff9640 1 -- 172.21.15.7:0/314756602 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8010670 con 0x7f29081022e0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29081a1b10 con 0x7f29081022e0 2024-03-19T00:39:02.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.150+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29081a2010 con 0x7f29081022e0 2024-03-19T00:39:02.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 Processor -- start 2024-03-19T00:39:02.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 -- start start 2024-03-19T00:39:02.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559bf752d4b0 con 0x559bf75cee50 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559bf74d5d70 con 0x559bf75c92f0 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2910d84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559bf7529550 con 0x559bf75c9ce0 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2904ff9640 1 -- 172.21.15.7:0/314756602 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f28f80037a0 con 0x7f29081022e0 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2907fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2907fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41962/0 (socket says 172.21.15.7:41962) 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2907fff640 1 -- 172.21.15.7:0/390281781 learned_addr learned my addr 172.21.15.7:0/390281781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.153+0000 7f2906ffd640 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2904ff9640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f28f00a2910 0x7f28f00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 -- 172.21.15.7:0/390281781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 msgr2=0x559bf75ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f29077fe640 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 -- 172.21.15.7:0/390281781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 msgr2=0x559bf75d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 -- 172.21.15.7:0/390281781 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf74f8210 con 0x559bf75c92f0 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2904ff9640 1 -- 172.21.15.7:0/314756602 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f28f80cf250 con 0x7f29081022e0 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f28f00a2910 0x7f28f00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.154+0000 7f2907fff640 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75c9710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2900005e30 tx=0x7f2900009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6487cd4c8d8e355 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f28edffb640 1 -- 172.21.15.7:0/390281781 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900039040 con 0x559bf75c92f0 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f28edffb640 1 -- 172.21.15.7:0/390281781 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2900003e20 con 0x559bf75c92f0 2024-03-19T00:39:02.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f28edffb640 1 -- 172.21.15.7:0/390281781 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290004b470 con 0x559bf75c92f0 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 -- 172.21.15.7:0/390281781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 msgr2=0x559bf75c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2907fff640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f28f00a2910 0x7f28f00a4dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f290004a000 tx=0x7f290004b710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75c9710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2900005e30 tx=0x7f2900009800 comp rx=0 tx=0).stop 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 -- 172.21.15.7:0/390281781 shutdown_connections 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 --2- 172.21.15.7:0/390281781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75c9710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 -- 172.21.15.7:0/390281781 >> 172.21.15.7:0/390281781 conn(0x559bf75c4860 msgr2=0x559bf75c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 -- 172.21.15.7:0/390281781 shutdown_connections 2024-03-19T00:39:02.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.155+0000 7f2910d84740 1 -- 172.21.15.7:0/390281781 wait complete. 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 -- start start 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 Processor -- start 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75dad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559bf75d1e00 con 0x559bf75cee50 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2907fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f29077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2907fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75dad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46962/0 (socket says 172.21.15.7:46962) 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559bf74e9310 con 0x559bf75c92f0 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2907fff640 1 -- 172.21.15.7:0/750968912 learned_addr learned my addr 172.21.15.7:0/750968912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f29077fe640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41970/0 (socket says 172.21.15.7:41970) 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559bf75d1f80 con 0x559bf75c9ce0 2024-03-19T00:39:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.156+0000 7f2906ffd640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f2910d84740 10 client.0 _ll_register_callbacks cb 0x559bf74dd2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f29077fe640 1 -- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 msgr2=0x559bf75d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f29077fe640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f29077fe640 1 -- 172.21.15.7:0/750968912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 msgr2=0x559bf75dad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f29077fe640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75dad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f29077fe640 1 -- 172.21.15.7:0/750968912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf72413f0 con 0x559bf75c92f0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f2907fff640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75dad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f29077fe640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d3810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cc840 tx=0x7f28f809b0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862f441ff988666f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f28e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f28e6ffd640 1 -- 172.21.15.7:0/750968912 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f809f820 con 0x559bf75c92f0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f28e6ffd640 1 -- 172.21.15.7:0/750968912 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f28f80b44f0 con 0x559bf75c92f0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2900003e20 con 0x559bf75c92f0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f28e6ffd640 1 -- 172.21.15.7:0/750968912 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28f80a3e70 con 0x559bf75c92f0 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f2910d84740 10 client.25535 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559bf730b7d0 con 0x559bf75c92f0 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.157+0000 7f28e6ffd640 1 -- 172.21.15.7:0/750968912 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f80cce50 con 0x559bf75c92f0 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f28e6ffd640 1 -- 172.21.15.7:0/750968912 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28f8097d40 con 0x559bf75c92f0 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2910d84740 10 client.25535 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2910d84740 10 client.25535 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2910d84740 20 client.25535 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2910d84740 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf75e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2910d84740 4 client.25535 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2900003e20 con 0x559bf75df520 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2906ffd640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf75e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f2906ffd640 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf75e19b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=187d81aa4da2158f server_cookie=bcf0f975ed524d1b in_seq=0 out_seq=0 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.158+0000 7f28e6ffd640 10 client.25535 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f28e6ffd640 1 -- 172.21.15.7:0/750968912 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f2900003e20 con 0x559bf75df520 2024-03-19T00:39:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f28e6ffd640 10 client.25535 handle_command_reply: tid=0 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 client.25535 shutdown 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 msgr2=0x559bf75d3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d3810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cc840 tx=0x7f28f809b0f0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 shutdown_connections 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf75e19b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75cee50 0x559bf75dad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f28e6ffd640 0 client.25535 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c9ce0 0x559bf75d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 --2- 172.21.15.7:0/750968912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d3810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 >> 172.21.15.7:0/750968912 conn(0x559bf75c4860 msgr2=0x559bf75c3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 shutdown_connections 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.159+0000 7f2910d84740 1 -- 172.21.15.7:0/750968912 wait complete. 2024-03-19T00:39:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.160+0000 7f2910d84740 20 client.25535 trim_cache size 0 max 16384 2024-03-19T00:39:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 Processor -- start 2024-03-19T00:39:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 -- start start 2024-03-19T00:39:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c92f0 0x559bf75d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75cee50 0x559bf75c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2906ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2906ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75dacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46976/0 (socket says 172.21.15.7:46976) 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2906ffd640 1 -- 172.21.15.7:0/2001856223 learned_addr learned my addr 172.21.15.7:0/2001856223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29081aee00 con 0x559bf75c9ce0 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29081ac6d0 con 0x559bf75cee50 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29081aa520 con 0x559bf75c92f0 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f29077fe640 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c92f0 0x559bf75d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.186+0000 7f2907fff640 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75cee50 0x559bf75c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f29077fe640 1 -- 172.21.15.7:0/2001856223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75cee50 msgr2=0x559bf75c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f29077fe640 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75cee50 0x559bf75c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f29077fe640 1 -- 172.21.15.7:0/2001856223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 msgr2=0x559bf75dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f29077fe640 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f29077fe640 1 -- 172.21.15.7:0/2001856223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf759a880 con 0x559bf75c92f0 2024-03-19T00:39:02.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f2906ffd640 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f29077fe640 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c92f0 0x559bf75d3cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28f800cfd0 tx=0x7f28f8099be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed29fd2e0ab57eb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f28ed7fa640 1 -- 172.21.15.7:0/2001856223 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f809f820 con 0x559bf75c92f0 2024-03-19T00:39:02.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f28ed7fa640 1 -- 172.21.15.7:0/2001856223 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f80cce50 con 0x559bf75c92f0 2024-03-19T00:39:02.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.187+0000 7f28ed7fa640 1 -- 172.21.15.7:0/2001856223 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80b42a0 con 0x559bf75c92f0 2024-03-19T00:39:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c92f0 msgr2=0x559bf75d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c92f0 0x559bf75d3cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28f800cfd0 tx=0x7f28f8099be0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 shutdown_connections 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75cee50 0x559bf75c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 --2- 172.21.15.7:0/2001856223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75c92f0 0x559bf75d3cb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 >> 172.21.15.7:0/2001856223 conn(0x559bf75c4860 msgr2=0x559bf75c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 shutdown_connections 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 -- 172.21.15.7:0/2001856223 wait complete. 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.188+0000 7f2910d84740 1 -- start start 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 Processor -- start 2024-03-19T00:39:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75cee50 0x559bf75d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29081aab90 con 0x559bf75c9ce0 2024-03-19T00:39:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29081ac000 con 0x559bf75c92f0 2024-03-19T00:39:02.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29081ab260 con 0x559bf75cee50 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2910d84740 10 client.0 _ll_register_callbacks cb 0x7f28f809b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41996/0 (socket says 172.21.15.7:41996) 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 -- 172.21.15.7:0/1791893330 learned_addr learned my addr 172.21.15.7:0/1791893330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2906ffd640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2907fff640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75cee50 0x559bf75d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 -- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75cee50 msgr2=0x559bf75d9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75cee50 0x559bf75d9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 -- 172.21.15.7:0/1791893330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 msgr2=0x559bf75d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f29077fe640 1 -- 172.21.15.7:0/1791893330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf75da330 con 0x559bf75c92f0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.189+0000 7f2906ffd640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f29077fe640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d87f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f28f8097b30 tx=0x7f28f8003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8e8df2f1d9b972 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f28e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f28e6ffd640 1 -- 172.21.15.7:0/1791893330 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f807e020 con 0x559bf75c92f0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f80cce50 con 0x559bf75c92f0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f28e6ffd640 1 -- 172.21.15.7:0/1791893330 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f28f809ad30 con 0x559bf75c92f0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f28e6ffd640 1 -- 172.21.15.7:0/1791893330 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28f809f820 con 0x559bf75c92f0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f2910d84740 10 client.25541 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559bf75da330 con 0x559bf75c92f0 2024-03-19T00:39:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.190+0000 7f28e6ffd640 1 -- 172.21.15.7:0/1791893330 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f80cce50 con 0x559bf75c92f0 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f28e6ffd640 1 -- 172.21.15.7:0/1791893330 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28f809a620 con 0x559bf75c92f0 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2910d84740 10 client.25541 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2910d84740 10 client.25541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2910d84740 20 client.25541 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2910d84740 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf7648220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2910d84740 4 client.25541 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bf758b3e0 con 0x559bf75df520 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.191+0000 7f2906ffd640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf7648220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f28e6ffd640 10 client.25541 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2906ffd640 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf7648220 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b90f742166e78133 server_cookie=c1b84d974efe09d8 in_seq=0 out_seq=0 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f28e6ffd640 1 -- 172.21.15.7:0/1791893330 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bf758b3e0 con 0x559bf75df520 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f28e6ffd640 10 client.25541 handle_command_reply: tid=0 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 client.25541 shutdown 2024-03-19T00:39:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 msgr2=0x559bf75d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d87f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f28f8097b30 tx=0x7f28f8003e30 comp rx=0 tx=0).stop 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 shutdown_connections 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559bf75df520 0x559bf7648220 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559bf75cee50 0x559bf75d9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bf75c9ce0 0x559bf75d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f28e6ffd640 0 client.25541 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 --2- 172.21.15.7:0/1791893330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559bf75c92f0 0x559bf75d87f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 >> 172.21.15.7:0/1791893330 conn(0x559bf75c4860 msgr2=0x559bf75d3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 shutdown_connections 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 1 -- 172.21.15.7:0/1791893330 wait complete. 2024-03-19T00:39:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.192+0000 7f2910d84740 20 client.25541 trim_cache size 0 max 16384 2024-03-19T00:39:02.202 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:39:02.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f28f00a2910 msgr2=0x7f28f00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f28f00a2910 0x7f28f00a4dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f290004a000 tx=0x7f290004b710 comp rx=0 tx=0).stop 2024-03-19T00:39:02.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 msgr2=0x7f2908101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f2908101c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9f0 tx=0x7f28f800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 shutdown_connections 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f28f00a2910 0x7f28f00a4dd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2908107540 0x7f29081007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2908104c10 0x7f2908100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 --2- 172.21.15.7:0/314756602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29081022e0 0x7f2908101c50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 >> 172.21.15.7:0/314756602 conn(0x7f29080fbc50 msgr2=0x7f2908102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 shutdown_connections 2024-03-19T00:39:02.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.193+0000 7f290e0c0640 1 -- 172.21.15.7:0/314756602 wait complete. 2024-03-19T00:39:02.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:39:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 -- 172.21.15.7:0/1543616631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 msgr2=0x7f36b8101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 --2- 172.21.15.7:0/1543616631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b8101d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009900 tx=0x7f36ac02bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 -- 172.21.15.7:0/1543616631 shutdown_connections 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 --2- 172.21.15.7:0/1543616631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b81022d0 0x7f36b8108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 --2- 172.21.15.7:0/1543616631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b8101d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 --2- 172.21.15.7:0/1543616631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b8100720 0x7f36b8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 -- 172.21.15.7:0/1543616631 >> 172.21.15.7:0/1543616631 conn(0x7f36b80fbc50 msgr2=0x7f36b80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 -- 172.21.15.7:0/1543616631 shutdown_connections 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.362+0000 7f36bd309640 1 -- 172.21.15.7:0/1543616631 wait complete. 2024-03-19T00:39:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.363+0000 7f36bd309640 1 Processor -- start 2024-03-19T00:39:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.363+0000 7f36bd309640 1 -- start start 2024-03-19T00:39:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.363+0000 7f36bd309640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b8100720 0x7f36b8069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.363+0000 7f36bd309640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.363+0000 7f36bd309640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b81022d0 0x7f36b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36bd309640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810c230 con 0x7f36b81022d0 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36bd309640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810c0b0 con 0x7f36b8101910 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36bd309640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810c3b0 con 0x7f36b8100720 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b81022d0 0x7f36b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b8100720 0x7f36b8069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b819c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42022/0 (socket says 172.21.15.7:42022) 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 -- 172.21.15.7:0/550429049 learned_addr learned my addr 172.21.15.7:0/550429049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 -- 172.21.15.7:0/550429049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b8100720 msgr2=0x7f36b8069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b8100720 0x7f36b8069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 -- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b81022d0 msgr2=0x7f36b819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b81022d0 0x7f36b819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 -- 172.21.15.7:0/550429049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b819f630 con 0x7f36b8101910 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b77fe640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b81022d0 0x7f36b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.364+0000 7f36b67fc640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b819c730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36ac006010 tx=0x7f36ac005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.365+0000 7f369ffff640 1 -- 172.21.15.7:0/550429049 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac043020 con 0x7f36b8101910 2024-03-19T00:39:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.365+0000 7f369ffff640 1 -- 172.21.15.7:0/550429049 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36ac0043d0 con 0x7f36b8101910 2024-03-19T00:39:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.365+0000 7f369ffff640 1 -- 172.21.15.7:0/550429049 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac003b80 con 0x7f36b8101910 2024-03-19T00:39:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.366+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b8196500 con 0x7f36b8101910 2024-03-19T00:39:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.366+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b81969d0 con 0x7f36b8101910 2024-03-19T00:39:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f369ffff640 1 -- 172.21.15.7:0/550429049 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f36ac046020 con 0x7f36b8101910 2024-03-19T00:39:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 Processor -- start 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 -- start start 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f369ffff640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36a00a2910 0x7f36a00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e66d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e6727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e6752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55562e5d14b0 con 0x55562e672e50 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55562e579d70 con 0x55562e66dce0 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.369+0000 7f36bffd4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55562e5cd550 con 0x55562e66d2f0 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f369ffff640 1 -- 172.21.15.7:0/550429049 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f36ac0f0a30 con 0x7f36b8101910 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e6727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e6752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36a00a2910 0x7f36a00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e6752e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47018/0 (socket says 172.21.15.7:47018) 2024-03-19T00:39:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e66d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 -- 172.21.15.7:0/2379312738 learned_addr learned my addr 172.21.15.7:0/2379312738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 -- 172.21.15.7:0/2379312738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 msgr2=0x55562e66d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e66d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 -- 172.21.15.7:0/2379312738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 msgr2=0x55562e6727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e6727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.370+0000 7f36b77fe640 1 -- 172.21.15.7:0/2379312738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562e59c210 con 0x55562e672e50 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.371+0000 7f36b77fe640 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e6752e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f36b000f520 tx=0x7f36b000f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b664662f9952a082 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.371+0000 7f369d7fa640 1 -- 172.21.15.7:0/2379312738 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b002b070 con 0x55562e672e50 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.371+0000 7f369d7fa640 1 -- 172.21.15.7:0/2379312738 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b0026900 con 0x55562e672e50 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.371+0000 7f369d7fa640 1 -- 172.21.15.7:0/2379312738 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0019040 con 0x55562e672e50 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.372+0000 7f36bffd4740 1 -- 172.21.15.7:0/2379312738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 msgr2=0x55562e6752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.372+0000 7f36bffd4740 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e6752e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f36b000f520 tx=0x7f36b000f9e0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.372+0000 7f36b77fe640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36a00a2910 0x7f36a00a4dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f36b00099e0 tx=0x7f36b001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 -- 172.21.15.7:0/2379312738 shutdown_connections 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e6752e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e6727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 --2- 172.21.15.7:0/2379312738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e66d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 -- 172.21.15.7:0/2379312738 >> 172.21.15.7:0/2379312738 conn(0x55562e668860 msgr2=0x55562e668c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 -- 172.21.15.7:0/2379312738 shutdown_connections 2024-03-19T00:39:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 -- 172.21.15.7:0/2379312738 wait complete. 2024-03-19T00:39:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 -- start start 2024-03-19T00:39:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 Processor -- start 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e677830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.373+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e677d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e677830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e677830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33120/0 (socket says 172.21.15.7:33120) 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e677d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e677d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47024/0 (socket says 172.21.15.7:47024) 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 -- 172.21.15.7:0/1476562813 learned_addr learned my addr 172.21.15.7:0/1476562813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e672e50 0x55562e67edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55562e675da0 con 0x55562e66dce0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55562e58d310 con 0x55562e672e50 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55562e675f20 con 0x55562e66d2f0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b77fe640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e672e50 0x55562e67edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 -- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e672e50 msgr2=0x55562e67edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e672e50 0x55562e67edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 -- 172.21.15.7:0/1476562813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 msgr2=0x55562e677d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e677d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b6ffd640 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562e2e53f0 con 0x55562e66d2f0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.374+0000 7f36b67fc640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e677d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.375+0000 7f36bffd4740 10 client.0 _ll_register_callbacks cb 0x55562e5812d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.375+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b0026900 con 0x55562e66d2f0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.375+0000 7f36b6ffd640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e677830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f36a8012700 tx=0x7f36a8012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a1a02b8bb0e2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.375+0000 7f369aff5640 10 client.25465 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:39:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f369aff5640 1 -- 172.21.15.7:0/1476562813 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004020 con 0x55562e66d2f0 2024-03-19T00:39:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f369aff5640 1 -- 172.21.15.7:0/1476562813 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f36a80041c0 con 0x55562e66d2f0 2024-03-19T00:39:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f369aff5640 1 -- 172.21.15.7:0/1476562813 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36a8004400 con 0x55562e66d2f0 2024-03-19T00:39:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 10 client.25465 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55562e3af7d0 con 0x55562e66d2f0 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f369aff5640 1 -- 172.21.15.7:0/1476562813 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36a80035f0 con 0x55562e66d2f0 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 10 client.25465 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 10 client.25465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 20 client.25465 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 4 client.25465 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.379+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55562e62f3e0 con 0x55562e683530 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.380+0000 7f36b67fc640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.380+0000 7f369aff5640 10 client.25465 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.380+0000 7f36b67fc640 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6859d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5882c69c01709e7b server_cookie=dd642cc2c1e02cf3 in_seq=0 out_seq=0 2024-03-19T00:39:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f369aff5640 1 -- 172.21.15.7:0/1476562813 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55562e62f3e0 con 0x55562e683530 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f369aff5640 10 client.25465 handle_command_reply: tid=0 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 client.25465 shutdown 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 msgr2=0x55562e677830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e677830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f36a8012700 tx=0x7f36a8012b00 comp rx=0 tx=0).stop 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 shutdown_connections 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6859d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e672e50 0x55562e67edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f369aff5640 0 client.25465 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e677d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1476562813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e677830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 >> 172.21.15.7:0/1476562813 conn(0x55562e668860 msgr2=0x55562e667ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 shutdown_connections 2024-03-19T00:39:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 1 -- 172.21.15.7:0/1476562813 wait complete. 2024-03-19T00:39:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.381+0000 7f36bffd4740 20 client.25465 trim_cache size 0 max 16384 2024-03-19T00:39:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 Processor -- start 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 -- start start 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e667310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e67ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e667850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81aed60 con 0x55562e672e50 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81ac630 con 0x55562e66dce0 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36bffd4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81aa480 con 0x55562e66d2f0 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e67ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e67ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42052/0 (socket says 172.21.15.7:42052) 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e667310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 -- 172.21.15.7:0/3702239829 learned_addr learned my addr 172.21.15.7:0/3702239829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b77fe640 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e667850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 -- 172.21.15.7:0/3702239829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 msgr2=0x55562e667310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e667310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 -- 172.21.15.7:0/3702239829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 msgr2=0x55562e667850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e667850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b67fc640 1 -- 172.21.15.7:0/3702239829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562e63e880 con 0x55562e66dce0 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b6ffd640 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e667310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.408+0000 7f36b77fe640 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e667850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.409+0000 7f36b67fc640 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e67ed70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005ab0 tx=0x7f36ac000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8663104889fa23d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.409+0000 7f369cff9640 1 -- 172.21.15.7:0/3702239829 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0bd720 con 0x55562e66dce0 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.409+0000 7f369cff9640 1 -- 172.21.15.7:0/3702239829 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36ac0bd020 con 0x55562e66dce0 2024-03-19T00:39:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.409+0000 7f369cff9640 1 -- 172.21.15.7:0/3702239829 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0ee2d0 con 0x55562e66dce0 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 -- 172.21.15.7:0/3702239829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 msgr2=0x55562e67ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e67ed70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005ab0 tx=0x7f36ac000bc0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 -- 172.21.15.7:0/3702239829 shutdown_connections 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e672e50 0x55562e667850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66dce0 0x55562e67ed70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 --2- 172.21.15.7:0/3702239829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e66d2f0 0x55562e667310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 -- 172.21.15.7:0/3702239829 >> 172.21.15.7:0/3702239829 conn(0x55562e668860 msgr2=0x55562e6762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 -- 172.21.15.7:0/3702239829 shutdown_connections 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 -- 172.21.15.7:0/3702239829 wait complete. 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 -- start start 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.410+0000 7f36bffd4740 1 Processor -- start 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66d2f0 0x55562e67ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e67bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e672e50 0x55562e6e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81aaaf0 con 0x55562e66dce0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e67bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66d2f0 0x55562e67ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e67bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47046/0 (socket says 172.21.15.7:47046) 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 -- 172.21.15.7:0/1744633834 learned_addr learned my addr 172.21.15.7:0/1744633834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81abf60 con 0x55562e66d2f0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b81ab1c0 con 0x55562e672e50 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b77fe640 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e672e50 0x55562e6e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36bffd4740 10 client.0 _ll_register_callbacks cb 0x7f36a8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 -- 172.21.15.7:0/1744633834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e672e50 msgr2=0x55562e6e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e672e50 0x55562e6e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 -- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66d2f0 msgr2=0x55562e67ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66d2f0 0x55562e67ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.411+0000 7f36b67fc640 1 -- 172.21.15.7:0/1744633834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562e67eff0 con 0x55562e66dce0 2024-03-19T00:39:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f36b67fc640 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e67bf70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f36ac004650 tx=0x7f36ac0bb530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87a51434af3805f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f369aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f369aff5640 1 -- 172.21.15.7:0/1744633834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0bd720 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f369aff5640 1 -- 172.21.15.7:0/1744633834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f36ac003d20 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ac0bd020 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f369aff5640 1 -- 172.21.15.7:0/1744633834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36ac0054b0 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f36bffd4740 10 client.15708 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.412+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55562e6e8e40 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f369aff5640 1 -- 172.21.15.7:0/1744633834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36ac0d8020 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f369aff5640 1 -- 172.21.15.7:0/1744633834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36ac0c5330 con 0x55562e66dce0 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36bffd4740 10 client.15708 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36bffd4740 10 client.15708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36bffd4740 20 client.15708 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36bffd4740 4 client.15708 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f36ac0bd020 con 0x55562e683530 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.413+0000 7f36b6ffd640 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36b6ffd640 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6eb5b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acfa4d1a15d15623 server_cookie=e24127463df1e673 in_seq=0 out_seq=0 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f369aff5640 10 client.15708 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f369aff5640 1 -- 172.21.15.7:0/1744633834 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f36ac0bd020 con 0x55562e683530 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f369aff5640 10 client.15708 handle_command_reply: tid=0 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 client.15708 shutdown 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 msgr2=0x55562e67bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e67bf70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f36ac004650 tx=0x7f36ac0bb530 comp rx=0 tx=0).stop 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 shutdown_connections 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55562e683530 0x55562e6eb5b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55562e672e50 0x55562e6e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f369aff5640 0 client.15708 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55562e66dce0 0x55562e67bf70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 --2- 172.21.15.7:0/1744633834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55562e66d2f0 0x55562e67ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.414+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 >> 172.21.15.7:0/1744633834 conn(0x55562e668860 msgr2=0x55562e66bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.415+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 shutdown_connections 2024-03-19T00:39:02.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.415+0000 7f36bffd4740 1 -- 172.21.15.7:0/1744633834 wait complete. 2024-03-19T00:39:02.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.415+0000 7f36bffd4740 20 client.15708 trim_cache size 0 max 16384 2024-03-19T00:39:02.420 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:39:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.415+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36a00a2910 msgr2=0x7f36a00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.415+0000 7f36bd309640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36a00a2910 0x7f36a00a4dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f36b00099e0 tx=0x7f36b001e000 comp rx=0 tx=0).stop 2024-03-19T00:39:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.415+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 msgr2=0x7f36b819c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b819c730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36ac006010 tx=0x7f36ac005e40 comp rx=0 tx=0).stop 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 shutdown_connections 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36a00a2910 0x7f36a00a4dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b81022d0 0x7f36b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b8101910 0x7f36b819c730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 --2- 172.21.15.7:0/550429049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b8100720 0x7f36b8069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 >> 172.21.15.7:0/550429049 conn(0x7f36b80fbc50 msgr2=0x7f36b8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 shutdown_connections 2024-03-19T00:39:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.416+0000 7f36bd309640 1 -- 172.21.15.7:0/550429049 wait complete. 2024-03-19T00:39:02.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:39:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 -- 172.21.15.7:0/4093013207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904101910 msgr2=0x7f5904101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 --2- 172.21.15.7:0/4093013207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904101910 0x7f5904101d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009900 tx=0x7f58f802bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 -- 172.21.15.7:0/4093013207 shutdown_connections 2024-03-19T00:39:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 --2- 172.21.15.7:0/4093013207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59041022d0 0x7f5904108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 --2- 172.21.15.7:0/4093013207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904101910 0x7f5904101d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 --2- 172.21.15.7:0/4093013207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5904100720 0x7f5904100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 -- 172.21.15.7:0/4093013207 >> 172.21.15.7:0/4093013207 conn(0x7f59040fbc50 msgr2=0x7f59040fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 -- 172.21.15.7:0/4093013207 shutdown_connections 2024-03-19T00:39:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.593+0000 7f590a122640 1 -- 172.21.15.7:0/4093013207 wait complete. 2024-03-19T00:39:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.594+0000 7f590a122640 1 Processor -- start 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.594+0000 7f590a122640 1 -- start start 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.594+0000 7f590a122640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 0x7f5904196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f590a122640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 0x7f5904196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f590a122640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59041022d0 0x7f590419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f59037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 0x7f5904196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 0x7f5904196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f59037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 0x7f5904196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42080/0 (socket says 172.21.15.7:42080) 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 0x7f5904196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33162/0 (socket says 172.21.15.7:33162) 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 -- 172.21.15.7:0/3628563181 learned_addr learned my addr 172.21.15.7:0/3628563181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f590410c280 con 0x7f59041022d0 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f590410c100 con 0x7f5904100720 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f590410c400 con 0x7f5904101910 2024-03-19T00:39:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5903fff640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59041022d0 0x7f590419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 -- 172.21.15.7:0/3628563181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 msgr2=0x7f5904196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 0x7f5904196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 -- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59041022d0 msgr2=0x7f590419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59041022d0 0x7f590419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5902ffd640 1 -- 172.21.15.7:0/3628563181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590419e550 con 0x7f5904101910 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f59037fe640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 0x7f5904196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.595+0000 7f5903fff640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59041022d0 0x7f590419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.596+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 0x7f5904196e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8004aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.596+0000 7f5900ff9640 1 -- 172.21.15.7:0/3628563181 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f803e650 con 0x7f5904101910 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.596+0000 7f5900ff9640 1 -- 172.21.15.7:0/3628563181 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58f803eca0 con 0x7f5904101910 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.596+0000 7f5900ff9640 1 -- 172.21.15.7:0/3628563181 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80035f0 con 0x7f5904101910 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.596+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5904068af0 con 0x7f5904101910 2024-03-19T00:39:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.596+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5904068ff0 con 0x7f5904101910 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 Processor -- start 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 -- start start 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f5900ff9640 1 -- 172.21.15.7:0/3628563181 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f58f8004e90 con 0x7f5904101910 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 0x559c177af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c177134b0 con 0x559c177b4e50 2024-03-19T00:39:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c176bbd70 con 0x559c177af2f0 2024-03-19T00:39:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.599+0000 7f590cdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559c1770f550 con 0x559c177afce0 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 0x559c177af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 0x559c177af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42088/0 (socket says 172.21.15.7:42088) 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 -- 172.21.15.7:0/3313292535 learned_addr learned my addr 172.21.15.7:0/3313292535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f59037fe640 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 -- 172.21.15.7:0/3313292535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 msgr2=0x559c177b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 -- 172.21.15.7:0/3313292535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 msgr2=0x559c177b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 -- 172.21.15.7:0/3313292535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c176de210 con 0x559c177af2f0 2024-03-19T00:39:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5900ff9640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58ec0a2910 0x7f58ec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5903fff640 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 0x559c177af710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00f790 tx=0x7f58fc00fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a99c8ba6245fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f59037fe640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58ec0a2910 0x7f58ec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.600+0000 7f5900ff9640 1 -- 172.21.15.7:0/3628563181 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f58f80f4d30 con 0x7f5904101910 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.601+0000 7f58e9ffb640 1 -- 172.21.15.7:0/3313292535 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc004b20 con 0x559c177af2f0 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.601+0000 7f58e9ffb640 1 -- 172.21.15.7:0/3313292535 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58fc00ee40 con 0x559c177af2f0 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.601+0000 7f59037fe640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58ec0a2910 0x7f58ec0a4dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x559c177b00e0 tx=0x7f58f4011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.601+0000 7f58e9ffb640 1 -- 172.21.15.7:0/3313292535 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc004370 con 0x559c177af2f0 2024-03-19T00:39:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 -- 172.21.15.7:0/3313292535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 msgr2=0x559c177af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 0x559c177af710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00f790 tx=0x7f58fc00fc50 comp rx=0 tx=0).stop 2024-03-19T00:39:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 -- 172.21.15.7:0/3313292535 shutdown_connections 2024-03-19T00:39:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 --2- 172.21.15.7:0/3313292535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177af2f0 0x559c177af710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 -- 172.21.15.7:0/3313292535 >> 172.21.15.7:0/3313292535 conn(0x559c177aa860 msgr2=0x559c177aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 -- 172.21.15.7:0/3313292535 shutdown_connections 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 -- 172.21.15.7:0/3313292535 wait complete. 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 -- start start 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.602+0000 7f590cdef740 1 Processor -- start 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177afce0 0x559c177b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c177b7e00 con 0x559c177b4e50 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c176cf310 con 0x559c177afce0 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559c177b7f80 con 0x559c177af2f0 2024-03-19T00:39:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f590cdef740 10 client.0 _ll_register_callbacks cb 0x559c176c32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f59037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f59037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33188/0 (socket says 172.21.15.7:33188) 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f59037fe640 1 -- 172.21.15.7:0/2613715563 learned_addr learned my addr 172.21.15.7:0/2613715563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5902ffd640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177afce0 0x559c177b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5903fff640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5902ffd640 1 -- 172.21.15.7:0/2613715563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 msgr2=0x559c177b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5902ffd640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5902ffd640 1 -- 172.21.15.7:0/2613715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 msgr2=0x559c177c0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5902ffd640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177c0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.603+0000 7f5902ffd640 1 -- 172.21.15.7:0/2613715563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c174273f0 con 0x559c177afce0 2024-03-19T00:39:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f59037fe640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f5903fff640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177c0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f5902ffd640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177afce0 0x559c177b9d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58f80f2430 tx=0x7f58f8004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2325cb05ffd16916 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f58e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f58e2ffd640 1 -- 172.21.15.7:0/2613715563 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80c1090 con 0x559c177afce0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f58e2ffd640 1 -- 172.21.15.7:0/2613715563 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f58f80f2570 con 0x559c177afce0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c177713e0 con 0x559c177afce0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f58e2ffd640 1 -- 172.21.15.7:0/2613715563 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58f80c0c20 con 0x559c177afce0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f590cdef740 10 client.25571 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c174f17d0 con 0x559c177afce0 2024-03-19T00:39:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.604+0000 7f58e2ffd640 1 -- 172.21.15.7:0/2613715563 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559c177713e0 con 0x559c177afce0 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f58e2ffd640 1 -- 172.21.15.7:0/2613715563 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f58f80e10b0 con 0x559c177afce0 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f590cdef740 10 client.25571 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f590cdef740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f590cdef740 20 client.25571 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f590cdef740 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c177c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f590cdef740 4 client.25571 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.605+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c177209f0 con 0x559c177c5680 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.606+0000 7f59037fe640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c177c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.606+0000 7f58e2ffd640 10 client.25571 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.606+0000 7f59037fe640 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c177c7a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95201ab488f47a2f server_cookie=d62073eb00b62f0f in_seq=0 out_seq=0 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f58e2ffd640 1 -- 172.21.15.7:0/2613715563 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559c177209f0 con 0x559c177c5680 2024-03-19T00:39:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f58e2ffd640 10 client.25571 handle_command_reply: tid=0 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 client.25571 shutdown 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177afce0 msgr2=0x559c177b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177afce0 0x559c177b9d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58f80f2430 tx=0x7f58f8004580 comp rx=0 tx=0).stop 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 shutdown_connections 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c177c7a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177b4e50 0x559c177c0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f58e2ffd640 0 client.25571 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177afce0 0x559c177b9d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 --2- 172.21.15.7:0/2613715563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 >> 172.21.15.7:0/2613715563 conn(0x559c177aa860 msgr2=0x559c177a9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 shutdown_connections 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 1 -- 172.21.15.7:0/2613715563 wait complete. 2024-03-19T00:39:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.607+0000 7f590cdef740 20 client.25571 trim_cache size 0 max 16384 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.633+0000 7f590cdef740 1 Processor -- start 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.633+0000 7f590cdef740 1 -- start start 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f590cdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f590cdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177afce0 0x559c177c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f590cdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c177b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f590cdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59041aa870 con 0x559c177afce0 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f590cdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59041a8140 con 0x559c177b4e50 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f590cdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59041a5f90 con 0x559c177af2f0 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5903fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c177b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f59037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f59037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33190/0 (socket says 172.21.15.7:33190) 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5903fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c177b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42118/0 (socket says 172.21.15.7:42118) 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177afce0 0x559c177c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f59037fe640 1 -- 172.21.15.7:0/3053638456 learned_addr learned my addr 172.21.15.7:0/3053638456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5902ffd640 1 -- 172.21.15.7:0/3053638456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 msgr2=0x559c177ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5902ffd640 1 -- 172.21.15.7:0/3053638456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 msgr2=0x559c177b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c177b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5902ffd640 1 -- 172.21.15.7:0/3053638456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c177b44c0 con 0x559c177afce0 2024-03-19T00:39:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f59037fe640 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.634+0000 7f5903fff640 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c177b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.635+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177afce0 0x559c177c0d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58f802ce10 tx=0x7f58f80056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f246b9b2b97c1165 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.635+0000 7f58e97fa640 1 -- 172.21.15.7:0/3053638456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80d9070 con 0x559c177afce0 2024-03-19T00:39:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.635+0000 7f58e97fa640 1 -- 172.21.15.7:0/3053638456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58f80f7070 con 0x559c177afce0 2024-03-19T00:39:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.635+0000 7f58e97fa640 1 -- 172.21.15.7:0/3053638456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80c0e00 con 0x559c177afce0 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 -- 172.21.15.7:0/3053638456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177afce0 msgr2=0x559c177c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177afce0 0x559c177c0d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58f802ce10 tx=0x7f58f80056f0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 -- 172.21.15.7:0/3053638456 shutdown_connections 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c177b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177afce0 0x559c177c0d20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 --2- 172.21.15.7:0/3053638456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177af2f0 0x559c177ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 -- 172.21.15.7:0/3053638456 >> 172.21.15.7:0/3053638456 conn(0x559c177aa860 msgr2=0x559c177adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 -- 172.21.15.7:0/3053638456 shutdown_connections 2024-03-19T00:39:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.636+0000 7f590cdef740 1 -- 172.21.15.7:0/3053638456 wait complete. 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.637+0000 7f590cdef740 1 -- start start 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.637+0000 7f590cdef740 1 Processor -- start 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.637+0000 7f590cdef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 0x559c177bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f590cdef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f590cdef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c178299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f59037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 0x559c177bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f590cdef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59041a6600 con 0x559c177af2f0 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f590cdef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59041a7a70 con 0x559c177b4e50 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f590cdef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59041a6cd0 con 0x559c177afce0 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f59037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 0x559c177bd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47116/0 (socket says 172.21.15.7:47116) 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f59037fe640 1 -- 172.21.15.7:0/3565639206 learned_addr learned my addr 172.21.15.7:0/3565639206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177bdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f590cdef740 10 client.0 _ll_register_callbacks cb 0x7f58f8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5903fff640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c178299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 -- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 msgr2=0x559c178299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c178299a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 -- 172.21.15.7:0/3565639206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 msgr2=0x559c177bd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 0x559c177bd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 -- 172.21.15.7:0/3565639206 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c177be540 con 0x559c177afce0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f59037fe640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 0x559c177bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f5902ffd640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177bdf30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58f80bd720 tx=0x7f58f8004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec7a72ad0442450 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.638+0000 7f58e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f58e2ffd640 1 -- 172.21.15.7:0/3565639206 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80d9050 con 0x559c177afce0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f58e2ffd640 1 -- 172.21.15.7:0/3565639206 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f58f80f24e0 con 0x559c177afce0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f80f7070 con 0x559c177afce0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f58e2ffd640 1 -- 172.21.15.7:0/3565639206 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58f80c0e00 con 0x559c177afce0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f590cdef740 10 client.25489 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c177be540 con 0x559c177afce0 2024-03-19T00:39:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.639+0000 7f58e2ffd640 1 -- 172.21.15.7:0/3565639206 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58f803ee50 con 0x559c177afce0 2024-03-19T00:39:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f58e2ffd640 1 -- 172.21.15.7:0/3565639206 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f58f80c1520 con 0x559c177afce0 2024-03-19T00:39:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f590cdef740 10 client.25489 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f590cdef740 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f590cdef740 20 client.25489 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f590cdef740 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c1782d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f590cdef740 4 client.25489 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f58f80f7070 con 0x559c177c5680 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.640+0000 7f59037fe640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c1782d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f58e2ffd640 10 client.25489 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f59037fe640 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c1782d540 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241403a22ba59b22 server_cookie=925fd8dbdbafa548 in_seq=0 out_seq=0 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f58e2ffd640 1 -- 172.21.15.7:0/3565639206 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f58f80f7070 con 0x559c177c5680 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f58e2ffd640 10 client.25489 handle_command_reply: tid=0 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 client.25489 shutdown 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 msgr2=0x559c177bdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177bdf30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58f80bd720 tx=0x7f58f8004580 comp rx=0 tx=0).stop 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 shutdown_connections 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c177b4e50 0x559c178299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c177afce0 0x559c177bdf30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559c177c5680 0x559c1782d540 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 --2- 172.21.15.7:0/3565639206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c177af2f0 0x559c177bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 >> 172.21.15.7:0/3565639206 conn(0x559c177aa860 msgr2=0x559c177ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f58e2ffd640 0 client.25489 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 shutdown_connections 2024-03-19T00:39:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 1 -- 172.21.15.7:0/3565639206 wait complete. 2024-03-19T00:39:02.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.641+0000 7f590cdef740 20 client.25489 trim_cache size 0 max 16384 2024-03-19T00:39:02.646 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:39:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58ec0a2910 msgr2=0x7f58ec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58ec0a2910 0x7f58ec0a4dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x559c177b00e0 tx=0x7f58f4011040 comp rx=0 tx=0).stop 2024-03-19T00:39:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 msgr2=0x7f5904196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 0x7f5904196e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8004aa0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 shutdown_connections 2024-03-19T00:39:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58ec0a2910 0x7f58ec0a4dd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59041022d0 0x7f590419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904101910 0x7f5904196e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 --2- 172.21.15.7:0/3628563181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5904100720 0x7f5904196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 >> 172.21.15.7:0/3628563181 conn(0x7f59040fbc50 msgr2=0x7f5904104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 shutdown_connections 2024-03-19T00:39:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:02.642+0000 7f590a122640 1 -- 172.21.15.7:0/3628563181 wait complete. 2024-03-19T00:39:02.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:02 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4269005120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:39:02.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:02 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4269005120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:39:02.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:02 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4269005120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:39:03.659 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:39:03.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:39:03.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:39:03.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:03 smithi007 ceph-mon[62854]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:39:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:03 smithi204 ceph-mon[66572]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:39:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 -- 172.21.15.7:0/863286114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec102320 msgr2=0x7f1bec104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 --2- 172.21.15.7:0/863286114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec102320 0x7f1bec104710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009900 tx=0x7f1bdc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 -- 172.21.15.7:0/863286114 shutdown_connections 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 --2- 172.21.15.7:0/863286114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec107580 0x7f1bec109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 --2- 172.21.15.7:0/863286114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 0x7f1bec107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 --2- 172.21.15.7:0/863286114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec102320 0x7f1bec104710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 -- 172.21.15.7:0/863286114 >> 172.21.15.7:0/863286114 conn(0x7f1bec0fbcb0 msgr2=0x7f1bec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 -- 172.21.15.7:0/863286114 shutdown_connections 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.768+0000 7f1bf1790640 1 -- 172.21.15.7:0/863286114 wait complete. 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 Processor -- start 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 -- start start 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 0x7f1bec197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 0x7f1bec198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec107580 0x7f1bec19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10d390 con 0x7f1bec102320 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10d210 con 0x7f1bec107580 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1bf1790640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10d510 con 0x7f1bec104c50 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 0x7f1bec197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 0x7f1bec197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47134/0 (socket says 172.21.15.7:47134) 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1beaffd640 1 -- 172.21.15.7:0/3730165518 learned_addr learned my addr 172.21.15.7:0/3730165518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.769+0000 7f1be3fff640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 0x7f1bec198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1beb7fe640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec107580 0x7f1bec19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be3fff640 1 -- 172.21.15.7:0/3730165518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec107580 msgr2=0x7f1bec19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be3fff640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec107580 0x7f1bec19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be3fff640 1 -- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 msgr2=0x7f1bec197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be3fff640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 0x7f1bec197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be3fff640 1 -- 172.21.15.7:0/3730165518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec19f700 con 0x7f1bec104c50 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1beaffd640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 0x7f1bec197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be3fff640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 0x7f1bec198070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400e9c0 tx=0x7f1bd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be8ff9640 1 -- 172.21.15.7:0/3730165518 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400ccc0 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be8ff9640 1 -- 172.21.15.7:0/3730165518 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd4005020 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1be8ff9640 1 -- 172.21.15.7:0/3730165518 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4004a00 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec19f9f0 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.770+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec19fef0 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.773+0000 7f1be8ff9640 1 -- 172.21.15.7:0/3730165518 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7f1bd40051d0 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.773+0000 7f1be8ff9640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bd80a2910 0x7f1bd80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.773+0000 7f1beaffd640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bd80a2910 0x7f1bd80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1be8ff9640 1 -- 172.21.15.7:0/3730165518 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1bd40cee80 con 0x7f1bec104c50 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 Processor -- start 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 -- start start 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1beaffd640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bd80a2910 0x7f1bd80a4dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0096c0 tx=0x7f1bdc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c3d90 0x5634212c8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 0x5634212cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563421228160 con 0x5634212c8f00 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5634211d0e10 con 0x5634212c3d90 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1bf437e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563421224200 con 0x5634212c33a0 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1beb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 0x5634212cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c3d90 0x5634212c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1beb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 0x5634212cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47140/0 (socket says 172.21.15.7:47140) 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.774+0000 7f1beb7fe640 1 -- 172.21.15.7:0/359918456 learned_addr learned my addr 172.21.15.7:0/359918456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1beaffd640 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be3fff640 1 -- 172.21.15.7:0/359918456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 msgr2=0x5634212c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be3fff640 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be3fff640 1 -- 172.21.15.7:0/359918456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 msgr2=0x5634212cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be3fff640 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 0x5634212cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be3fff640 1 -- 172.21.15.7:0/359918456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634210047e0 con 0x5634212c3d90 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1beb7fe640 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 0x5634212cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be3fff640 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c3d90 0x5634212c8890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40cc4d0 tx=0x7f1bd40056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b68d42390dde7b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be17fa640 1 -- 172.21.15.7:0/359918456 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd40b9e60 con 0x5634212c3d90 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be17fa640 1 -- 172.21.15.7:0/359918456 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd40a3b00 con 0x5634212c3d90 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.775+0000 7f1be17fa640 1 -- 172.21.15.7:0/359918456 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd401c040 con 0x5634212c3d90 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 -- 172.21.15.7:0/359918456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c3d90 msgr2=0x5634212c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c3d90 0x5634212c8890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40cc4d0 tx=0x7f1bd40056a0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 -- 172.21.15.7:0/359918456 shutdown_connections 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c8f00 0x5634212cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c3d90 0x5634212c8890 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 --2- 172.21.15.7:0/359918456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 -- 172.21.15.7:0/359918456 >> 172.21.15.7:0/359918456 conn(0x5634212be910 msgr2=0x5634212bed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 -- 172.21.15.7:0/359918456 shutdown_connections 2024-03-19T00:39:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 -- 172.21.15.7:0/359918456 wait complete. 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.776+0000 7f1bf437e740 1 -- start start 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 Processor -- start 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5634212cbeb0 con 0x5634212c33a0 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47156/0 (socket says 172.21.15.7:47156) 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 -- 172.21.15.7:0/759705989 learned_addr learned my addr 172.21.15.7:0/759705989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5634211e41a0 con 0x5634212c8f00 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5634212cc030 con 0x5634212c3d90 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1be3fff640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beb7fe640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1bf437e740 10 client.0 _ll_register_callbacks cb 0x563420f3bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 -- 172.21.15.7:0/759705989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 msgr2=0x5634212cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 -- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 msgr2=0x5634212d4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212d4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1beaffd640 1 -- 172.21.15.7:0/759705989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563421064f70 con 0x5634212c33a0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.777+0000 7f1be3fff640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1beaffd640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd890 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc036040 tx=0x7f1bdc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e688e134b34b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bd27fc640 1 -- 172.21.15.7:0/759705989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc050030 con 0x5634212c33a0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bd27fc640 1 -- 172.21.15.7:0/759705989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1bdc03a040 con 0x5634212c33a0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd40a3b00 con 0x5634212c33a0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bd27fc640 1 -- 172.21.15.7:0/759705989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bdc05d050 con 0x5634212c33a0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bf437e740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563420efc170 con 0x5634212c33a0 2024-03-19T00:39:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.778+0000 7f1bd27fc640 1 -- 172.21.15.7:0/759705989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd40a3b00 con 0x5634212c33a0 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bd27fc640 1 -- 172.21.15.7:0/759705989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1bdc041460 con 0x5634212c33a0 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bf437e740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bf437e740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bf437e740 20 client.15738 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bf437e740 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5634212d9b60 0x5634212dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bf437e740 4 client.15738 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634212356a0 con 0x5634212d9b60 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1be3fff640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5634212d9b60 0x5634212dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1be3fff640 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5634212d9b60 0x5634212dc000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8035678265c31b server_cookie=7f536cc97eca5998 in_seq=0 out_seq=0 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.779+0000 7f1bd27fc640 10 client.15738 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bd27fc640 1 -- 172.21.15.7:0/759705989 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5634212356a0 con 0x5634212d9b60 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bd27fc640 10 client.15738 handle_command_reply: tid=0 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 client.15738 shutdown 2024-03-19T00:39:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 msgr2=0x5634212cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd890 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc036040 tx=0x7f1bdc009260 comp rx=0 tx=0).stop 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 shutdown_connections 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212d4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5634212d9b60 0x5634212dc000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 --2- 172.21.15.7:0/759705989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd890 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 >> 172.21.15.7:0/759705989 conn(0x5634212be910 msgr2=0x5634212bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bd27fc640 0 client.15738 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 shutdown_connections 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 1 -- 172.21.15.7:0/759705989 wait complete. 2024-03-19T00:39:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.780+0000 7f1bf437e740 20 client.15738 trim_cache size 0 max 16384 2024-03-19T00:39:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.814+0000 7f1bf437e740 1 Processor -- start 2024-03-19T00:39:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.814+0000 7f1bf437e740 1 -- start start 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1bf437e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec1acb80 con 0x5634212c33a0 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1bf437e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec1aa450 con 0x5634212c8f00 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1bf437e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec1a82a0 con 0x5634212c3d90 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212d4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33268/0 (socket says 172.21.15.7:33268) 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47160/0 (socket says 172.21.15.7:47160) 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 -- 172.21.15.7:0/2017507948 learned_addr learned my addr 172.21.15.7:0/2017507948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1beb7fe640 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 -- 172.21.15.7:0/2017507948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 msgr2=0x5634212cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 -- 172.21.15.7:0/2017507948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 msgr2=0x5634212cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 -- 172.21.15.7:0/2017507948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634212ba160 con 0x5634212c3d90 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1beaffd640 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.815+0000 7f1be3fff640 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212d4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1bd409b520 tx=0x7f1bd40cc410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd24791ef68942e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.816+0000 7f1be0ff9640 1 -- 172.21.15.7:0/2017507948 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd401c070 con 0x5634212c3d90 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.816+0000 7f1be0ff9640 1 -- 172.21.15.7:0/2017507948 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd40a3b00 con 0x5634212c3d90 2024-03-19T00:39:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.816+0000 7f1be0ff9640 1 -- 172.21.15.7:0/2017507948 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd40b9e60 con 0x5634212c3d90 2024-03-19T00:39:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.816+0000 7f1bf437e740 1 -- 172.21.15.7:0/2017507948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 msgr2=0x5634212d4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.816+0000 7f1bf437e740 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212d4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1bd409b520 tx=0x7f1bd40cc410 comp rx=0 tx=0).stop 2024-03-19T00:39:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- 172.21.15.7:0/2017507948 shutdown_connections 2024-03-19T00:39:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x5634212cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c3d90 0x5634212d4dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 --2- 172.21.15.7:0/2017507948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c33a0 0x5634212cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- 172.21.15.7:0/2017507948 >> 172.21.15.7:0/2017507948 conn(0x5634212be910 msgr2=0x5634212c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- 172.21.15.7:0/2017507948 shutdown_connections 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- 172.21.15.7:0/2017507948 wait complete. 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- start start 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 Processor -- start 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c3d90 0x5634212d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x56342133df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec1a8910 con 0x5634212c3d90 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec1a9d80 con 0x5634212c8f00 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1bf437e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec1a8fe0 con 0x5634212c33a0 2024-03-19T00:39:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.817+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c3d90 0x5634212d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bf437e740 10 client.0 _ll_register_callbacks cb 0x7f1bdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33274/0 (socket says 172.21.15.7:33274) 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 -- 172.21.15.7:0/1649099514 learned_addr learned my addr 172.21.15.7:0/1649099514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beb7fe640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x56342133df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 -- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 msgr2=0x56342133df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x56342133df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 -- 172.21.15.7:0/1649099514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c3d90 msgr2=0x5634212d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c3d90 0x5634212d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 -- 172.21.15.7:0/1649099514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634213428b0 con 0x5634212c33a0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1be3fff640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c3d90 0x5634212d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1beaffd640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212d1dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000c00 tx=0x7f1bdc041d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14872189953964f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bd27fc640 1 -- 172.21.15.7:0/1649099514 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc05a070 con 0x5634212c33a0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bd27fc640 1 -- 172.21.15.7:0/1649099514 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1bdc041460 con 0x5634212c33a0 2024-03-19T00:39:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd40a3b00 con 0x5634212c33a0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bd27fc640 1 -- 172.21.15.7:0/1649099514 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bdc050040 con 0x5634212c33a0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bf437e740 10 client.25507 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.818+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563421343c40 con 0x5634212c33a0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bd27fc640 1 -- 172.21.15.7:0/1649099514 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd40a3b00 con 0x5634212c33a0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bd27fc640 1 -- 172.21.15.7:0/1649099514 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1bdc003960 con 0x5634212c33a0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bf437e740 10 client.25507 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bf437e740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bf437e740 20 client.25507 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bf437e740 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563421344480 0x5634213468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bf437e740 4 client.25507 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56342122f9f0 con 0x563421344480 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1be3fff640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563421344480 0x5634213468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1be3fff640 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563421344480 0x5634213468a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76a42f204fc8cfe server_cookie=20d98fba53705ec2 in_seq=0 out_seq=0 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.819+0000 7f1bd27fc640 10 client.25507 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.820+0000 7f1bd27fc640 1 -- 172.21.15.7:0/1649099514 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56342122f9f0 con 0x563421344480 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.820+0000 7f1bd27fc640 10 client.25507 handle_command_reply: tid=0 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.820+0000 7f1bf437e740 1 client.25507 shutdown 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 msgr2=0x5634212d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212d1dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000c00 tx=0x7f1bdc041d40 comp rx=0 tx=0).stop 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 shutdown_connections 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634212c8f00 0x56342133df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563421344480 0x5634213468a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bd27fc640 0 client.25507 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634212c3d90 0x5634212d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 --2- 172.21.15.7:0/1649099514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634212c33a0 0x5634212d1dd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 >> 172.21.15.7:0/1649099514 conn(0x5634212be910 msgr2=0x5634212cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 shutdown_connections 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 1 -- 172.21.15.7:0/1649099514 wait complete. 2024-03-19T00:39:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf437e740 20 client.25507 trim_cache size 0 max 16384 2024-03-19T00:39:03.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:39:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:39:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "0658daf7-69e4-4c9a-b94c-cf1124fdf6c7", 2024-03-19T00:39:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:39:03.827 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bd80a2910 msgr2=0x7f1bd80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf1790640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bd80a2910 0x7f1bd80a4dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0096c0 tx=0x7f1bdc02c7e0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 msgr2=0x7f1bec198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.821+0000 7f1bf1790640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 0x7f1bec198070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400e9c0 tx=0x7f1bd400ee80 comp rx=0 tx=0).stop 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 shutdown_connections 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bd80a2910 0x7f1bd80a4dd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec107580 0x7f1bec19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec104c50 0x7f1bec198070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 --2- 172.21.15.7:0/3730165518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bec102320 0x7f1bec197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 >> 172.21.15.7:0/3730165518 conn(0x7f1bec0fbcb0 msgr2=0x7f1bec102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 shutdown_connections 2024-03-19T00:39:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.822+0000 7f1bf1790640 1 -- 172.21.15.7:0/3730165518 wait complete. 2024-03-19T00:39:03.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:39:03.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:39:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:03 smithi044 ceph-mon[66366]: pgmap v384: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:39:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 -- 172.21.15.7:0/2280488457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d41022e0 msgr2=0x7ff1d4108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/2280488457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d41022e0 0x7ff1d4108b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009be0 tx=0x7ff1cc02bef0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 -- 172.21.15.7:0/2280488457 shutdown_connections 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/2280488457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d41022e0 0x7ff1d4108b70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/2280488457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d4101920 0x7ff1d4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/2280488457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d4100730 0x7ff1d4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.991+0000 7ff1d9c12640 1 -- 172.21.15.7:0/2280488457 >> 172.21.15.7:0/2280488457 conn(0x7ff1d40fbc80 msgr2=0x7ff1d40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.992+0000 7ff1d9c12640 1 -- 172.21.15.7:0/2280488457 shutdown_connections 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.992+0000 7ff1d9c12640 1 -- 172.21.15.7:0/2280488457 wait complete. 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.992+0000 7ff1d9c12640 1 Processor -- start 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.992+0000 7ff1d9c12640 1 -- start start 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.992+0000 7ff1d9c12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 0x7ff1d406aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.992+0000 7ff1d9c12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d4101920 0x7ff1d419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d9c12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d41022e0 0x7ff1d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d9c12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410c290 con 0x7ff1d4101920 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d9c12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410c110 con 0x7ff1d41022e0 2024-03-19T00:39:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d9c12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410c410 con 0x7ff1d4100730 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 0x7ff1d406aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 0x7ff1d406aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33296/0 (socket says 172.21.15.7:33296) 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 -- 172.21.15.7:0/4038642 learned_addr learned my addr 172.21.15.7:0/4038642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d4101920 0x7ff1d419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d3fff640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d41022e0 0x7ff1d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 -- 172.21.15.7:0/4038642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d41022e0 msgr2=0x7ff1d419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d41022e0 0x7ff1d419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 -- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d4101920 msgr2=0x7ff1d419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d4101920 0x7ff1d419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 -- 172.21.15.7:0/4038642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d419f680 con 0x7ff1d4100730 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 0x7ff1d406aa50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400e9c0 tx=0x7ff1c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d0ff9640 1 -- 172.21.15.7:0/4038642 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400cd80 con 0x7ff1d4100730 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d0ff9640 1 -- 172.21.15.7:0/4038642 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1c40035f0 con 0x7ff1d4100730 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d419f910 con 0x7ff1d4100730 2024-03-19T00:39:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.993+0000 7ff1d0ff9640 1 -- 172.21.15.7:0/4038642 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4010690 con 0x7ff1d4100730 2024-03-19T00:39:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.994+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d419fe60 con 0x7ff1d4100730 2024-03-19T00:39:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.996+0000 7ff1d0ff9640 1 -- 172.21.15.7:0/4038642 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 137340+0+0 (secure 0 0 0) 0x7ff1c40037a0 con 0x7ff1d4100730 2024-03-19T00:39:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.996+0000 7ff1d0ff9640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1bc0a2890 0x7ff1bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.996+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1bc0a2890 0x7ff1bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.996+0000 7ff1d0ff9640 1 -- 172.21.15.7:0/4038642 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c40cf2e0 con 0x7ff1d4100730 2024-03-19T00:39:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.997+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1bc0a2890 0x7ff1bc0a4d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1d406a7c0 tx=0x7ff1c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.997+0000 7ff1dc803740 1 Processor -- start 2024-03-19T00:39:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.997+0000 7ff1dc803740 1 -- start start 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.997+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 0x55baab6c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.997+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c1440 0x55baab6c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.997+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 0x55baab6c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 0x55baab6c0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42188/0 (socket says 172.21.15.7:42188) 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c1440 0x55baab6c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c1440 0x55baab6c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33306/0 (socket says 172.21.15.7:33306) 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 -- 172.21.15.7:0/3034757128 learned_addr learned my addr 172.21.15.7:0/3034757128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55baab625500 con 0x55baab6c65b0 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55baab5cdbe0 con 0x55baab6c0a50 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55baab6215a0 con 0x55baab6c1440 2024-03-19T00:39:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d3fff640 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 -- 172.21.15.7:0/3034757128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c1440 msgr2=0x55baab6c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c1440 0x55baab6c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 -- 172.21.15.7:0/3034757128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 msgr2=0x55baab6c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.998+0000 7ff1d37fe640 1 -- 172.21.15.7:0/3034757128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baab5ef000 con 0x55baab6c0a50 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.999+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 0x55baab6c0e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40a3a30 tx=0x7ff1c409b120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47647df60f829af0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.999+0000 7ff1b9ffb640 1 -- 172.21.15.7:0/3034757128 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c409bec0 con 0x55baab6c0a50 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.999+0000 7ff1b9ffb640 1 -- 172.21.15.7:0/3034757128 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1c409b7c0 con 0x55baab6c0a50 2024-03-19T00:39:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:03.999+0000 7ff1b9ffb640 1 -- 172.21.15.7:0/3034757128 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4097e30 con 0x55baab6c0a50 2024-03-19T00:39:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 msgr2=0x55baab6c0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 0x55baab6c0e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40a3a30 tx=0x7ff1c409b120 comp rx=0 tx=0).stop 2024-03-19T00:39:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 shutdown_connections 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c1440 0x55baab6c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 --2- 172.21.15.7:0/3034757128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c0a50 0x55baab6c0e70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 >> 172.21.15.7:0/3034757128 conn(0x55baab6bbfc0 msgr2=0x55baab6bc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 shutdown_connections 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 -- 172.21.15.7:0/3034757128 wait complete. 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.000+0000 7ff1dc803740 1 -- start start 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1dc803740 1 Processor -- start 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1dc803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55baab6c9500 con 0x55baab6c1440 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1dc803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55baab5e1160 con 0x55baab6c65b0 2024-03-19T00:39:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47192/0 (socket says 172.21.15.7:47192) 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33308/0 (socket says 172.21.15.7:33308) 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.001+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/4096623955 learned_addr learned my addr 172.21.15.7:0/4096623955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d3fff640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55baab6c9680 con 0x55baab6c0a50 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1dc803740 10 client.0 _ll_register_callbacks cb 0x55baab61da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/4096623955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 msgr2=0x55baab6cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 msgr2=0x55baab6d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/4096623955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baab4037d0 con 0x55baab6c1440 2024-03-19T00:39:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c409b7c0 con 0x55baab6c1440 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6cb3d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff1c801b9b0 tx=0x7ff1c8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd284d76511a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.002+0000 7ff1b2ffd640 10 client.15756 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/4096623955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800ed40 con 0x55baab6c1440 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/4096623955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c80047a0 con 0x55baab6c1440 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/4096623955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1c80049e0 con 0x55baab6c1440 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55baab3393f0 con 0x55baab6c1440 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/4096623955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1c8026070 con 0x55baab6c1440 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 20 client.15756 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab6d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 4 client.15756 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baab632a40 con 0x55baab6d6b90 2024-03-19T00:39:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.003+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab6d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.004+0000 7ff1b2ffd640 10 client.15756 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.004+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab6d9050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71f80eb4e16f22 server_cookie=7c7bc8b3537da652 in_seq=0 out_seq=0 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.004+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/4096623955 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55baab632a40 con 0x55baab6d6b90 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.004+0000 7ff1b2ffd640 10 client.15756 handle_command_reply: tid=0 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.004+0000 7ff1dc803740 1 client.15756 shutdown 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 msgr2=0x55baab6cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6cb3d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff1c801b9b0 tx=0x7ff1c8013040 comp rx=0 tx=0).stop 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 shutdown_connections 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab6d9050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6cb3d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1b2ffd640 0 client.15756 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 --2- 172.21.15.7:0/4096623955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 >> 172.21.15.7:0/4096623955 conn(0x55baab6bbfc0 msgr2=0x55baab6bb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 shutdown_connections 2024-03-19T00:39:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 1 -- 172.21.15.7:0/4096623955 wait complete. 2024-03-19T00:39:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.005+0000 7ff1dc803740 20 client.15756 trim_cache size 0 max 16384 2024-03-19T00:39:04.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 Processor -- start 2024-03-19T00:39:04.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 -- start start 2024-03-19T00:39:04.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41acbb0 con 0x55baab6c1440 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41aa480 con 0x55baab6c65b0 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1dc803740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41a82d0 con 0x55baab6c0a50 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6ba950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42208/0 (socket says 172.21.15.7:42208) 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1d3fff640 1 -- 172.21.15.7:0/2973886305 learned_addr learned my addr 172.21.15.7:0/2973886305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.039+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d3fff640 1 -- 172.21.15.7:0/2973886305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 msgr2=0x55baab6b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d3fff640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d3fff640 1 -- 172.21.15.7:0/2973886305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 msgr2=0x55baab6d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d3fff640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d3fff640 1 -- 172.21.15.7:0/2973886305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baab2f90f0 con 0x55baab6c65b0 2024-03-19T00:39:04.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:39:04.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:04.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1d3fff640 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6ba950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc02c500 tx=0x7ff1cc033590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ce478d3c413125 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:04.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1b97fa640 1 -- 172.21.15.7:0/2973886305 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0337c0 con 0x55baab6c65b0 2024-03-19T00:39:04.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1b97fa640 1 -- 172.21.15.7:0/2973886305 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1cc003970 con 0x55baab6c65b0 2024-03-19T00:39:04.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.040+0000 7ff1b97fa640 1 -- 172.21.15.7:0/2973886305 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc02cdc0 con 0x55baab6c65b0 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 -- 172.21.15.7:0/2973886305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 msgr2=0x55baab6ba950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6ba950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc02c500 tx=0x7ff1cc033590 comp rx=0 tx=0).stop 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 -- 172.21.15.7:0/2973886305 shutdown_connections 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c65b0 0x55baab6ba950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c1440 0x55baab6d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2973886305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 -- 172.21.15.7:0/2973886305 >> 172.21.15.7:0/2973886305 conn(0x55baab6bbfc0 msgr2=0x55baab6c2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 -- 172.21.15.7:0/2973886305 shutdown_connections 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 -- 172.21.15.7:0/2973886305 wait complete. 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.041+0000 7ff1dc803740 1 -- start start 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 Processor -- start 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 0x55baab6d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41a8940 con 0x55baab6c65b0 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41a9db0 con 0x55baab6c1440 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41a9010 con 0x55baab6c0a50 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1dc803740 10 client.0 _ll_register_callbacks cb 0x7ff1c8009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 0x55baab6d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 0x55baab6d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42222/0 (socket says 172.21.15.7:42222) 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33332/0 (socket says 172.21.15.7:33332) 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/2420836683 learned_addr learned my addr 172.21.15.7:0/2420836683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/2420836683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 msgr2=0x55baab6cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/2420836683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 msgr2=0x55baab6d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.042+0000 7ff1d2ffd640 1 -- 172.21.15.7:0/2420836683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baab6d1980 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1d2ffd640 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 0x55baab6d0420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8000c00 tx=0x7ff1c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33368f6c392ab2b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/2420836683 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c803e060 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc003970 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/2420836683 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c8027900 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/2420836683 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1c8045050 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1dc803740 10 client.25613 fetch_fsmap learned FSMap version 25 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55baab741760 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.043+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/2420836683 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1cc003970 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/2420836683 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1c8026040 con 0x55baab6c1440 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 10 client.25613 fetch_fsmap finished waiting for FSMap version 25 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 20 client.25613 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab744110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 4 client.25613 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55baab62cd90 con 0x55baab6d6b90 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab744110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1b2ffd640 10 client.25613 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1d37fe640 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab744110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abdcae719b2e391 server_cookie=787e22e3e05285bf in_seq=0 out_seq=0 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1b2ffd640 1 -- 172.21.15.7:0/2420836683 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55baab62cd90 con 0x55baab6d6b90 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1b2ffd640 10 client.25613 handle_command_reply: tid=0 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.044+0000 7ff1dc803740 1 client.25613 shutdown 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 msgr2=0x55baab6d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 0x55baab6d0420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8000c00 tx=0x7ff1c8004730 comp rx=0 tx=0).stop 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 shutdown_connections 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55baab6d6b90 0x55baab744110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baab6c65b0 0x55baab6d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55baab6c1440 0x55baab6d0420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 --2- 172.21.15.7:0/2420836683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55baab6c0a50 0x55baab6cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 >> 172.21.15.7:0/2420836683 conn(0x55baab6bbfc0 msgr2=0x55baab6bf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1b2ffd640 0 client.25613 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 shutdown_connections 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 1 -- 172.21.15.7:0/2420836683 wait complete. 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1dc803740 20 client.25613 trim_cache size 0 max 16384 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:39:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "0658daf7-69e4-4c9a-b94c-cf1124fdf6c7": { 2024-03-19T00:39:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:39:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "0658daf7-69e4-4c9a-b94c-cf1124fdf6c7", 2024-03-19T00:39:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:39:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:39:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:39:04.052 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1bc0a2890 msgr2=0x7ff1bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1bc0a2890 0x7ff1bc0a4d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff1d406a7c0 tx=0x7ff1c8009260 comp rx=0 tx=0).stop 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 msgr2=0x7ff1d406aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.045+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 0x7ff1d406aa50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400e9c0 tx=0x7ff1c400ee80 comp rx=0 tx=0).stop 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 shutdown_connections 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff1bc0a2890 0x7ff1bc0a4d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d41022e0 0x7ff1d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d4101920 0x7ff1d419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 --2- 172.21.15.7:0/4038642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1d4100730 0x7ff1d406aa50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 >> 172.21.15.7:0/4038642 conn(0x7ff1d40fbc80 msgr2=0x7ff1d4104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 shutdown_connections 2024-03-19T00:39:04.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:04.046+0000 7ff1d9c12640 1 -- 172.21.15.7:0/4038642 wait complete. 2024-03-19T00:39:04.060 INFO:tasks.cephfs.filesystem:scrub status for tag:0658daf7-69e4-4c9a-b94c-cf1124fdf6c7 - {'path': '/', 'tag': '0658daf7-69e4-4c9a-b94c-cf1124fdf6c7', 'options': 'recursive,force'} 2024-03-19T00:39:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:04 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:39:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:04 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:04 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:39:04.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:04 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:04 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:39:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:04 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:04 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:39:04.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:04 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:04.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:04 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:39:04.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:04 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:04.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:04 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:39:04.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:04 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:05.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:05 smithi007 ceph-mon[62854]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:39:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:05 smithi204 ceph-mon[66572]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:39:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:05 smithi044 ceph-mon[66366]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:39:06.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:06 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:39:06.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:06 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:39:06.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:06 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:39:07.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:07 smithi007 ceph-mon[62854]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:39:07.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:07 smithi204 ceph-mon[66572]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:39:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:07 smithi044 ceph-mon[66366]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:39:09.658 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:09 smithi044 ceph-mon[66366]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-19T00:39:09.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:09 smithi007 ceph-mon[62854]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-19T00:39:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:09 smithi204 ceph-mon[66572]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to 'e48b1f6c0608b2615e1890cc46286c3877d6e910'. 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.906 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-19T00:39:09.907 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at e48b1f6c060 testfix 2024-03-19T00:39:09.911 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-19T00:39:09.940 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-19T00:39:09.942 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-19T00:39:09.942 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-19T00:39:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-19T00:39:10.842 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-19T00:39:10.882 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-19T00:39:10.884 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-19T00:39:10.884 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-19T00:39:10.924 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-19T00:39:10.927 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:39:10.927 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-19T00:39:10.983 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-03-19T00:39:10.984 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-03-19T00:39:10.985 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-03-19T00:39:11.048 INFO:tasks.workunit.client.0.smithi007.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-03-19T00:39:11.048 INFO:tasks.workunit.client.0.smithi007.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-03-19T00:39:11.049 INFO:tasks.workunit.client.0.smithi007.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-03-19T00:39:11.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:11 smithi007 ceph-mon[62854]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-19T00:39:11.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:11 smithi204 ceph-mon[66572]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-19T00:39:11.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:11 smithi044 ceph-mon[66366]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-19T00:39:13.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:13 smithi007 ceph-mon[62854]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-19T00:39:13.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:13 smithi204 ceph-mon[66572]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-19T00:39:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:13 smithi044 ceph-mon[66366]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-19T00:39:15.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:15 smithi007 ceph-mon[62854]: pgmap v390: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-19T00:39:15.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:15 smithi204 ceph-mon[66572]: pgmap v390: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-19T00:39:15.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:15 smithi044 ceph-mon[66366]: pgmap v390: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-19T00:39:17.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:17 smithi007 ceph-mon[62854]: pgmap v391: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-03-19T00:39:17.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:17 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:39:17.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:17 smithi204 ceph-mon[66572]: pgmap v391: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-03-19T00:39:17.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:17 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:39:17.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:17 smithi044 ceph-mon[66366]: pgmap v391: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-03-19T00:39:17.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:17 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:39:19.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:19 smithi007 ceph-mon[62854]: pgmap v392: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-03-19T00:39:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:19 smithi204 ceph-mon[66572]: pgmap v392: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-03-19T00:39:19.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:19 smithi044 ceph-mon[66366]: pgmap v392: 129 pgs: 129 active+clean; 693 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-03-19T00:39:21.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:21 smithi007 ceph-mon[62854]: pgmap v393: 129 pgs: 129 active+clean; 744 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-03-19T00:39:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:21 smithi204 ceph-mon[66572]: pgmap v393: 129 pgs: 129 active+clean; 744 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-03-19T00:39:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:21 smithi044 ceph-mon[66366]: pgmap v393: 129 pgs: 129 active+clean; 744 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-03-19T00:39:23.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:23 smithi007 ceph-mon[62854]: pgmap v394: 129 pgs: 129 active+clean; 744 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.3 KiB/s wr, 9 op/s 2024-03-19T00:39:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:23 smithi204 ceph-mon[66572]: pgmap v394: 129 pgs: 129 active+clean; 744 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.3 KiB/s wr, 9 op/s 2024-03-19T00:39:23.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:23 smithi044 ceph-mon[66366]: pgmap v394: 129 pgs: 129 active+clean; 744 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.3 KiB/s wr, 9 op/s 2024-03-19T00:39:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:25 smithi007 ceph-mon[62854]: pgmap v395: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-03-19T00:39:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:25 smithi204 ceph-mon[66572]: pgmap v395: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-03-19T00:39:25.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:25 smithi044 ceph-mon[66366]: pgmap v395: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-03-19T00:39:27.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:27 smithi007 ceph-mon[62854]: pgmap v396: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-19T00:39:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:27 smithi204 ceph-mon[66572]: pgmap v396: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-19T00:39:27.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:27 smithi044 ceph-mon[66366]: pgmap v396: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-19T00:39:29.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:29 smithi007 ceph-mon[62854]: pgmap v397: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-03-19T00:39:29.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:29 smithi007 ceph-mon[62854]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-19T00:39:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:29 smithi204 ceph-mon[66572]: pgmap v397: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-03-19T00:39:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:29 smithi204 ceph-mon[66572]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-19T00:39:29.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:29 smithi044 ceph-mon[66366]: pgmap v397: 129 pgs: 129 active+clean; 772 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-03-19T00:39:29.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:29 smithi044 ceph-mon[66366]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-19T00:39:31.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:31 smithi007 ceph-mon[62854]: pgmap v398: 129 pgs: 129 active+clean; 789 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-19T00:39:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:31 smithi204 ceph-mon[66572]: pgmap v398: 129 pgs: 129 active+clean; 789 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-19T00:39:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:31 smithi044 ceph-mon[66366]: pgmap v398: 129 pgs: 129 active+clean; 789 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-19T00:39:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:33 smithi007 ceph-mon[62854]: pgmap v399: 129 pgs: 129 active+clean; 789 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-03-19T00:39:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:33 smithi204 ceph-mon[66572]: pgmap v399: 129 pgs: 129 active+clean; 789 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-03-19T00:39:33.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:33 smithi044 ceph-mon[66366]: pgmap v399: 129 pgs: 129 active+clean; 789 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-03-19T00:39:34.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:39:34.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:39:34.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 -- 172.21.15.7:0/3482429877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70104c40 msgr2=0x7f0a70107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/3482429877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70104c40 0x7f0a70107030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009900 tx=0x7f0a6002bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 -- 172.21.15.7:0/3482429877 shutdown_connections 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/3482429877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a70107570 0x7f0a70109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/3482429877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70104c40 0x7f0a70107030 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/3482429877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 0x7f0a70104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 -- 172.21.15.7:0/3482429877 >> 172.21.15.7:0/3482429877 conn(0x7f0a700fbc80 msgr2=0x7f0a700fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 -- 172.21.15.7:0/3482429877 shutdown_connections 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.178+0000 7f0a77ec0640 1 -- 172.21.15.7:0/3482429877 wait complete. 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 Processor -- start 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 -- start start 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 0x7f0a70197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a70104c40 0x7f0a70197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 0x7f0a7019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7010d330 con 0x7f0a70107570 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7010d1b0 con 0x7f0a70102310 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a77ec0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7010d4b0 con 0x7f0a70104c40 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a7645d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 0x7f0a7019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 0x7f0a70197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a7545b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a70104c40 0x7f0a70197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a7645d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 0x7f0a7019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42706/0 (socket says 172.21.15.7:42706) 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 0x7f0a70197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39426/0 (socket says 172.21.15.7:39426) 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.179+0000 7f0a7645d640 1 -- 172.21.15.7:0/1500152618 learned_addr learned my addr 172.21.15.7:0/1500152618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a7645d640 1 -- 172.21.15.7:0/1500152618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a70104c40 msgr2=0x7f0a70197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a7645d640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a70104c40 0x7f0a70197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a7645d640 1 -- 172.21.15.7:0/1500152618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 msgr2=0x7f0a70197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a7645d640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 0x7f0a70197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a7645d640 1 -- 172.21.15.7:0/1500152618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7019f6c0 con 0x7f0a70107570 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a7645d640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 0x7f0a7019f050 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800eb10 tx=0x7f0a6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a66ffd640 1 -- 172.21.15.7:0/1500152618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004b20 con 0x7f0a70107570 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a66ffd640 1 -- 172.21.15.7:0/1500152618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a68004cc0 con 0x7f0a70107570 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7019f9b0 con 0x7f0a70107570 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a66ffd640 1 -- 172.21.15.7:0/1500152618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68010690 con 0x7f0a70107570 2024-03-19T00:39:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.180+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7019fed0 con 0x7f0a70107570 2024-03-19T00:39:34.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.183+0000 7f0a66ffd640 1 -- 172.21.15.7:0/1500152618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0a68005020 con 0x7f0a70107570 2024-03-19T00:39:34.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.183+0000 7f0a7ab84740 1 Processor -- start 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.183+0000 7f0a7ab84740 1 -- start start 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bf2262a500 con 0x55bf226c6440 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf225d2be0 con 0x55bf226c5a50 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bf226265a0 con 0x55bf226cb5b0 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42718/0 (socket says 172.21.15.7:42718) 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3338344457 learned_addr learned my addr 172.21.15.7:0/3338344457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a66ffd640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a5c0a2880 0x7f0a5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7545b640 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7545b640 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a7645d640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a5c0a2880 0x7f0a5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3338344457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 msgr2=0x55bf226cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3338344457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 msgr2=0x55bf226c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3338344457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf225f4000 con 0x55bf226c6440 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a66ffd640 1 -- 172.21.15.7:0/1500152618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a680cf330 con 0x7f0a70107570 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226caf40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009cf0 tx=0x7f0a6c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d042041bfb32ea34 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.184+0000 7f0a5bfff640 1 -- 172.21.15.7:0/3338344457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c01e070 con 0x55bf226c6440 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7645d640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a5c0a2880 0x7f0a5c0a4d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a680a3b80 tx=0x7f0a68096340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a5bfff640 1 -- 172.21.15.7:0/3338344457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a6c009de0 con 0x55bf226c6440 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a5bfff640 1 -- 172.21.15.7:0/3338344457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c019450 con 0x55bf226c6440 2024-03-19T00:39:34.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3338344457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 msgr2=0x55bf226caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226caf40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009cf0 tx=0x7f0a6c011810 comp rx=0 tx=0).stop 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3338344457 shutdown_connections 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226caf40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3338344457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3338344457 >> 172.21.15.7:0/3338344457 conn(0x55bf226c0fc0 msgr2=0x55bf226c13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3338344457 shutdown_connections 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.185+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3338344457 wait complete. 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 -- start start 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 Processor -- start 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bf226ce560 con 0x55bf226c6440 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf225e6160 con 0x55bf226c5a50 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bf226ce6e0 con 0x55bf226cb5b0 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7ab84740 10 client.0 _ll_register_callbacks cb 0x55bf22622a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42734/0 (socket says 172.21.15.7:42734) 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a75c5c640 1 -- 172.21.15.7:0/2389499598 learned_addr learned my addr 172.21.15.7:0/2389499598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.186+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a75c5c640 1 -- 172.21.15.7:0/2389499598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 msgr2=0x55bf226d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a75c5c640 1 -- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 msgr2=0x55bf226cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a75c5c640 1 -- 172.21.15.7:0/2389499598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf224087d0 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d0380 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0052f0 tx=0x7f0a6c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76ebb73f022b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a597fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a597fa640 1 -- 172.21.15.7:0/2389499598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c01e040 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a597fa640 1 -- 172.21.15.7:0/2389499598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a6c0054d0 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a6c009de0 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a597fa640 1 -- 172.21.15.7:0/2389499598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c030050 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a7ab84740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf2233e3f0 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a597fa640 1 -- 172.21.15.7:0/2389499598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a6c009de0 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a597fa640 1 -- 172.21.15.7:0/2389499598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0a6c021070 con 0x55bf226c6440 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a7ab84740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.187+0000 7f0a7ab84740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a7ab84740 20 client.15774 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf226dba50 0x55bf226ddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a7ab84740 4 client.15774 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf22637a40 con 0x55bf226dba50 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf226dba50 0x55bf226ddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf226dba50 0x55bf226ddef0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66c1007c77426bf server_cookie=aa56660b73345f7 in_seq=0 out_seq=0 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.188+0000 7f0a597fa640 10 client.15774 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a597fa640 1 -- 172.21.15.7:0/2389499598 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf22637a40 con 0x55bf226dba50 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a597fa640 10 client.15774 handle_command_reply: tid=0 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 client.15774 shutdown 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 msgr2=0x55bf226d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d0380 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0052f0 tx=0x7f0a6c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 shutdown_connections 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf226dba50 0x55bf226ddef0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a597fa640 0 client.15774 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:34.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2389499598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 >> 172.21.15.7:0/2389499598 conn(0x55bf226c0fc0 msgr2=0x55bf226c01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 shutdown_connections 2024-03-19T00:39:34.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2389499598 wait complete. 2024-03-19T00:39:34.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.189+0000 7f0a7ab84740 20 client.15774 trim_cache size 0 max 16384 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 Processor -- start 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 -- start start 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 0x55bf226d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701acbb0 con 0x55bf226cb5b0 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701aa480 con 0x55bf226c6440 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701a82d0 con 0x55bf226c5a50 2024-03-19T00:39:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7545b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 0x55bf226d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7545b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 0x55bf226d7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42748/0 (socket says 172.21.15.7:42748) 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7545b640 1 -- 172.21.15.7:0/2309917500 learned_addr learned my addr 172.21.15.7:0/2309917500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.207+0000 7f0a7545b640 1 -- 172.21.15.7:0/2309917500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226c5a50 msgr2=0x55bf226cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7545b640 1 -- 172.21.15.7:0/2309917500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c6440 msgr2=0x55bf226d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7545b640 1 -- 172.21.15.7:0/2309917500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf222fe0f0 con 0x55bf226cb5b0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7545b640 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 0x55bf226d7380 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009230 tx=0x7f0a60009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61399180101a3af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a5b7fe640 1 -- 172.21.15.7:0/2309917500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60050ba0 con 0x55bf226cb5b0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a5b7fe640 1 -- 172.21.15.7:0/2309917500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a60050810 con 0x55bf226cb5b0 2024-03-19T00:39:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a5b7fe640 1 -- 172.21.15.7:0/2309917500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600046c0 con 0x55bf226cb5b0 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2309917500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 msgr2=0x55bf226d7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 0x55bf226d7380 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009230 tx=0x7f0a60009260 comp rx=0 tx=0).stop 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2309917500 shutdown_connections 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226cb5b0 0x55bf226d7380 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c6440 0x55bf226d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/2309917500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226c5a50 0x55bf226cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.208+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2309917500 >> 172.21.15.7:0/2309917500 conn(0x55bf226c0fc0 msgr2=0x55bf226c13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2309917500 shutdown_connections 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 -- 172.21.15.7:0/2309917500 wait complete. 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 -- start start 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 Processor -- start 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701a8940 con 0x55bf226c6440 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701a9db0 con 0x55bf226c5a50 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701a9010 con 0x55bf226cb5b0 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7ab84740 10 client.0 _ll_register_callbacks cb 0x7f0a6c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a75c5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42750/0 (socket says 172.21.15.7:42750) 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3263718003 learned_addr learned my addr 172.21.15.7:0/3263718003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7545b640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.209+0000 7f0a7545b640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3263718003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 msgr2=0x55bf226d6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 msgr2=0x55bf226d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226d4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a75c5c640 1 -- 172.21.15.7:0/3263718003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf226d6610 con 0x55bf226c6440 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a75c5c640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d5010 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c004790 tx=0x7f0a6c004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926d783358297b21 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a597fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a597fa640 1 -- 172.21.15.7:0/3263718003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c039060 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a597fa640 1 -- 172.21.15.7:0/3263718003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a6c019450 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60050810 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a597fa640 1 -- 172.21.15.7:0/3263718003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c040050 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a7ab84740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf22741df0 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a597fa640 1 -- 172.21.15.7:0/3263718003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a60050810 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.210+0000 7f0a597fa640 1 -- 172.21.15.7:0/3263718003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0a6c021040 con 0x55bf226c6440 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 20 client.15780 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf227427f0 0x55bf22744c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 4 client.15780 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:39:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf22631d90 con 0x55bf227427f0 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7545b640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf227427f0 0x55bf22744c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7545b640 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf227427f0 0x55bf22744c10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899414f88b8b91ff server_cookie=be5f47d4c94f9309 in_seq=0 out_seq=0 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a597fa640 10 client.15780 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a597fa640 1 -- 172.21.15.7:0/3263718003 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bf22631d90 con 0x55bf227427f0 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a597fa640 10 client.15780 handle_command_reply: tid=0 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 client.15780 shutdown 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 msgr2=0x55bf226d5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d5010 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c004790 tx=0x7f0a6c004c10 comp rx=0 tx=0).stop 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 shutdown_connections 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf226cb5b0 0x55bf226d6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bf227427f0 0x55bf22744c10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf226c6440 0x55bf226d5010 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 --2- 172.21.15.7:0/3263718003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf226c5a50 0x55bf226d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a597fa640 0 client.15780 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 >> 172.21.15.7:0/3263718003 conn(0x55bf226c0fc0 msgr2=0x55bf226cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.211+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 shutdown_connections 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a7ab84740 1 -- 172.21.15.7:0/3263718003 wait complete. 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a7ab84740 20 client.15780 trim_cache size 0 max 16384 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:39:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a5c0a2880 msgr2=0x7f0a5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a5c0a2880 0x7f0a5c0a4d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a680a3b80 tx=0x7f0a68096340 comp rx=0 tx=0).stop 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 msgr2=0x7f0a7019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 0x7f0a7019f050 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800eb10 tx=0x7f0a6800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 shutdown_connections 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a5c0a2880 0x7f0a5c0a4d40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a70107570 0x7f0a7019f050 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a70104c40 0x7f0a70197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 --2- 172.21.15.7:0/1500152618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a70102310 0x7f0a70197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 >> 172.21.15.7:0/1500152618 conn(0x7f0a700fbc80 msgr2=0x7f0a701041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 shutdown_connections 2024-03-19T00:39:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.212+0000 7f0a77ec0640 1 -- 172.21.15.7:0/1500152618 wait complete. 2024-03-19T00:39:34.224 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:39:34.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:39:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 -- 172.21.15.7:0/3547803991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380102aa0 msgr2=0x7f1380102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 --2- 172.21.15.7:0/3547803991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380102aa0 0x7f1380102f20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1370009900 tx=0x7f137002bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 -- 172.21.15.7:0/3547803991 shutdown_connections 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 --2- 172.21.15.7:0/3547803991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1380103460 0x7f1380109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 --2- 172.21.15.7:0/3547803991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380102aa0 0x7f1380102f20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 --2- 172.21.15.7:0/3547803991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13801018b0 0x7f1380101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.326+0000 7f1387663640 1 -- 172.21.15.7:0/3547803991 >> 172.21.15.7:0/3547803991 conn(0x7f13800fce40 msgr2=0x7f13800ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 -- 172.21.15.7:0/3547803991 shutdown_connections 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 -- 172.21.15.7:0/3547803991 wait complete. 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 Processor -- start 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 -- start start 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13801018b0 0x7f1380197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1380102aa0 0x7f1380197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 0x7f138019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f138010d370 con 0x7f1380103460 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f138010d1f0 con 0x7f13801018b0 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1387663640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f138010d4f0 con 0x7f1380102aa0 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1385c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 0x7f138019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1385c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 0x7f138019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42768/0 (socket says 172.21.15.7:42768) 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1385c00640 1 -- 172.21.15.7:0/2942696699 learned_addr learned my addr 172.21.15.7:0/2942696699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f13853ff640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13801018b0 0x7f1380197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.327+0000 7f1384bfe640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1380102aa0 0x7f1380197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1385c00640 1 -- 172.21.15.7:0/2942696699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1380102aa0 msgr2=0x7f1380197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1385c00640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1380102aa0 0x7f1380197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1385c00640 1 -- 172.21.15.7:0/2942696699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13801018b0 msgr2=0x7f1380197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1385c00640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13801018b0 0x7f1380197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1385c00640 1 -- 172.21.15.7:0/2942696699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138019f6c0 con 0x7f1380103460 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f13853ff640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13801018b0 0x7f1380197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:34.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1385c00640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 0x7f138019f050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f137800efd0 tx=0x7f137800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378003b70 con 0x7f1380103460 2024-03-19T00:39:34.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1378003d10 con 0x7f1380103460 2024-03-19T00:39:34.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138019f9b0 con 0x7f1380103460 2024-03-19T00:39:34.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378010980 con 0x7f1380103460 2024-03-19T00:39:34.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.328+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138019fed0 con 0x7f1380103460 2024-03-19T00:39:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.329+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1380108bf0 con 0x7f1380103460 2024-03-19T00:39:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.330+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1378004b40 con 0x7f1380103460 2024-03-19T00:39:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.331+0000 7f13767fc640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f136c0a2880 0x7f136c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.331+0000 7f13853ff640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f136c0a2880 0x7f136c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.331+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f13780cf520 con 0x7f1380103460 2024-03-19T00:39:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.331+0000 7f13853ff640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f136c0a2880 0x7f136c0a4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1380102900 tx=0x7f137c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.335+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f137809c200 con 0x7f1380103460 2024-03-19T00:39:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.557+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f138002d3b0 con 0x7f1380103460 2024-03-19T00:39:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.558+0000 7f13767fc640 1 -- 172.21.15.7:0/2942696699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11856 (secure 0 0 0) 0x7f137809bb00 con 0x7f1380103460 2024-03-19T00:39:34.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:39:34.559 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:39:16.038000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:39:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-19T00:39:34.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.559+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f136c0a2880 msgr2=0x7f136c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.559+0000 7f1387663640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f136c0a2880 0x7f136c0a4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1380102900 tx=0x7f137c005f90 comp rx=0 tx=0).stop 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 msgr2=0x7f138019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 0x7f138019f050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f137800efd0 tx=0x7f137800c6a0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 shutdown_connections 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f136c0a2880 0x7f136c0a4d40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1380103460 0x7f138019f050 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1380102aa0 0x7f1380197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 --2- 172.21.15.7:0/2942696699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13801018b0 0x7f1380197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 >> 172.21.15.7:0/2942696699 conn(0x7f13800fce40 msgr2=0x7f1380105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 shutdown_connections 2024-03-19T00:39:34.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.560+0000 7f1387663640 1 -- 172.21.15.7:0/2942696699 wait complete. 2024-03-19T00:39:34.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:39:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 -- 172.21.15.7:0/829407711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae68100720 msgr2=0x7fae68100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 --2- 172.21.15.7:0/829407711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae68100720 0x7fae68100b40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae6402bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 -- 172.21.15.7:0/829407711 shutdown_connections 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 --2- 172.21.15.7:0/829407711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae681022d0 0x7fae68108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 --2- 172.21.15.7:0/829407711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68101910 0x7fae68101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 --2- 172.21.15.7:0/829407711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae68100720 0x7fae68100b40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 -- 172.21.15.7:0/829407711 >> 172.21.15.7:0/829407711 conn(0x7fae680fbc50 msgr2=0x7fae680fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 -- 172.21.15.7:0/829407711 shutdown_connections 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 -- 172.21.15.7:0/829407711 wait complete. 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.676+0000 7fae70710640 1 Processor -- start 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 -- start start 2024-03-19T00:39:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae68100720 0x7fae6810b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68101910 0x7fae681096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 0x7fae68109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810d330 con 0x7fae681022d0 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810d1b0 con 0x7fae68100720 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810d4b0 con 0x7fae68101910 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 0x7fae68109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68101910 0x7fae681096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 0x7fae68109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39238/0 (socket says 172.21.15.7:39238) 2024-03-19T00:39:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae68100720 0x7fae6810b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 -- 172.21.15.7:0/2411034649 learned_addr learned my addr 172.21.15.7:0/2411034649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 -- 172.21.15.7:0/2411034649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68101910 msgr2=0x7fae681096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68101910 0x7fae681096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 -- 172.21.15.7:0/2411034649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae68100720 msgr2=0x7fae6810b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae68100720 0x7fae6810b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 -- 172.21.15.7:0/2411034649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6810a4e0 con 0x7fae681022d0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae6ecad640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 0x7fae68109c20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae6000eb10 tx=0x7fae6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae5f7fe640 1 -- 172.21.15.7:0/2411034649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60004b20 con 0x7fae681022d0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae5f7fe640 1 -- 172.21.15.7:0/2411034649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae60004cc0 con 0x7fae681022d0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae681a1bb0 con 0x7fae681022d0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.677+0000 7fae5f7fe640 1 -- 172.21.15.7:0/2411034649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60010690 con 0x7fae681022d0 2024-03-19T00:39:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.678+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae681a20d0 con 0x7fae681022d0 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae5f7fe640 1 -- 172.21.15.7:0/2411034649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fae60005020 con 0x7fae681022d0 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 Processor -- start 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 -- start start 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cc6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477ccb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a477c2a4b0 con 0x55a477cc62f0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a477bd2d70 con 0x55a477cc6ce0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae733de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a477c26550 con 0x55a477ccbe50 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.680+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cc6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae5f7fe640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae540a2880 0x7fae540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cc6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39248/0 (socket says 172.21.15.7:39248) 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6dcab640 1 -- 172.21.15.7:0/311748648 learned_addr learned my addr 172.21.15.7:0/311748648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477ccb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6dcab640 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6ecad640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae540a2880 0x7fae540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae5f7fe640 1 -- 172.21.15.7:0/2411034649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae600cf2f0 con 0x7fae681022d0 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 -- 172.21.15.7:0/311748648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 msgr2=0x55a477cce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 -- 172.21.15.7:0/311748648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 msgr2=0x55a477cc6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cc6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 -- 172.21.15.7:0/311748648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a477bf5210 con 0x55a477cc6ce0 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477ccb7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae64036040 tx=0x7fae64009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8064fda2cb352bd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae6ecad640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae540a2880 0x7fae540a4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae600a3b40 tx=0x7fae60090db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae5cff9640 1 -- 172.21.15.7:0/311748648 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64045020 con 0x55a477cc6ce0 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae5cff9640 1 -- 172.21.15.7:0/311748648 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae64004040 con 0x55a477cc6ce0 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.681+0000 7fae5cff9640 1 -- 172.21.15.7:0/311748648 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640046c0 con 0x55a477cc6ce0 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 -- 172.21.15.7:0/311748648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 msgr2=0x55a477ccb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477ccb7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae64036040 tx=0x7fae64009560 comp rx=0 tx=0).stop 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 -- 172.21.15.7:0/311748648 shutdown_connections 2024-03-19T00:39:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477ccb7e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 --2- 172.21.15.7:0/311748648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cc6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 -- 172.21.15.7:0/311748648 >> 172.21.15.7:0/311748648 conn(0x55a477cc1860 msgr2=0x55a477cc1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 -- 172.21.15.7:0/311748648 shutdown_connections 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 -- 172.21.15.7:0/311748648 wait complete. 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 -- start start 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.682+0000 7fae733de740 1 Processor -- start 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477ccbe50 0x55a477cd7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a477cceda0 con 0x55a477cc62f0 2024-03-19T00:39:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a477be6310 con 0x55a477ccbe50 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a477ccef20 con 0x55a477cc6ce0 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 10 client.0 _ll_register_callbacks cb 0x55a477bda2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39264/0 (socket says 172.21.15.7:39264) 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:58300/0 (socket says 172.21.15.7:58300) 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6dcab640 1 -- 172.21.15.7:0/2324165873 learned_addr learned my addr 172.21.15.7:0/2324165873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6dcab640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477ccbe50 0x55a477cd7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 -- 172.21.15.7:0/2324165873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477ccbe50 msgr2=0x55a477cd7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477ccbe50 0x55a477cd7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 -- 172.21.15.7:0/2324165873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 msgr2=0x55a477cd07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 -- 172.21.15.7:0/2324165873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47793e3f0 con 0x55a477cc6ce0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6dcab640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477ccbe50 0x55a477cd7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd0d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55a477cc27d0 tx=0x7fae64004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657cac9a225e62dc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae51ffb640 1 -- 172.21.15.7:0/2324165873 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640035f0 con 0x55a477cc6ce0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae64004040 con 0x55a477cc6ce0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.683+0000 7fae51ffb640 1 -- 172.21.15.7:0/2324165873 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae64003790 con 0x55a477cc6ce0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae51ffb640 1 -- 172.21.15.7:0/2324165873 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae6405b050 con 0x55a477cc6ce0 2024-03-19T00:39:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 10 client.25561 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a477a087d0 con 0x55a477cc6ce0 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae51ffb640 1 -- 172.21.15.7:0/2324165873 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae64004040 con 0x55a477cc6ce0 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae51ffb640 1 -- 172.21.15.7:0/2324165873 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fae64004bd0 con 0x55a477cc6ce0 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 10 client.25561 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 10 client.25561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 20 client.25561 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477cdc230 0x55a477cde620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 4 client.25561 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a477c883e0 con 0x55a477cdc230 2024-03-19T00:39:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.684+0000 7fae6dcab640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477cdc230 0x55a477cde620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae51ffb640 10 client.25561 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae6dcab640 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477cdc230 0x55a477cde620 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4768d475495c40 server_cookie=550a3990d06b34be in_seq=0 out_seq=0 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae51ffb640 1 -- 172.21.15.7:0/2324165873 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a477c883e0 con 0x55a477cdc230 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae51ffb640 10 client.25561 handle_command_reply: tid=0 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 client.25561 shutdown 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 msgr2=0x55a477cd0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd0d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55a477cc27d0 tx=0x7fae64004420 comp rx=0 tx=0).stop 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 shutdown_connections 2024-03-19T00:39:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477cdc230 0x55a477cde620 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477ccbe50 0x55a477cd7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd0d00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae51ffb640 0 client.25561 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 --2- 172.21.15.7:0/2324165873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 >> 172.21.15.7:0/2324165873 conn(0x55a477cc1860 msgr2=0x55a477cc0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 shutdown_connections 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.685+0000 7fae733de740 1 -- 172.21.15.7:0/2324165873 wait complete. 2024-03-19T00:39:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.686+0000 7fae733de740 20 client.25561 trim_cache size 0 max 16384 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.698+0000 7fae733de740 1 Processor -- start 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 -- start start 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc62f0 0x55a477cbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477ccbe50 0x55a477cc0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae681aebc0 con 0x55a477ccbe50 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae681ac490 con 0x55a477cc62f0 2024-03-19T00:39:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae733de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae681aa2e0 con 0x55a477cc6ce0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477ccbe50 0x55a477cc0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:58302/0 (socket says 172.21.15.7:58302) 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 -- 172.21.15.7:0/4276084762 learned_addr learned my addr 172.21.15.7:0/4276084762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6dcab640 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc62f0 0x55a477cbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 -- 172.21.15.7:0/4276084762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc62f0 msgr2=0x55a477cbca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc62f0 0x55a477cbca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 -- 172.21.15.7:0/4276084762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477ccbe50 msgr2=0x55a477cc0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477ccbe50 0x55a477cc0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 -- 172.21.15.7:0/4276084762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a477c97880 con 0x55a477cc6ce0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6dcab640 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc62f0 0x55a477cbca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd7d00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a8a0b6f8fd764f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae53fff640 1 -- 172.21.15.7:0/4276084762 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64045020 con 0x55a477cc6ce0 2024-03-19T00:39:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae53fff640 1 -- 172.21.15.7:0/4276084762 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae64004490 con 0x55a477cc6ce0 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.699+0000 7fae53fff640 1 -- 172.21.15.7:0/4276084762 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64004920 con 0x55a477cc6ce0 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- 172.21.15.7:0/4276084762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 msgr2=0x55a477cd7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd7d00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64009520 comp rx=0 tx=0).stop 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- 172.21.15.7:0/4276084762 shutdown_connections 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477ccbe50 0x55a477cc0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477cc6ce0 0x55a477cd7d00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- 172.21.15.7:0/4276084762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc62f0 0x55a477cbca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- 172.21.15.7:0/4276084762 >> 172.21.15.7:0/4276084762 conn(0x55a477cc1860 msgr2=0x55a477cc1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- 172.21.15.7:0/4276084762 shutdown_connections 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- 172.21.15.7:0/4276084762 wait complete. 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- start start 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 Processor -- start 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477cd5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cd6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae681aa950 con 0x55a477cc62f0 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae681abdc0 con 0x55a477cc6ce0 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae681ab020 con 0x55a477ccbe50 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.700+0000 7fae733de740 10 client.0 _ll_register_callbacks cb 0x7fae58009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6dcab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477cd5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477cd5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:37960/0 (socket says 172.21.15.7:37960) 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 -- 172.21.15.7:0/1850363097 learned_addr learned my addr 172.21.15.7:0/1850363097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6dcab640 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cd6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 -- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 msgr2=0x55a477cd6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cd6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 -- 172.21.15.7:0/1850363097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 msgr2=0x55a477cd5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 -- 172.21.15.7:0/1850363097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a477cd6f80 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae6e4ac640 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477cd5980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae64004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874278919ceba4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae51ffb640 1 -- 172.21.15.7:0/1850363097 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64045040 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae64004490 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae51ffb640 1 -- 172.21.15.7:0/1850363097 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae64041690 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae51ffb640 1 -- 172.21.15.7:0/1850363097 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fae6406e050 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae733de740 10 client.25655 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.701+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a477d424f0 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae51ffb640 1 -- 172.21.15.7:0/1850363097 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae64004490 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae51ffb640 1 -- 172.21.15.7:0/1850363097 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fae6405e070 con 0x55a477cc6ce0 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 10 client.25655 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 10 client.25655 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 20 client.25655 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477d42db0 0x55a477d451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 4 client.25655 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a477c379f0 con 0x55a477d42db0 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae6dcab640 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477d42db0 0x55a477d451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae51ffb640 10 client.25655 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae6dcab640 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477d42db0 0x55a477d451d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5cebf0ff98bc604 server_cookie=7ccac80dd6fec64c in_seq=0 out_seq=0 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae51ffb640 1 -- 172.21.15.7:0/1850363097 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a477c379f0 con 0x55a477d42db0 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae51ffb640 10 client.25655 handle_command_reply: tid=0 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 client.25655 shutdown 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 msgr2=0x55a477cd5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477cd5980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae64004e50 comp rx=0 tx=0).stop 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 shutdown_connections 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.702+0000 7fae733de740 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a477d42db0 0x55a477d451d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a477ccbe50 0x55a477cd6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a477cc6ce0 0x55a477cd5980 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae51ffb640 0 client.25655 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:39:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 1 --2- 172.21.15.7:0/1850363097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a477cc62f0 0x55a477cd5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 >> 172.21.15.7:0/1850363097 conn(0x55a477cc1860 msgr2=0x55a477cc4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 shutdown_connections 2024-03-19T00:39:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 1 -- 172.21.15.7:0/1850363097 wait complete. 2024-03-19T00:39:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae733de740 20 client.25655 trim_cache size 0 max 16384 2024-03-19T00:39:34.713 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae540a2880 msgr2=0x7fae540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae540a2880 0x7fae540a4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae600a3b40 tx=0x7fae60090db0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 msgr2=0x7fae68109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 0x7fae68109c20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae6000eb10 tx=0x7fae6000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 shutdown_connections 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae540a2880 0x7fae540a4d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae681022d0 0x7fae68109c20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae68101910 0x7fae681096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 --2- 172.21.15.7:0/2411034649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae68100720 0x7fae6810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 >> 172.21.15.7:0/2411034649 conn(0x7fae680fbc50 msgr2=0x7fae68104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 shutdown_connections 2024-03-19T00:39:34.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.703+0000 7fae70710640 1 -- 172.21.15.7:0/2411034649 wait complete. 2024-03-19T00:39:34.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:39:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 -- 172.21.15.7:0/2449255472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098101920 msgr2=0x7f5098101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 --2- 172.21.15.7:0/2449255472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098101920 0x7f5098101da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5080009900 tx=0x7f508002bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 -- 172.21.15.7:0/2449255472 shutdown_connections 2024-03-19T00:39:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 --2- 172.21.15.7:0/2449255472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f5098108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 --2- 172.21.15.7:0/2449255472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098101920 0x7f5098101da0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 --2- 172.21.15.7:0/2449255472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5098100730 0x7f5098100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 -- 172.21.15.7:0/2449255472 >> 172.21.15.7:0/2449255472 conn(0x7f50980fbc80 msgr2=0x7f50980fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 -- 172.21.15.7:0/2449255472 shutdown_connections 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 -- 172.21.15.7:0/2449255472 wait complete. 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 Processor -- start 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.820+0000 7f509cf09640 1 -- start start 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098100730 0x7f5098197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5098101920 0x7f5098197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f509819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f509810c240 con 0x7f50981022e0 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f509810c0c0 con 0x7f5098100730 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f509810c3c0 con 0x7f5098101920 2024-03-19T00:39:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f509819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f509819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39304/0 (socket says 172.21.15.7:39304) 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 -- 172.21.15.7:0/3799480387 learned_addr learned my addr 172.21.15.7:0/3799480387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509659c640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098100730 0x7f5098197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f508ffff640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5098101920 0x7f5098197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 -- 172.21.15.7:0/3799480387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5098101920 msgr2=0x7f5098197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5098101920 0x7f5098197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 -- 172.21.15.7:0/3799480387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098100730 msgr2=0x7f5098197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098100730 0x7f5098197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 -- 172.21.15.7:0/3799480387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509819f6b0 con 0x7f50981022e0 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509659c640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098100730 0x7f5098197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f5096d9d640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f509819f040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f509000eb10 tx=0x7f509000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f508f7fe640 1 -- 172.21.15.7:0/3799480387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090004b20 con 0x7f50981022e0 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509819f9a0 con 0x7f50981022e0 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f508f7fe640 1 -- 172.21.15.7:0/3799480387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5090004cc0 con 0x7f50981022e0 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.821+0000 7f508f7fe640 1 -- 172.21.15.7:0/3799480387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090010690 con 0x7f50981022e0 2024-03-19T00:39:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.822+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509819fec0 con 0x7f50981022e0 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.824+0000 7f508f7fe640 1 -- 172.21.15.7:0/3799480387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5090005020 con 0x7f50981022e0 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.824+0000 7f509fbd3740 1 Processor -- start 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 -- start start 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92da6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92dabbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92dae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55da92d064b0 con 0x55da92dac260 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da92caed70 con 0x55da92da6700 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509fbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da92d02550 con 0x55da92da70f0 2024-03-19T00:39:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92dae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92dabbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92dae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39314/0 (socket says 172.21.15.7:39314) 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92dabbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:58336/0 (socket says 172.21.15.7:58336) 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 -- 172.21.15.7:0/3814551722 learned_addr learned my addr 172.21.15.7:0/3814551722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92da6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508f7fe640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50840a2800 0x7f50840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f5096d9d640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50840a2800 0x7f50840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 -- 172.21.15.7:0/3814551722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 msgr2=0x55da92dabbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92dabbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 -- 172.21.15.7:0/3814551722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 msgr2=0x55da92da6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92da6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 -- 172.21.15.7:0/3814551722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da92cd1210 con 0x55da92dac260 2024-03-19T00:39:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f509659c640 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92dabbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508f7fe640 1 -- 172.21.15.7:0/3799480387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50900cf350 con 0x7f50981022e0 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.825+0000 7f508ffff640 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92dae6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f50800096c0 tx=0x7f508002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2b21fa02de2021 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f5096d9d640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50840a2800 0x7f50840a4cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f509009b690 tx=0x7f509000efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f508cff9640 1 -- 172.21.15.7:0/3814551722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080004040 con 0x55da92dac260 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f508cff9640 1 -- 172.21.15.7:0/3814551722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50800041e0 con 0x55da92dac260 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f508cff9640 1 -- 172.21.15.7:0/3814551722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080043450 con 0x55da92dac260 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 -- 172.21.15.7:0/3814551722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 msgr2=0x55da92dae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92dae6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f50800096c0 tx=0x7f508002bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 -- 172.21.15.7:0/3814551722 shutdown_connections 2024-03-19T00:39:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92dae6f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92dabbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3814551722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92da6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.826+0000 7f509fbd3740 1 -- 172.21.15.7:0/3814551722 >> 172.21.15.7:0/3814551722 conn(0x55da92d9d860 msgr2=0x55da92d9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 -- 172.21.15.7:0/3814551722 shutdown_connections 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 -- 172.21.15.7:0/3814551722 wait complete. 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 -- start start 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 Processor -- start 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92db0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da70f0 0x55da92db1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92dac260 0x55da92db8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55da92daf1b0 con 0x55da92da70f0 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da92cc2310 con 0x55da92da6700 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da92daf330 con 0x55da92dac260 2024-03-19T00:39:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509fbd3740 10 client.0 _ll_register_callbacks cb 0x55da92cb62d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da70f0 0x55da92db1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f508ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92db0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f508ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92db0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:37992/0 (socket says 172.21.15.7:37992) 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f509659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da70f0 0x55da92db1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39318/0 (socket says 172.21.15.7:39318) 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f508ffff640 1 -- 172.21.15.7:0/3955114403 learned_addr learned my addr 172.21.15.7:0/3955114403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.827+0000 7f508ffff640 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92dac260 0x55da92db8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f508ffff640 1 -- 172.21.15.7:0/3955114403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92dac260 msgr2=0x55da92db8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f508ffff640 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92dac260 0x55da92db8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f508ffff640 1 -- 172.21.15.7:0/3955114403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da70f0 msgr2=0x55da92db1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f508ffff640 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da70f0 0x55da92db1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f508ffff640 1 -- 172.21.15.7:0/3955114403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da92a1a3f0 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f508ffff640 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92db0c10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5080001eb0 tx=0x7f5080046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3dcad7fca9dd2b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f507dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f507dffb640 1 -- 172.21.15.7:0/3955114403 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080005070 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50800041e0 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f507dffb640 1 -- 172.21.15.7:0/3955114403 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5080005210 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f507dffb640 1 -- 172.21.15.7:0/3955114403 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5080005410 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f509fbd3740 10 client.25667 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da92ae47d0 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f507dffb640 1 -- 172.21.15.7:0/3955114403 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50800041e0 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.828+0000 7f507dffb640 1 -- 172.21.15.7:0/3955114403 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f508005a070 con 0x55da92da6700 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509fbd3740 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509fbd3740 10 client.25667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:39:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509fbd3740 20 client.25667 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92dbc680 0x55da92dbea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509fbd3740 4 client.25667 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da92d643e0 con 0x55da92dbc680 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509659c640 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92dbc680 0x55da92dbea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f509659c640 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92dbc680 0x55da92dbea70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fd6370e721a10a7 server_cookie=3e0349bf1bb112c4 in_seq=0 out_seq=0 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.829+0000 7f507dffb640 10 client.25667 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f507dffb640 1 -- 172.21.15.7:0/3955114403 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55da92d643e0 con 0x55da92dbc680 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f507dffb640 10 client.25667 handle_command_reply: tid=0 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 client.25667 shutdown 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 msgr2=0x55da92db0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92db0c10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5080001eb0 tx=0x7f5080046040 comp rx=0 tx=0).stop 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 shutdown_connections 2024-03-19T00:39:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92dbc680 0x55da92dbea70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92dac260 0x55da92db8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da70f0 0x55da92db1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 --2- 172.21.15.7:0/3955114403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da6700 0x55da92db0c10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f507dffb640 0 client.25667 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 >> 172.21.15.7:0/3955114403 conn(0x55da92d9d860 msgr2=0x55da92d9cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 shutdown_connections 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 1 -- 172.21.15.7:0/3955114403 wait complete. 2024-03-19T00:39:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.830+0000 7f509fbd3740 20 client.25667 trim_cache size 0 max 16384 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 Processor -- start 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 -- start start 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da6700 0x55da92dafee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da70f0 0x55da92db8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92db0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50981aa880 con 0x55da92dac260 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50981a8150 con 0x55da92da70f0 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509fbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50981a5fa0 con 0x55da92da6700 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f508ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92db0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f508ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92db0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39330/0 (socket says 172.21.15.7:39330) 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f508ffff640 1 -- 172.21.15.7:0/1875296614 learned_addr learned my addr 172.21.15.7:0/1875296614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f509659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da70f0 0x55da92db8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.843+0000 7f508ffff640 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da6700 0x55da92dafee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f508ffff640 1 -- 172.21.15.7:0/1875296614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da6700 msgr2=0x55da92dafee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f508ffff640 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da6700 0x55da92dafee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f508ffff640 1 -- 172.21.15.7:0/1875296614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da70f0 msgr2=0x55da92db8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f508ffff640 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da70f0 0x55da92db8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f508ffff640 1 -- 172.21.15.7:0/1875296614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da92d73880 con 0x55da92dac260 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509659c640 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da70f0 0x55da92db8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f508ffff640 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92db0420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f508002cb60 tx=0x7f50800055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae831a3908537fac server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f507ffff640 1 -- 172.21.15.7:0/1875296614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508002c8e0 con 0x55da92dac260 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f507ffff640 1 -- 172.21.15.7:0/1875296614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5080043450 con 0x55da92dac260 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f507ffff640 1 -- 172.21.15.7:0/1875296614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080004920 con 0x55da92dac260 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 -- 172.21.15.7:0/1875296614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 msgr2=0x55da92db0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92db0420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f508002cb60 tx=0x7f50800055b0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 -- 172.21.15.7:0/1875296614 shutdown_connections 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92dac260 0x55da92db0420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92da70f0 0x55da92db8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1875296614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da6700 0x55da92dafee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 -- 172.21.15.7:0/1875296614 >> 172.21.15.7:0/1875296614 conn(0x55da92d9d860 msgr2=0x55da92da1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.844+0000 7f509fbd3740 1 -- 172.21.15.7:0/1875296614 shutdown_connections 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 -- 172.21.15.7:0/1875296614 wait complete. 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 -- start start 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 Processor -- start 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 0x55da92db5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92db5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92dac260 0x55da92db6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50981a6610 con 0x55da92da6700 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50981a7a80 con 0x55da92dac260 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50981a6ce0 con 0x55da92da70f0 2024-03-19T00:39:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509fbd3740 10 client.0 _ll_register_callbacks cb 0x7f5080009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 0x55da92db5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 0x55da92db5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39332/0 (socket says 172.21.15.7:39332) 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 -- 172.21.15.7:0/1373969495 learned_addr learned my addr 172.21.15.7:0/1373969495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f509659c640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92db5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92dac260 0x55da92db6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 -- 172.21.15.7:0/1373969495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 msgr2=0x55da92db5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92db5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 -- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92dac260 msgr2=0x55da92db6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92dac260 0x55da92db6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.845+0000 7f508ffff640 1 -- 172.21.15.7:0/1373969495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da92db7280 con 0x55da92da6700 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f508ffff640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 0x55da92db5740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5080005ab0 tx=0x7f5080005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6877cbb78385e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f507dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f507dffb640 1 -- 172.21.15.7:0/1373969495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080003a10 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f507dffb640 1 -- 172.21.15.7:0/1373969495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5080050430 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5080043450 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f507dffb640 1 -- 172.21.15.7:0/1373969495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5080050630 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 10 client.15822 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da92e228a0 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f507dffb640 1 -- 172.21.15.7:0/1373969495 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5080043450 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f507dffb640 1 -- 172.21.15.7:0/1373969495 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f508005a070 con 0x55da92da6700 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 10 client.15822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 20 client.15822 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92e22e60 0x55da92e25280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 4 client.15822 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:39:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da92d139f0 con 0x55da92e22e60 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.846+0000 7f509659c640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92e22e60 0x55da92e25280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f507dffb640 10 client.15822 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509659c640 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92e22e60 0x55da92e25280 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f3c894dc1c4654e server_cookie=5dcdcdeb6dde6f64 in_seq=0 out_seq=0 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f507dffb640 1 -- 172.21.15.7:0/1373969495 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da92d139f0 con 0x55da92e22e60 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f507dffb640 10 client.15822 handle_command_reply: tid=0 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 client.15822 shutdown 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 msgr2=0x55da92db5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 0x55da92db5740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5080005ab0 tx=0x7f5080005fd0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 shutdown_connections 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55da92e22e60 0x55da92e25280 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da92dac260 0x55da92db6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da92da70f0 0x55da92db5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 --2- 172.21.15.7:0/1373969495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da92da6700 0x55da92db5740 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f507dffb640 0 client.15822 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 >> 172.21.15.7:0/1373969495 conn(0x55da92d9d860 msgr2=0x55da92d9c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 shutdown_connections 2024-03-19T00:39:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 1 -- 172.21.15.7:0/1373969495 wait complete. 2024-03-19T00:39:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.847+0000 7f509fbd3740 20 client.15822 trim_cache size 0 max 16384 2024-03-19T00:39:34.858 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:39:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50840a2800 msgr2=0x7f50840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50840a2800 0x7f50840a4cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f509009b690 tx=0x7f509000efd0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 msgr2=0x7f509819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f509819f040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f509000eb10 tx=0x7f509000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 shutdown_connections 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50840a2800 0x7f50840a4cc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50981022e0 0x7f509819f040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5098101920 0x7f5098197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 --2- 172.21.15.7:0/3799480387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5098100730 0x7f5098197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 >> 172.21.15.7:0/3799480387 conn(0x7f50980fbc80 msgr2=0x7f5098069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 shutdown_connections 2024-03-19T00:39:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.848+0000 7f509cf09640 1 -- 172.21.15.7:0/3799480387 wait complete. 2024-03-19T00:39:34.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:39:34.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 -- 172.21.15.7:0/650096433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc1033e0 msgr2=0x7f46fc109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 --2- 172.21.15.7:0/650096433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc1033e0 0x7f46fc109c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009900 tx=0x7f46ec02bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 -- 172.21.15.7:0/650096433 shutdown_connections 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 --2- 172.21.15.7:0/650096433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc1033e0 0x7f46fc109c70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 --2- 172.21.15.7:0/650096433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46fc102a20 0x7f46fc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 --2- 172.21.15.7:0/650096433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101830 0x7f46fc101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.964+0000 7f4702d87640 1 -- 172.21.15.7:0/650096433 >> 172.21.15.7:0/650096433 conn(0x7f46fc0fcd60 msgr2=0x7f46fc0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 -- 172.21.15.7:0/650096433 shutdown_connections 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 -- 172.21.15.7:0/650096433 wait complete. 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 Processor -- start 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 -- start start 2024-03-19T00:39:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46fc101830 0x7f46fc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc102a20 0x7f46fc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 0x7f46fc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10d340 con 0x7f46fc1033e0 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10d1c0 con 0x7f46fc102a20 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4702d87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10d4c0 con 0x7f46fc101830 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4701324640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 0x7f46fc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4701324640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 0x7f46fc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39348/0 (socket says 172.21.15.7:39348) 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4701324640 1 -- 172.21.15.7:0/2440255091 learned_addr learned my addr 172.21.15.7:0/2440255091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f46fbfff640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc102a20 0x7f46fc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.965+0000 7f4700b23640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46fc101830 0x7f46fc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4701324640 1 -- 172.21.15.7:0/2440255091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46fc101830 msgr2=0x7f46fc197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4701324640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46fc101830 0x7f46fc197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4701324640 1 -- 172.21.15.7:0/2440255091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc102a20 msgr2=0x7f46fc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4701324640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc102a20 0x7f46fc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4701324640 1 -- 172.21.15.7:0/2440255091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc19f5b0 con 0x7f46fc1033e0 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4701324640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 0x7f46fc19ef40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009230 tx=0x7f46ec005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f46f9ffb640 1 -- 172.21.15.7:0/2440255091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec043020 con 0x7f46fc1033e0 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f46f9ffb640 1 -- 172.21.15.7:0/2440255091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46ec0043d0 con 0x7f46fc1033e0 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc19f840 con 0x7f46fc1033e0 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f46f9ffb640 1 -- 172.21.15.7:0/2440255091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec003b40 con 0x7f46fc1033e0 2024-03-19T00:39:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.966+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc19fd90 con 0x7f46fc1033e0 2024-03-19T00:39:34.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.968+0000 7f46f9ffb640 1 -- 172.21.15.7:0/2440255091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f46ec046020 con 0x7f46fc1033e0 2024-03-19T00:39:34.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 Processor -- start 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 -- start start 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x56523520db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235212bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x5652352156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56523516d4b0 con 0x56523520e0f0 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565235115d70 con 0x56523520d700 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4705972740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565235169550 con 0x565235213260 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46f9ffb640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f46e40a2800 0x7f46e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4700b23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235212bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x56523520db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4700b23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235212bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39352/0 (socket says 172.21.15.7:39352) 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4700b23640 1 -- 172.21.15.7:0/3582297352 learned_addr learned my addr 172.21.15.7:0/3582297352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x5652352156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f4701324640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f46e40a2800 0x7f46e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46f9ffb640 1 -- 172.21.15.7:0/2440255091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f46ec0f0fa0 con 0x7f46fc1033e0 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 -- 172.21.15.7:0/3582297352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 msgr2=0x5652352156f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x5652352156f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 -- 172.21.15.7:0/3582297352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 msgr2=0x565235212bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235212bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.969+0000 7f46fbfff640 1 -- 172.21.15.7:0/3582297352 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565235138210 con 0x56523520d700 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f46fbfff640 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x56523520db20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009730 tx=0x7f46f00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce545171ca104516 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f4701324640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f46e40a2800 0x7f46e40a4cc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0b99e0 tx=0x7f46ec04a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f46e2ffd640 1 -- 172.21.15.7:0/3582297352 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004050 con 0x56523520d700 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f46e2ffd640 1 -- 172.21.15.7:0/3582297352 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46f00054d0 con 0x56523520d700 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f46e2ffd640 1 -- 172.21.15.7:0/3582297352 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004b50 con 0x56523520d700 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f4705972740 1 -- 172.21.15.7:0/3582297352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 msgr2=0x56523520db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.970+0000 7f4705972740 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x56523520db20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009730 tx=0x7f46f00096c0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- 172.21.15.7:0/3582297352 shutdown_connections 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x5652352156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235212bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 --2- 172.21.15.7:0/3582297352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x56523520db20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- 172.21.15.7:0/3582297352 >> 172.21.15.7:0/3582297352 conn(0x565235204860 msgr2=0x565235204c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- 172.21.15.7:0/3582297352 shutdown_connections 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- 172.21.15.7:0/3582297352 wait complete. 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- start start 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 Processor -- start 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x565235217c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235218180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565235216210 con 0x56523520e0f0 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565235129310 con 0x56523520d700 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565235216390 con 0x565235213260 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.971+0000 7f4705972740 10 client.0 _ll_register_callbacks cb 0x56523511d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235218180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x565235217c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235218180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39362/0 (socket says 172.21.15.7:39362) 2024-03-19T00:39:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 -- 172.21.15.7:0/1217917654 learned_addr learned my addr 172.21.15.7:0/1217917654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 -- 172.21.15.7:0/1217917654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 msgr2=0x56523521f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 -- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 msgr2=0x565235217c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x565235217c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 -- 172.21.15.7:0/1217917654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565234e813f0 con 0x56523520e0f0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x565235217c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4700b23640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235218180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46f4011d10 tx=0x7f46f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d6ef2f9688d87f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46dbfff640 1 -- 172.21.15.7:0/1217917654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4003970 con 0x56523520e0f0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46dbfff640 1 -- 172.21.15.7:0/1217917654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f46f4003b10 con 0x56523520e0f0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f00054d0 con 0x56523520e0f0 2024-03-19T00:39:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f46dbfff640 1 -- 172.21.15.7:0/1217917654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46f4003d50 con 0x56523520e0f0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4705972740 10 client.15834 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.972+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565234f4b7d0 con 0x56523520e0f0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f46dbfff640 1 -- 172.21.15.7:0/1217917654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46f00054d0 con 0x56523520e0f0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f46dbfff640 1 -- 172.21.15.7:0/1217917654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f46f401b620 con 0x56523520e0f0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f4705972740 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f4705972740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f4705972740 20 client.15834 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f4705972740 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565235223bb0 0x565235226030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f4705972740 4 client.15834 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652351cb3e0 con 0x565235223bb0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565235223bb0 0x565235226030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f46dbfff640 10 client.15834 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.973+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565235223bb0 0x565235226030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d85c5b1dcd69192 server_cookie=2f01561ec575757a in_seq=0 out_seq=0 2024-03-19T00:39:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f46dbfff640 1 -- 172.21.15.7:0/1217917654 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5652351cb3e0 con 0x565235223bb0 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f46dbfff640 10 client.15834 handle_command_reply: tid=0 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 client.15834 shutdown 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 msgr2=0x565235218180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235218180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46f4011d10 tx=0x7f46f400f790 comp rx=0 tx=0).stop 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 shutdown_connections 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565235223bb0 0x565235226030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520e0f0 0x565235218180 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 --2- 172.21.15.7:0/1217917654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520d700 0x565235217c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f46dbfff640 0 client.15834 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:34.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.974+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 >> 172.21.15.7:0/1217917654 conn(0x565235204860 msgr2=0x565235203ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.975+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 shutdown_connections 2024-03-19T00:39:34.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.975+0000 7f4705972740 1 -- 172.21.15.7:0/1217917654 wait complete. 2024-03-19T00:39:34.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.975+0000 7f4705972740 20 client.15834 trim_cache size 0 max 16384 2024-03-19T00:39:34.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 Processor -- start 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 -- start start 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523520d700 0x56523521fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 0x565235220150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1aa890 con 0x565235213260 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a8160 con 0x56523520e0f0 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4705972740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a5fb0 con 0x56523520d700 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 0x565235220150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 0x565235220150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39372/0 (socket says 172.21.15.7:39372) 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 -- 172.21.15.7:0/1817798425 learned_addr learned my addr 172.21.15.7:0/1817798425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f4700b23640 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523520d700 0x56523521fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 -- 172.21.15.7:0/1817798425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523520d700 msgr2=0x56523521fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523520d700 0x56523521fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 -- 172.21.15.7:0/1817798425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 msgr2=0x56523521f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.988+0000 7f46fbfff640 1 -- 172.21.15.7:0/1817798425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565235216510 con 0x565235213260 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f46fbfff640 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 0x565235220150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f46f0005fd0 tx=0x7f46f0009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd3f380c8e227d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f46e27fc640 1 -- 172.21.15.7:0/1817798425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004680 con 0x565235213260 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f46e27fc640 1 -- 172.21.15.7:0/1817798425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46f00313d0 con 0x565235213260 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f46e27fc640 1 -- 172.21.15.7:0/1817798425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0005410 con 0x565235213260 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 -- 172.21.15.7:0/1817798425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 msgr2=0x565235220150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 0x565235220150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f46f0005fd0 tx=0x7f46f0009ac0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 -- 172.21.15.7:0/1817798425 shutdown_connections 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565235213260 0x565235220150 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 --2- 172.21.15.7:0/1817798425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523520d700 0x56523521fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 -- 172.21.15.7:0/1817798425 >> 172.21.15.7:0/1817798425 conn(0x565235204860 msgr2=0x56523520f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.989+0000 7f4705972740 1 -- 172.21.15.7:0/1817798425 shutdown_connections 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 -- 172.21.15.7:0/1817798425 wait complete. 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 -- start start 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 Processor -- start 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 0x56523521c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a6620 con 0x56523520d700 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a7a90 con 0x56523520e0f0 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a6cf0 con 0x565235213260 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4705972740 10 client.0 _ll_register_callbacks cb 0x7f46f4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f46fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 0x56523521c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4700b23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f46fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 0x56523521c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39386/0 (socket says 172.21.15.7:39386) 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f4700b23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38072/0 (socket says 172.21.15.7:38072) 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.990+0000 7f46fbfff640 1 -- 172.21.15.7:0/207314420 learned_addr learned my addr 172.21.15.7:0/207314420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46fbfff640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4700b23640 1 -- 172.21.15.7:0/207314420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 msgr2=0x56523521d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4700b23640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4700b23640 1 -- 172.21.15.7:0/207314420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 msgr2=0x56523521c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4700b23640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 0x56523521c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4700b23640 1 -- 172.21.15.7:0/207314420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523521dfb0 con 0x56523520e0f0 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46fbfff640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 0x56523521c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4700b23640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521c9b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006010 tx=0x7f46f401cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3b11ff19494ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46dbfff640 1 -- 172.21.15.7:0/207314420 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f401b630 con 0x56523520e0f0 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f00313d0 con 0x56523520e0f0 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46dbfff640 1 -- 172.21.15.7:0/207314420 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f46f401b7d0 con 0x56523520e0f0 2024-03-19T00:39:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46dbfff640 1 -- 172.21.15.7:0/207314420 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46f4037050 con 0x56523520e0f0 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4705972740 10 client.25691 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56523521f320 con 0x56523520e0f0 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.991+0000 7f46dbfff640 1 -- 172.21.15.7:0/207314420 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46f00313d0 con 0x56523520e0f0 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46dbfff640 1 -- 172.21.15.7:0/207314420 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f46f4032440 con 0x56523520e0f0 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 10 client.25691 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 20 client.25691 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56523528a570 0x56523528c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 4 client.25691 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56523517a9f0 con 0x56523528a570 2024-03-19T00:39:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46fbfff640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56523528a570 0x56523528c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46dbfff640 10 client.25691 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46fbfff640 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56523528a570 0x56523528c990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829349a01d4279bb server_cookie=7263f51737519ce0 in_seq=0 out_seq=0 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46dbfff640 1 -- 172.21.15.7:0/207314420 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56523517a9f0 con 0x56523528a570 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46dbfff640 10 client.25691 handle_command_reply: tid=0 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 client.25691 shutdown 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 msgr2=0x56523521c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521c9b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006010 tx=0x7f46f401cf90 comp rx=0 tx=0).stop 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 shutdown_connections 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565235213260 0x56523521d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56523520e0f0 0x56523521c9b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56523528a570 0x56523528c990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 --2- 172.21.15.7:0/207314420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56523520d700 0x56523521c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f46dbfff640 0 client.25691 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 >> 172.21.15.7:0/207314420 conn(0x565235204860 msgr2=0x56523521fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.992+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 shutdown_connections 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4705972740 1 -- 172.21.15.7:0/207314420 wait complete. 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4705972740 20 client.25691 trim_cache size 0 max 16384 2024-03-19T00:39:35.003 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f46e40a2800 msgr2=0x7f46e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f46e40a2800 0x7f46e40a4cc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0b99e0 tx=0x7f46ec04a040 comp rx=0 tx=0).stop 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 msgr2=0x7f46fc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 0x7f46fc19ef40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009230 tx=0x7f46ec005e40 comp rx=0 tx=0).stop 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 shutdown_connections 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f46e40a2800 0x7f46e40a4cc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1033e0 0x7f46fc19ef40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46fc102a20 0x7f46fc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 --2- 172.21.15.7:0/2440255091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46fc101830 0x7f46fc197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 >> 172.21.15.7:0/2440255091 conn(0x7f46fc0fcd60 msgr2=0x7f46fc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 shutdown_connections 2024-03-19T00:39:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:34.993+0000 7f4702d87640 1 -- 172.21.15.7:0/2440255091 wait complete. 2024-03-19T00:39:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:35 smithi007 ceph-mon[62854]: pgmap v400: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-03-19T00:39:35.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:35 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2942696699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:39:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:35 smithi204 ceph-mon[66572]: pgmap v400: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-03-19T00:39:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:35 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2942696699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:39:35.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:35 smithi044 ceph-mon[66366]: pgmap v400: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-03-19T00:39:35.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:35 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2942696699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:39:36.007 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:39:36.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:39:36.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:39:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 -- 172.21.15.7:0/1486698708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 msgr2=0x7f9d20100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 --2- 172.21.15.7:0/1486698708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d20100eb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009900 tx=0x7f9d1002bc80 comp rx=0 tx=0).stop 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 -- 172.21.15.7:0/1486698708 shutdown_connections 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 --2- 172.21.15.7:0/1486698708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201013f0 0x7f9d20101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 --2- 172.21.15.7:0/1486698708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d20100eb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 --2- 172.21.15.7:0/1486698708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d201000f0 0x7f9d20100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 -- 172.21.15.7:0/1486698708 >> 172.21.15.7:0/1486698708 conn(0x7f9d200fbcb0 msgr2=0x7f9d200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 -- 172.21.15.7:0/1486698708 shutdown_connections 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 -- 172.21.15.7:0/1486698708 wait complete. 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 Processor -- start 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.126+0000 7f9d284a1640 1 -- start start 2024-03-19T00:39:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 0x7f9d20199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d2019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d201013f0 0x7f9d201a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010f5b0 con 0x7f9d20100a50 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010f430 con 0x7f9d201000f0 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010f730 con 0x7f9d201013f0 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d25a3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d2019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 0x7f9d20199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d25a3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d2019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39406/0 (socket says 172.21.15.7:39406) 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 0x7f9d20199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38096/0 (socket says 172.21.15.7:38096) 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d25a3c640 1 -- 172.21.15.7:0/3622386406 learned_addr learned my addr 172.21.15.7:0/3622386406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d26a3e640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d201013f0 0x7f9d201a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 -- 172.21.15.7:0/3622386406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d201013f0 msgr2=0x7f9d201a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d201013f0 0x7f9d201a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 -- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 msgr2=0x7f9d2019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d2019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 -- 172.21.15.7:0/3622386406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d201a1990 con 0x7f9d201000f0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d2019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d2623d640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 0x7f9d20199d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800e9f0 tx=0x7f9d1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d177fe640 1 -- 172.21.15.7:0/3622386406 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800ccc0 con 0x7f9d201000f0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d201a1c80 con 0x7f9d201000f0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.127+0000 7f9d177fe640 1 -- 172.21.15.7:0/3622386406 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d18005020 con 0x7f9d201000f0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.128+0000 7f9d177fe640 1 -- 172.21.15.7:0/3622386406 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18004a00 con 0x7f9d201000f0 2024-03-19T00:39:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.128+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d201a21d0 con 0x7f9d201000f0 2024-03-19T00:39:36.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.129+0000 7f9d177fe640 1 -- 172.21.15.7:0/3622386406 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f9d18004ba0 con 0x7f9d201000f0 2024-03-19T00:39:36.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.130+0000 7f9d2b16d740 1 Processor -- start 2024-03-19T00:39:36.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.130+0000 7f9d2b16d740 1 -- start start 2024-03-19T00:39:36.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.130+0000 7f9d177fe640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d0c0a2800 0x7f9d0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e7f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e7fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e800390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56201e75d160 con 0x56201e7f83a0 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56201e705e10 con 0x56201e7fdf00 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56201e759200 con 0x56201e7f8d90 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d26a3e640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d0c0a2800 0x7f9d0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d177fe640 1 -- 172.21.15.7:0/3622386406 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d180ced70 con 0x7f9d201000f0 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e7f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e7fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e7f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39414/0 (socket says 172.21.15.7:39414) 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d26a3e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e800390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2623d640 1 -- 172.21.15.7:0/2942551485 learned_addr learned my addr 172.21.15.7:0/2942551485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 -- 172.21.15.7:0/2942551485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 msgr2=0x56201e800390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e800390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 -- 172.21.15.7:0/2942551485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 msgr2=0x56201e7f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e7f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 -- 172.21.15.7:0/2942551485 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201e5397e0 con 0x56201e7f8d90 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d2623d640 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e7f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d26a3e640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d0c0a2800 0x7f9d0c0a4cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c009d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.131+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e7fd890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006010 tx=0x7f9d10009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f23607e9ac6db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d14bfe640 1 -- 172.21.15.7:0/2942551485 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10045020 con 0x56201e7f8d90 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d14bfe640 1 -- 172.21.15.7:0/2942551485 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d10005040 con 0x56201e7f8d90 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d14bfe640 1 -- 172.21.15.7:0/2942551485 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100048c0 con 0x56201e7f8d90 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 -- 172.21.15.7:0/2942551485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 msgr2=0x56201e7fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e7fd890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006010 tx=0x7f9d10009560 comp rx=0 tx=0).stop 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 -- 172.21.15.7:0/2942551485 shutdown_connections 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e800390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e7fd890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/2942551485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e7f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 -- 172.21.15.7:0/2942551485 >> 172.21.15.7:0/2942551485 conn(0x56201e7f3910 msgr2=0x56201e7f3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 -- 172.21.15.7:0/2942551485 shutdown_connections 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.132+0000 7f9d2b16d740 1 -- 172.21.15.7:0/2942551485 wait complete. 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 -- start start 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 Processor -- start 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7f8d90 0x56201e802c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7fdf00 0x56201e809c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56201e800e50 con 0x56201e7f83a0 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56201e7191a0 con 0x56201e7f8d90 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56201e800fd0 con 0x56201e7fdf00 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2b16d740 10 client.0 _ll_register_callbacks cb 0x56201e470c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39420/0 (socket says 172.21.15.7:39420) 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2623d640 1 -- 172.21.15.7:0/3360422472 learned_addr learned my addr 172.21.15.7:0/3360422472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7f8d90 0x56201e802c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d26a3e640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7fdf00 0x56201e809c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2623d640 1 -- 172.21.15.7:0/3360422472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7fdf00 msgr2=0x56201e809c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.133+0000 7f9d2623d640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7fdf00 0x56201e809c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2623d640 1 -- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7f8d90 msgr2=0x56201e802c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2623d640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7f8d90 0x56201e802c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2623d640 1 -- 172.21.15.7:0/3360422472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201e599f70 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2623d640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d18004fd0 tx=0x7f9d1809ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaf8e1c4a7a485 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d09ffb640 1 -- 172.21.15.7:0/3360422472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801c040 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d09ffb640 1 -- 172.21.15.7:0/3360422472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d180996b0 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d10005040 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d09ffb640 1 -- 172.21.15.7:0/3360422472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d1809f500 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 10 client.15852 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56201e431170 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d09ffb640 1 -- 172.21.15.7:0/3360422472 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d10005040 con 0x56201e7f83a0 2024-03-19T00:39:36.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d09ffb640 1 -- 172.21.15.7:0/3360422472 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9d1809f160 con 0x56201e7f83a0 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 20 client.15852 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e80e550 0x56201e810a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 4 client.15852 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.134+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56201e76a6a0 con 0x56201e80e550 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.135+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e80e550 0x56201e810a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.135+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e80e550 0x56201e810a10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fde5030794c61dc server_cookie=eb6d583781c3aa2b in_seq=0 out_seq=0 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.135+0000 7f9d09ffb640 10 client.15852 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d09ffb640 1 -- 172.21.15.7:0/3360422472 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56201e76a6a0 con 0x56201e80e550 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d09ffb640 10 client.15852 handle_command_reply: tid=0 2024-03-19T00:39:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 client.15852 shutdown 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 msgr2=0x56201e802700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9d18004fd0 tx=0x7f9d1809ad90 comp rx=0 tx=0).stop 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 shutdown_connections 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7fdf00 0x56201e809c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7f8d90 0x56201e802c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e80e550 0x56201e810a10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/3360422472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802700 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d09ffb640 0 client.15852 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 >> 172.21.15.7:0/3360422472 conn(0x56201e7f3910 msgr2=0x56201e7f2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 shutdown_connections 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 1 -- 172.21.15.7:0/3360422472 wait complete. 2024-03-19T00:39:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.136+0000 7f9d2b16d740 20 client.15852 trim_cache size 0 max 16384 2024-03-19T00:39:36.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 Processor -- start 2024-03-19T00:39:36.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 -- start start 2024-03-19T00:39:36.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e809c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e7f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201aed70 con 0x56201e7f83a0 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201ac640 con 0x56201e7fdf00 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201aa490 con 0x56201e7f8d90 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39436/0 (socket says 172.21.15.7:39436) 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 -- 172.21.15.7:0/1287902125 learned_addr learned my addr 172.21.15.7:0/1287902125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d26a3e640 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e7f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e809c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 -- 172.21.15.7:0/1287902125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 msgr2=0x56201e809c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e809c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 -- 172.21.15.7:0/1287902125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 msgr2=0x56201e7f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e7f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 -- 172.21.15.7:0/1287902125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201e7f4680 con 0x56201e7f83a0 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.154+0000 7f9d2623d640 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802ba0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d1808dcf0 tx=0x7f9d1809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131f7b3da32a6d8b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d0bfff640 1 -- 172.21.15.7:0/1287902125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801c070 con 0x56201e7f83a0 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d0bfff640 1 -- 172.21.15.7:0/1287902125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d180a3b50 con 0x56201e7f83a0 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d0bfff640 1 -- 172.21.15.7:0/1287902125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180bb0d0 con 0x56201e7f83a0 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1287902125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 msgr2=0x56201e802ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802ba0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d1808dcf0 tx=0x7f9d1809b530 comp rx=0 tx=0).stop 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1287902125 shutdown_connections 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e7f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e809c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1287902125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e802ba0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1287902125 >> 172.21.15.7:0/1287902125 conn(0x56201e7f3910 msgr2=0x56201e7f6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1287902125 shutdown_connections 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1287902125 wait complete. 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.155+0000 7f9d2b16d740 1 -- start start 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 Processor -- start 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e808380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e8088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e808e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201aab00 con 0x56201e7f83a0 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201abf70 con 0x56201e7fdf00 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201ab1d0 con 0x56201e7f8d90 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2b16d740 10 client.0 _ll_register_callbacks cb 0x7f9d1809aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e808380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e808380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39446/0 (socket says 172.21.15.7:39446) 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 -- 172.21.15.7:0/1550483150 learned_addr learned my addr 172.21.15.7:0/1550483150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d26a3e640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e808e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e8088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 -- 172.21.15.7:0/1550483150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 msgr2=0x56201e8088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e8088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 -- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 msgr2=0x56201e808e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e808e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 -- 172.21.15.7:0/1550483150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201e874910 con 0x56201e7f83a0 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d2623d640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e808380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9d18099660 tx=0x7f9d18097780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2efd8baf55671d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.156+0000 7f9d09ffb640 1 -- 172.21.15.7:0/1550483150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801c070 con 0x56201e7f83a0 2024-03-19T00:39:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d09ffb640 1 -- 172.21.15.7:0/1550483150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d180bb0d0 con 0x56201e7f83a0 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d180a3b50 con 0x56201e7f83a0 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d09ffb640 1 -- 172.21.15.7:0/1550483150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d180b4260 con 0x56201e7f83a0 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56201e875b80 con 0x56201e7f83a0 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d09ffb640 1 -- 172.21.15.7:0/1550483150 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d180a3b50 con 0x56201e7f83a0 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d09ffb640 1 -- 172.21.15.7:0/1550483150 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9d18097190 con 0x56201e7f83a0 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 20 client.15858 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e876450 0x56201e878870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 4 client.15858 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56201e7649f0 con 0x56201e876450 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.157+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e876450 0x56201e878870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.158+0000 7f9d09ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.158+0000 7f9d25a3c640 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e876450 0x56201e878870 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0b6ffaff9d9810 server_cookie=ac779e1d107f497b in_seq=0 out_seq=0 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.158+0000 7f9d09ffb640 1 -- 172.21.15.7:0/1550483150 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56201e7649f0 con 0x56201e876450 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.158+0000 7f9d09ffb640 10 client.15858 handle_command_reply: tid=0 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.158+0000 7f9d2b16d740 1 client.15858 shutdown 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 msgr2=0x56201e808380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e808380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9d18099660 tx=0x7f9d18097780 comp rx=0 tx=0).stop 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 shutdown_connections 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56201e7fdf00 0x56201e808e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56201e7f8d90 0x56201e8088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56201e876450 0x56201e878870 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 --2- 172.21.15.7:0/1550483150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56201e7f83a0 0x56201e808380 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 >> 172.21.15.7:0/1550483150 conn(0x56201e7f3910 msgr2=0x56201e7f7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d09ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 shutdown_connections 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 1 -- 172.21.15.7:0/1550483150 wait complete. 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d2b16d740 20 client.15858 trim_cache size 0 max 16384 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:39:36.168 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:39:36.169 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "4ea36a86-4d65-42a5-85f5-eb59e8b6d91b", 2024-03-19T00:39:36.169 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:39:36.169 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:39:36.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d0c0a2800 msgr2=0x7f9d0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d0c0a2800 0x7f9d0c0a4cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c009d80 comp rx=0 tx=0).stop 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 msgr2=0x7f9d20199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 0x7f9d20199d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800e9f0 tx=0x7f9d1800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 shutdown_connections 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d0c0a2800 0x7f9d0c0a4cc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d201013f0 0x7f9d201a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d20100a50 0x7f9d2019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 --2- 172.21.15.7:0/3622386406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d201000f0 0x7f9d20199d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 >> 172.21.15.7:0/3622386406 conn(0x7f9d200fbcb0 msgr2=0x7f9d200fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.159+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 shutdown_connections 2024-03-19T00:39:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.160+0000 7f9d284a1640 1 -- 172.21.15.7:0/3622386406 wait complete. 2024-03-19T00:39:36.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:39:36.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:39:36.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 -- 172.21.15.7:0/942008780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264100740 msgr2=0x7ff264100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/942008780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264100740 0x7ff264100b60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009900 tx=0x7ff25c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 -- 172.21.15.7:0/942008780 shutdown_connections 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/942008780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2641022f0 0x7ff264108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/942008780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264101930 0x7ff264101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/942008780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264100740 0x7ff264100b60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 -- 172.21.15.7:0/942008780 >> 172.21.15.7:0/942008780 conn(0x7ff2640fbcb0 msgr2=0x7ff2640fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.279+0000 7ff26b4a1640 1 -- 172.21.15.7:0/942008780 shutdown_connections 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 -- 172.21.15.7:0/942008780 wait complete. 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 Processor -- start 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 -- start start 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264100740 0x7ff2641046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 0x7ff264102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641022f0 0x7ff264103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff26410c2c0 con 0x7ff264101930 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff26410c140 con 0x7ff2641022f0 2024-03-19T00:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26b4a1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff26410c440 con 0x7ff264100740 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff268a3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 0x7ff264102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff268a3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 0x7ff264102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39470/0 (socket says 172.21.15.7:39470) 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff268a3c640 1 -- 172.21.15.7:0/4127468466 learned_addr learned my addr 172.21.15.7:0/4127468466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff269a3e640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641022f0 0x7ff264103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.280+0000 7ff26923d640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264100740 0x7ff2641046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff268a3c640 1 -- 172.21.15.7:0/4127468466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264100740 msgr2=0x7ff2641046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff268a3c640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264100740 0x7ff2641046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff268a3c640 1 -- 172.21.15.7:0/4127468466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641022f0 msgr2=0x7ff264103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff268a3c640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641022f0 0x7ff264103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff268a3c640 1 -- 172.21.15.7:0/4127468466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264103ae0 con 0x7ff264101930 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff268a3c640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 0x7ff264102d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff25800ea60 tx=0x7ff25800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff2627fc640 1 -- 172.21.15.7:0/4127468466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25800cde0 con 0x7ff264101930 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff2627fc640 1 -- 172.21.15.7:0/4127468466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2580035f0 con 0x7ff264101930 2024-03-19T00:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641a1bc0 con 0x7ff264101930 2024-03-19T00:39:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff2627fc640 1 -- 172.21.15.7:0/4127468466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258010690 con 0x7ff264101930 2024-03-19T00:39:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.281+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a2110 con 0x7ff264101930 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.283+0000 7ff2627fc640 1 -- 172.21.15.7:0/4127468466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7ff2580037a0 con 0x7ff264101930 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 Processor -- start 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 -- start start 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8daa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55da3d837500 con 0x55da3d8d3440 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da3d7dfbe0 con 0x55da3d8d85b0 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26e16a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da3d8335a0 con 0x55da3d8d2a50 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff2627fc640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff24c0a2800 0x7ff24c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26923d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff268a3c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8daa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26923d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8d7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39480/0 (socket says 172.21.15.7:39480) 2024-03-19T00:39:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26923d640 1 -- 172.21.15.7:0/3069855675 learned_addr learned my addr 172.21.15.7:0/3069855675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff2627fc640 1 -- 172.21.15.7:0/4127468466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff2580cf0c0 con 0x7ff264101930 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff269a3e640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff24c0a2800 0x7ff24c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff269a3e640 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26923d640 1 -- 172.21.15.7:0/3069855675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 msgr2=0x55da3d8d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.284+0000 7ff26923d640 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff26923d640 1 -- 172.21.15.7:0/3069855675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 msgr2=0x55da3d8daa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff26923d640 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8daa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff26923d640 1 -- 172.21.15.7:0/3069855675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3d801000 con 0x55da3d8d3440 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff268a3c640 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8daa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff269a3e640 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff269a3e640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff24c0a2800 0x7ff24c0a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55da3d8d3ca0 tx=0x7ff254009be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff26923d640 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8d7f40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68fa50706fbb599 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff24b7fe640 1 -- 172.21.15.7:0/3069855675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c045020 con 0x55da3d8d3440 2024-03-19T00:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff24b7fe640 1 -- 172.21.15.7:0/3069855675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff25c02c9a0 con 0x55da3d8d3440 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.285+0000 7ff24b7fe640 1 -- 172.21.15.7:0/3069855675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0055d0 con 0x55da3d8d3440 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 -- 172.21.15.7:0/3069855675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 msgr2=0x55da3d8d7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8d7f40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c0095f0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 -- 172.21.15.7:0/3069855675 shutdown_connections 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8daa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8d7f40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3069855675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 -- 172.21.15.7:0/3069855675 >> 172.21.15.7:0/3069855675 conn(0x55da3d8cdfc0 msgr2=0x55da3d8ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 -- 172.21.15.7:0/3069855675 shutdown_connections 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 -- 172.21.15.7:0/3069855675 wait complete. 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 -- start start 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 Processor -- start 2024-03-19T00:39:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.286+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55da3d8db500 con 0x55da3d8d3440 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da3d7f3160 con 0x55da3d8d85b0 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55da3d8db680 con 0x55da3d8d2a50 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 10 client.0 _ll_register_callbacks cb 0x55da3d82fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8dd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39492/0 (socket says 172.21.15.7:39492) 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 -- 172.21.15.7:0/1343290296 learned_addr learned my addr 172.21.15.7:0/1343290296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26923d640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff269a3e640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 -- 172.21.15.7:0/1343290296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 msgr2=0x55da3d8dce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8dce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 -- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 msgr2=0x55da3d8e43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 -- 172.21.15.7:0/1343290296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3d6157d0 con 0x55da3d8d3440 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26923d640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff269a3e640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8dd380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff25800d010 tx=0x7ff25800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f447d2f4ddba2687 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff248ff9640 1 -- 172.21.15.7:0/1343290296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258003b30 con 0x55da3d8d3440 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff248ff9640 1 -- 172.21.15.7:0/1343290296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff2580ab3d0 con 0x55da3d8d3440 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff25c02c9a0 con 0x55da3d8d3440 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff248ff9640 1 -- 172.21.15.7:0/1343290296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2580ab5d0 con 0x55da3d8d3440 2024-03-19T00:39:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.287+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da3d54b3f0 con 0x55da3d8d3440 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff248ff9640 1 -- 172.21.15.7:0/1343290296 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff25c02c9a0 con 0x55da3d8d3440 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff248ff9640 1 -- 172.21.15.7:0/1343290296 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff258007c00 con 0x55da3d8d3440 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26e16a740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26e16a740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26e16a740 20 client.15870 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d8e8dc0 0x55da3d8eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26e16a740 4 client.15870 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da3d844a40 con 0x55da3d8e8dc0 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26923d640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d8e8dc0 0x55da3d8eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff248ff9640 10 client.15870 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.288+0000 7ff26923d640 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d8e8dc0 0x55da3d8eb240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900812edc407a215 server_cookie=3b8b687a8cc3a349 in_seq=0 out_seq=0 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff248ff9640 1 -- 172.21.15.7:0/1343290296 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55da3d844a40 con 0x55da3d8e8dc0 2024-03-19T00:39:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff248ff9640 10 client.15870 handle_command_reply: tid=0 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 client.15870 shutdown 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 msgr2=0x55da3d8dd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8dd380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff25800d010 tx=0x7ff25800a620 comp rx=0 tx=0).stop 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 shutdown_connections 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d8e8dc0 0x55da3d8eb240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d3440 0x55da3d8dd380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1343290296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff248ff9640 0 client.15870 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 >> 172.21.15.7:0/1343290296 conn(0x55da3d8cdfc0 msgr2=0x55da3d8cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 shutdown_connections 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 1 -- 172.21.15.7:0/1343290296 wait complete. 2024-03-19T00:39:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.289+0000 7ff26e16a740 20 client.15870 trim_cache size 0 max 16384 2024-03-19T00:39:36.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 Processor -- start 2024-03-19T00:39:36.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 -- start start 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d3440 0x55da3d8e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 0x55da3d8cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641aec00 con 0x55da3d8d85b0 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641ac4d0 con 0x55da3d8d3440 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26e16a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641aa320 con 0x55da3d8d2a50 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 0x55da3d8cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 0x55da3d8cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39494/0 (socket says 172.21.15.7:39494) 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 -- 172.21.15.7:0/3433575131 learned_addr learned my addr 172.21.15.7:0/3433575131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff26923d640 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff268a3c640 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d3440 0x55da3d8e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 -- 172.21.15.7:0/3433575131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 msgr2=0x55da3d8cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 -- 172.21.15.7:0/3433575131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d3440 msgr2=0x55da3d8e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d3440 0x55da3d8e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 -- 172.21.15.7:0/3433575131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3d50b0f0 con 0x55da3d8d85b0 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.307+0000 7ff269a3e640 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 0x55da3d8cce90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff254003e60 tx=0x7ff254009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e1fa8d028c4fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff24affd640 1 -- 172.21.15.7:0/3433575131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25400fe00 con 0x55da3d8d85b0 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff24affd640 1 -- 172.21.15.7:0/3433575131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2540045d0 con 0x55da3d8d85b0 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff24affd640 1 -- 172.21.15.7:0/3433575131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254005700 con 0x55da3d8d85b0 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 -- 172.21.15.7:0/3433575131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 msgr2=0x55da3d8cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 0x55da3d8cce90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff254003e60 tx=0x7ff254009500 comp rx=0 tx=0).stop 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 -- 172.21.15.7:0/3433575131 shutdown_connections 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d85b0 0x55da3d8cce90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d3440 0x55da3d8e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 --2- 172.21.15.7:0/3433575131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d2a50 0x55da3d8cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 -- 172.21.15.7:0/3433575131 >> 172.21.15.7:0/3433575131 conn(0x55da3d8cdfc0 msgr2=0x55da3d8dce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 -- 172.21.15.7:0/3433575131 shutdown_connections 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.308+0000 7ff26e16a740 1 -- 172.21.15.7:0/3433575131 wait complete. 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 -- start start 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 Processor -- start 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 0x55da3d8e1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d3440 0x55da3d8e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641aa990 con 0x55da3d8d2a50 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641abe00 con 0x55da3d8d85b0 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641ab060 con 0x55da3d8d3440 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26e16a740 10 client.0 _ll_register_callbacks cb 0x7ff258009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 0x55da3d8e1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 0x55da3d8e1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39496/0 (socket says 172.21.15.7:39496) 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 -- 172.21.15.7:0/1730933226 learned_addr learned my addr 172.21.15.7:0/1730933226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:39:36.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff269a3e640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d3440 0x55da3d8e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 -- 172.21.15.7:0/1730933226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d3440 msgr2=0x55da3d8e1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d3440 0x55da3d8e1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 -- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 msgr2=0x55da3d8e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff26923d640 1 -- 172.21.15.7:0/1730933226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3d8e31b0 con 0x55da3d8d2a50 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff269a3e640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.309+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d3440 0x55da3d8e1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26923d640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 0x55da3d8e1670 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009900 tx=0x7ff25c02c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18827b7ddb1146c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff248ff9640 1 -- 172.21.15.7:0/1730933226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c045040 con 0x55da3d8d2a50 2024-03-19T00:39:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff248ff9640 1 -- 172.21.15.7:0/1730933226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff25c0035f0 con 0x55da3d8d2a50 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2540045d0 con 0x55da3d8d2a50 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff248ff9640 1 -- 172.21.15.7:0/1730933226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff25c05e050 con 0x55da3d8d2a50 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da3d8e4530 con 0x55da3d8d2a50 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff248ff9640 1 -- 172.21.15.7:0/1730933226 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2540045d0 con 0x55da3d8d2a50 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff248ff9640 1 -- 172.21.15.7:0/1730933226 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff25c0415a0 con 0x55da3d8d2a50 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 20 client.15876 populate_metadata read hostname 'smithi007' 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d94f3c0 0x55da3d9517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 4 client.15876 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da3d83ed90 con 0x55da3d94f3c0 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.310+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d94f3c0 0x55da3d9517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:39:36.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff268a3c640 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d94f3c0 0x55da3d9517b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef40115671680e0d server_cookie=11d807743442d77c in_seq=0 out_seq=0 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff248ff9640 10 client.15876 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff248ff9640 1 -- 172.21.15.7:0/1730933226 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da3d83ed90 con 0x55da3d94f3c0 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff248ff9640 10 client.15876 handle_command_reply: tid=0 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 client.15876 shutdown 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 msgr2=0x55da3d8e1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 0x55da3d8e1670 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009900 tx=0x7ff25c02c610 comp rx=0 tx=0).stop 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 shutdown_connections 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da3d8d85b0 0x55da3d8e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55da3d8d3440 0x55da3d8e1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55da3d94f3c0 0x55da3d9517b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 --2- 172.21.15.7:0/1730933226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55da3d8d2a50 0x55da3d8e1670 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff248ff9640 0 client.15876 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:39:36.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 >> 172.21.15.7:0/1730933226 conn(0x55da3d8cdfc0 msgr2=0x55da3d8d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 shutdown_connections 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 1 -- 172.21.15.7:0/1730933226 wait complete. 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.311+0000 7ff26e16a740 20 client.15876 trim_cache size 0 max 16384 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "4ea36a86-4d65-42a5-85f5-eb59e8b6d91b": { 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:39:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "4ea36a86-4d65-42a5-85f5-eb59e8b6d91b", 2024-03-19T00:39:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:39:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:39:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:39:36.321 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff24c0a2800 msgr2=0x7ff24c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff24c0a2800 0x7ff24c0a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55da3d8d3ca0 tx=0x7ff254009be0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 msgr2=0x7ff264102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 0x7ff264102d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff25800ea60 tx=0x7ff25800ef20 comp rx=0 tx=0).stop 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 shutdown_connections 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff24c0a2800 0x7ff24c0a4cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641022f0 0x7ff264103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff264101930 0x7ff264102d10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 --2- 172.21.15.7:0/4127468466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff264100740 0x7ff2641046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:39:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 >> 172.21.15.7:0/4127468466 conn(0x7ff2640fbcb0 msgr2=0x7ff2640fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:39:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 shutdown_connections 2024-03-19T00:39:36.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:39:36.312+0000 7ff26b4a1640 1 -- 172.21.15.7:0/4127468466 wait complete. 2024-03-19T00:39:36.325 INFO:tasks.cephfs.filesystem:scrub status for tag:4ea36a86-4d65-42a5-85f5-eb59e8b6d91b - {'path': '/', 'tag': '4ea36a86-4d65-42a5-85f5-eb59e8b6d91b', 'options': 'recursive,force'} 2024-03-19T00:39:36.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:36 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:39:36.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:36 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:36.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:36 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:39:36.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:36 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:36.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:36 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:39:36.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:36 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:37 smithi007 ceph-mon[62854]: pgmap v401: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-03-19T00:39:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:37 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:39:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:37 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:37 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:39:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:37 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:37 smithi204 ceph-mon[66572]: pgmap v401: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-03-19T00:39:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:37 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:39:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:37 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:37 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:39:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:37 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:37 smithi044 ceph-mon[66366]: pgmap v401: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-03-19T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:37 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:37 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:37 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:39:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:37 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:39:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:39 smithi007 ceph-mon[62854]: pgmap v402: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-03-19T00:39:39.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:39 smithi204 ceph-mon[66572]: pgmap v402: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-03-19T00:39:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:39 smithi044 ceph-mon[66366]: pgmap v402: 129 pgs: 129 active+clean; 812 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-03-19T00:39:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:41 smithi007 ceph-mon[62854]: pgmap v403: 129 pgs: 129 active+clean; 835 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-03-19T00:39:41.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:41 smithi204 ceph-mon[66572]: pgmap v403: 129 pgs: 129 active+clean; 835 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-03-19T00:39:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:41 smithi044 ceph-mon[66366]: pgmap v403: 129 pgs: 129 active+clean; 835 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:42 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:42 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:42.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:42 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:43.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:43 smithi007 ceph-mon[62854]: pgmap v404: 129 pgs: 129 active+clean; 835 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:39:43.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:39:43.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:39:43.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:43.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:43 smithi204 ceph-mon[66572]: pgmap v404: 129 pgs: 129 active+clean; 835 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:39:43.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:39:43.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:39:43.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:43 smithi044 ceph-mon[66366]: pgmap v404: 129 pgs: 129 active+clean; 835 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:39:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:39:45.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:45 smithi007 ceph-mon[62854]: pgmap v405: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:39:45.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:45 smithi204 ceph-mon[66572]: pgmap v405: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:39:45.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:45 smithi044 ceph-mon[66366]: pgmap v405: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:39:46.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:39:46.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:39:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:39:46.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:39:46.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:39:46.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:39:47.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:47 smithi007 ceph-mon[62854]: pgmap v406: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:39:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:47 smithi204 ceph-mon[66572]: pgmap v406: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:39:47.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:47 smithi044 ceph-mon[66366]: pgmap v406: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:39:49.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:49 smithi007 ceph-mon[62854]: pgmap v407: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:39:49.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:49 smithi204 ceph-mon[66572]: pgmap v407: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:39:49.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:49 smithi044 ceph-mon[66366]: pgmap v407: 129 pgs: 129 active+clean; 858 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:39:51.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:51 smithi007 ceph-mon[62854]: pgmap v408: 129 pgs: 129 active+clean; 874 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:39:51.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:51 smithi204 ceph-mon[66572]: pgmap v408: 129 pgs: 129 active+clean; 874 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:39:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:51 smithi044 ceph-mon[66366]: pgmap v408: 129 pgs: 129 active+clean; 874 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:39:53.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:53 smithi204 ceph-mon[66572]: pgmap v409: 129 pgs: 129 active+clean; 874 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:53 smithi044 ceph-mon[66366]: pgmap v409: 129 pgs: 129 active+clean; 874 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:53.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:53 smithi007 ceph-mon[62854]: pgmap v409: 129 pgs: 129 active+clean; 874 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:55 smithi204 ceph-mon[66572]: pgmap v410: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:39:55.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:55 smithi044 ceph-mon[66366]: pgmap v410: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:39:55.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:55 smithi007 ceph-mon[62854]: pgmap v410: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:39:57.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:57 smithi204 ceph-mon[66572]: pgmap v411: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:57 smithi044 ceph-mon[66366]: pgmap v411: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:57.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:57 smithi007 ceph-mon[62854]: pgmap v411: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:39:59 smithi204 ceph-mon[66572]: pgmap v412: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:59.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:39:59 smithi044 ceph-mon[66366]: pgmap v412: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:39:59.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:39:59 smithi007 ceph-mon[62854]: pgmap v412: 129 pgs: 129 active+clean; 897 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:00.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:00 smithi204 ceph-mon[66572]: overall HEALTH_OK 2024-03-19T00:40:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:00 smithi044 ceph-mon[66366]: overall HEALTH_OK 2024-03-19T00:40:00.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:00 smithi007 ceph-mon[62854]: overall HEALTH_OK 2024-03-19T00:40:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:01 smithi204 ceph-mon[66572]: pgmap v413: 129 pgs: 129 active+clean; 914 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:40:01.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:01 smithi044 ceph-mon[66366]: pgmap v413: 129 pgs: 129 active+clean; 914 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:40:01.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:01 smithi007 ceph-mon[62854]: pgmap v413: 129 pgs: 129 active+clean; 914 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:40:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:03 smithi204 ceph-mon[66572]: pgmap v414: 129 pgs: 129 active+clean; 914 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:40:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:03 smithi044 ceph-mon[66366]: pgmap v414: 129 pgs: 129 active+clean; 914 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:40:03.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:03 smithi007 ceph-mon[62854]: pgmap v414: 129 pgs: 129 active+clean; 914 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:40:05.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:05 smithi204 ceph-mon[66572]: pgmap v415: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-03-19T00:40:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:05 smithi044 ceph-mon[66366]: pgmap v415: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-03-19T00:40:05.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:05 smithi007 ceph-mon[62854]: pgmap v415: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-03-19T00:40:06.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:40:06.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:40:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 -- 172.21.15.7:0/3184486369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 msgr2=0x7f0320073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 --2- 172.21.15.7:0/3184486369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f0320073cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0310009930 tx=0x7f031002bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 -- 172.21.15.7:0/3184486369 shutdown_connections 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 --2- 172.21.15.7:0/3184486369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f032010de20 0x7f0320110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 --2- 172.21.15.7:0/3184486369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0320074280 0x7f0320074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 --2- 172.21.15.7:0/3184486369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f0320073cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 -- 172.21.15.7:0/3184486369 >> 172.21.15.7:0/3184486369 conn(0x7f03200fcde0 msgr2=0x7f03200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 -- 172.21.15.7:0/3184486369 shutdown_connections 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.446+0000 7f0326812640 1 -- 172.21.15.7:0/3184486369 wait complete. 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 Processor -- start 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 -- start start 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0320074280 0x7f032019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f032019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 0x7f03201a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03201139a0 con 0x7f032010de20 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0320113820 con 0x7f0320075860 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0326812640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0320113b20 con 0x7f0320074280 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0324daf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 0x7f03201a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f031ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0320074280 0x7f032019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0324daf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 0x7f03201a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54602/0 (socket says 172.21.15.7:54602) 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f032019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.447+0000 7f0324daf640 1 -- 172.21.15.7:0/3306816953 learned_addr learned my addr 172.21.15.7:0/3306816953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0324daf640 1 -- 172.21.15.7:0/3306816953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0320074280 msgr2=0x7f032019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0324daf640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0320074280 0x7f032019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0324daf640 1 -- 172.21.15.7:0/3306816953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 msgr2=0x7f032019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0324daf640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f032019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0324daf640 1 -- 172.21.15.7:0/3306816953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03201a5ce0 con 0x7f032010de20 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f031ffff640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0320074280 0x7f032019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f031f7fe640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f032019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0324daf640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 0x7f03201a5670 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f031800eb10 tx=0x7f031800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f031d7fa640 1 -- 172.21.15.7:0/3306816953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318004b20 con 0x7f032010de20 2024-03-19T00:40:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f031d7fa640 1 -- 172.21.15.7:0/3306816953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0318004cc0 con 0x7f032010de20 2024-03-19T00:40:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a5fd0 con 0x7f032010de20 2024-03-19T00:40:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f031d7fa640 1 -- 172.21.15.7:0/3306816953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318010690 con 0x7f032010de20 2024-03-19T00:40:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.448+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a6440 con 0x7f032010de20 2024-03-19T00:40:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.450+0000 7f031d7fa640 1 -- 172.21.15.7:0/3306816953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0318005020 con 0x7f032010de20 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 Processor -- start 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 -- start start 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994441e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d994446f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d994449a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d9943a6500 con 0x55d994441a50 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d99434ebe0 con 0x55d9944475b0 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0329404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d9943a25a0 con 0x55d994442440 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031d7fa640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03080a2800 0x7f03080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994441e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994441e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54604/0 (socket says 172.21.15.7:54604) 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031f7fe640 1 -- 172.21.15.7:0/3640521460 learned_addr learned my addr 172.21.15.7:0/3640521460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031ffff640 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d994446f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031f7fe640 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d994449a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f0324daf640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03080a2800 0x7f03080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031d7fa640 1 -- 172.21.15.7:0/3306816953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f03180cf330 con 0x7f032010de20 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031ffff640 1 -- 172.21.15.7:0/3640521460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 msgr2=0x55d994449a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031ffff640 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d994449a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031ffff640 1 -- 172.21.15.7:0/3640521460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 msgr2=0x55d994441e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031ffff640 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994441e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031ffff640 1 -- 172.21.15.7:0/3640521460 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d994370000 con 0x55d994442440 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.451+0000 7f031f7fe640 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994441e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.452+0000 7f0324daf640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03080a2800 0x7f03080a4cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0318097dd0 tx=0x7f0318066730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.452+0000 7f031ffff640 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d994446f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0310036040 tx=0x7f03100095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d103828736ebdbc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.452+0000 7f03067fc640 1 -- 172.21.15.7:0/3640521460 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310045020 con 0x55d994442440 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.452+0000 7f03067fc640 1 -- 172.21.15.7:0/3640521460 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031002cc90 con 0x55d994442440 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.452+0000 7f03067fc640 1 -- 172.21.15.7:0/3640521460 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310004920 con 0x55d994442440 2024-03-19T00:40:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 -- 172.21.15.7:0/3640521460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 msgr2=0x55d994446f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d994446f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0310036040 tx=0x7f03100095e0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 -- 172.21.15.7:0/3640521460 shutdown_connections 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d994449a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d994446f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 --2- 172.21.15.7:0/3640521460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994441e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 -- 172.21.15.7:0/3640521460 >> 172.21.15.7:0/3640521460 conn(0x55d99443cfc0 msgr2=0x55d99443d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 -- 172.21.15.7:0/3640521460 shutdown_connections 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 -- 172.21.15.7:0/3640521460 wait complete. 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 -- start start 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 Processor -- start 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994441a50 0x55d99444be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.453+0000 7f0329404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d99444c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f0329404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9944475b0 0x55d9944533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f0329404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d99444a500 con 0x55d9944475b0 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f0329404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d994362160 con 0x55d994441a50 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f0329404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d99444a680 con 0x55d994442440 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f0329404740 10 client.0 _ll_register_callbacks cb 0x55d99439ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994441a50 0x55d99444be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d99444c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d99444c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:32894/0 (socket says 172.21.15.7:32894) 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994441a50 0x55d99444be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41396/0 (socket says 172.21.15.7:41396) 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 -- 172.21.15.7:0/3021837935 learned_addr learned my addr 172.21.15.7:0/3021837935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031f7fe640 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9944475b0 0x55d9944533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 -- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994441a50 msgr2=0x55d99444be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994441a50 0x55d99444be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 -- 172.21.15.7:0/3021837935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9944475b0 msgr2=0x55d9944533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9944475b0 0x55d9944533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 -- 172.21.15.7:0/3021837935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9941847d0 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f031ffff640 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d99444c3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f031002c170 tx=0x7f03100056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e4be66378de4a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f02ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f02ff7fe640 1 -- 172.21.15.7:0/3021837935 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310045040 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f02ff7fe640 1 -- 172.21.15.7:0/3021837935 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0310003b00 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.454+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031002cc90 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f02ff7fe640 1 -- 172.21.15.7:0/3021837935 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0310059050 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 10 client.25651 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9940ba3f0 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f02ff7fe640 1 -- 172.21.15.7:0/3021837935 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031002cc90 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f02ff7fe640 1 -- 172.21.15.7:0/3021837935 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f03100554e0 con 0x55d994442440 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 10 client.25651 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 20 client.25651 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d994457980 0x55d994459db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 4 client.25651 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9943b3a40 con 0x55d994457980 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.455+0000 7f031f7fe640 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d994457980 0x55d994459db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.456+0000 7f02ff7fe640 10 client.25651 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.456+0000 7f031f7fe640 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d994457980 0x55d994459db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b2083f81af35af server_cookie=16e3b84a2521be2b in_seq=0 out_seq=0 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f02ff7fe640 1 -- 172.21.15.7:0/3021837935 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d9943b3a40 con 0x55d994457980 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f02ff7fe640 10 client.25651 handle_command_reply: tid=0 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 client.25651 shutdown 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 msgr2=0x55d99444c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d99444c3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f031002c170 tx=0x7f03100056b0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 shutdown_connections 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d994457980 0x55d994459db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9944475b0 0x55d9944533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994442440 0x55d99444c3b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f02ff7fe640 0 client.25651 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 --2- 172.21.15.7:0/3021837935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994441a50 0x55d99444be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 >> 172.21.15.7:0/3021837935 conn(0x55d99443cfc0 msgr2=0x55d99443c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 shutdown_connections 2024-03-19T00:40:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 1 -- 172.21.15.7:0/3021837935 wait complete. 2024-03-19T00:40:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.457+0000 7f0329404740 20 client.25651 trim_cache size 0 max 16384 2024-03-19T00:40:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 Processor -- start 2024-03-19T00:40:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 -- start start 2024-03-19T00:40:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994441a50 0x55d9944438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 0x55d9944533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d99445e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03201b30b0 con 0x55d994442440 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03201b0980 con 0x55d9944475b0 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0329404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03201ae7d0 con 0x55d994441a50 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 0x55d9944533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 0x55d9944533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54612/0 (socket says 172.21.15.7:54612) 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 -- 172.21.15.7:0/874377430 learned_addr learned my addr 172.21.15.7:0/874377430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031f7fe640 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994441a50 0x55d9944438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031f7fe640 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d99445e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 -- 172.21.15.7:0/874377430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994441a50 msgr2=0x55d9944438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994441a50 0x55d9944438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 -- 172.21.15.7:0/874377430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 msgr2=0x55d99445e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d99445e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 -- 172.21.15.7:0/874377430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99407a0f0 con 0x55d994442440 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031f7fe640 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994441a50 0x55d9944438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f031ffff640 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 0x55d9944533b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f03100098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5679e1cf605050 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.475+0000 7f0305ffb640 1 -- 172.21.15.7:0/874377430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310045040 con 0x55d994442440 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0305ffb640 1 -- 172.21.15.7:0/874377430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031002cc90 con 0x55d994442440 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0305ffb640 1 -- 172.21.15.7:0/874377430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310055490 con 0x55d994442440 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 -- 172.21.15.7:0/874377430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 msgr2=0x55d9944533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 0x55d9944533b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f03100098c0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 -- 172.21.15.7:0/874377430 shutdown_connections 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d9944475b0 0x55d99445e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994442440 0x55d9944533b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 --2- 172.21.15.7:0/874377430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d994441a50 0x55d9944438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 -- 172.21.15.7:0/874377430 >> 172.21.15.7:0/874377430 conn(0x55d99443cfc0 msgr2=0x55d994440ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 -- 172.21.15.7:0/874377430 shutdown_connections 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 -- 172.21.15.7:0/874377430 wait complete. 2024-03-19T00:40:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.476+0000 7f0329404740 1 -- start start 2024-03-19T00:40:06.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 Processor -- start 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994451ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994442440 0x55d994452010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9944475b0 0x55d994452550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03201aee40 con 0x55d994441a50 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03201b02b0 con 0x55d994442440 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03201af510 con 0x55d9944475b0 2024-03-19T00:40:06.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f0329404740 10 client.0 _ll_register_callbacks cb 0x7f0314009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994451ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994451ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54624/0 (socket says 172.21.15.7:54624) 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 -- 172.21.15.7:0/2060746333 learned_addr learned my addr 172.21.15.7:0/2060746333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994442440 0x55d994452010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9944475b0 0x55d994452550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 -- 172.21.15.7:0/2060746333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9944475b0 msgr2=0x55d994452550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9944475b0 0x55d994452550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 -- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994442440 msgr2=0x55d994452010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994442440 0x55d994452010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 -- 172.21.15.7:0/2060746333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9944bdcb0 con 0x55d994441a50 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f031f7fe640 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994451ad0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55d99445a5f0 tx=0x7f0310005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb4e415b303ac08 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.477+0000 7f02ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f02ff7fe640 1 -- 172.21.15.7:0/2060746333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031402d020 con 0x55d994441a50 2024-03-19T00:40:06.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f02ff7fe640 1 -- 172.21.15.7:0/2060746333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f03140275a0 con 0x55d994441a50 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031002cc90 con 0x55d994441a50 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f02ff7fe640 1 -- 172.21.15.7:0/2060746333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f031401c040 con 0x55d994441a50 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9944bef50 con 0x55d994441a50 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f02ff7fe640 1 -- 172.21.15.7:0/2060746333 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031002cc90 con 0x55d994441a50 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f02ff7fe640 1 -- 172.21.15.7:0/2060746333 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f03140050d0 con 0x55d994441a50 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 20 client.15894 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d9944bf960 0x55d9944c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 4 client.15894 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9943add90 con 0x55d9944bf960 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.478+0000 7f031ffff640 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d9944bf960 0x55d9944c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f02ff7fe640 10 client.15894 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f031ffff640 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d9944bf960 0x55d9944c1d80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a11d570698c9b2 server_cookie=95c23e40b003a6f3 in_seq=0 out_seq=0 2024-03-19T00:40:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f02ff7fe640 1 -- 172.21.15.7:0/2060746333 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d9943add90 con 0x55d9944bf960 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f02ff7fe640 10 client.15894 handle_command_reply: tid=0 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 client.15894 shutdown 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 msgr2=0x55d994451ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994451ad0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55d99445a5f0 tx=0x7f0310005e50 comp rx=0 tx=0).stop 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 shutdown_connections 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9944475b0 0x55d994452550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d994442440 0x55d994452010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d9944bf960 0x55d9944c1d80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 --2- 172.21.15.7:0/2060746333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d994441a50 0x55d994451ad0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f02ff7fe640 0 client.15894 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 >> 172.21.15.7:0/2060746333 conn(0x55d99443cfc0 msgr2=0x55d99443b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 shutdown_connections 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 1 -- 172.21.15.7:0/2060746333 wait complete. 2024-03-19T00:40:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.479+0000 7f0329404740 20 client.15894 trim_cache size 0 max 16384 2024-03-19T00:40:06.487 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:40:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:40:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:40:06.487 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03080a2800 msgr2=0x7f03080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03080a2800 0x7f03080a4cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0318097dd0 tx=0x7f0318066730 comp rx=0 tx=0).stop 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 msgr2=0x7f03201a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 0x7f03201a5670 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f031800eb10 tx=0x7f031800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 shutdown_connections 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03080a2800 0x7f03080a4cc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f032010de20 0x7f03201a5670 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0320075860 0x7f032019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 --2- 172.21.15.7:0/3306816953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0320074280 0x7f032019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 >> 172.21.15.7:0/3306816953 conn(0x7f03200fcde0 msgr2=0x7f03201011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 shutdown_connections 2024-03-19T00:40:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.480+0000 7f0326812640 1 -- 172.21.15.7:0/3306816953 wait complete. 2024-03-19T00:40:06.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:40:06.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 -- 172.21.15.7:0/3275309965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc101940 msgr2=0x7fe9dc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 --2- 172.21.15.7:0/3275309965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc101940 0x7fe9dc101dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009900 tx=0x7fe9c402bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 -- 172.21.15.7:0/3275309965 shutdown_connections 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 --2- 172.21.15.7:0/3275309965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9dc102300 0x7fe9dc108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 --2- 172.21.15.7:0/3275309965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc101940 0x7fe9dc101dc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 --2- 172.21.15.7:0/3275309965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9dc100750 0x7fe9dc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 -- 172.21.15.7:0/3275309965 >> 172.21.15.7:0/3275309965 conn(0x7fe9dc0fbcc0 msgr2=0x7fe9dc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 -- 172.21.15.7:0/3275309965 shutdown_connections 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.595+0000 7fe9e1297640 1 -- 172.21.15.7:0/3275309965 wait complete. 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 Processor -- start 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 -- start start 2024-03-19T00:40:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9dc100750 0x7fe9dc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9dc101940 0x7fe9dc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 0x7fe9dc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10c280 con 0x7fe9dc102300 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10c100 con 0x7fe9dc100750 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9e1297640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10c400 con 0x7fe9dc101940 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 0x7fe9dc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 0x7fe9dc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54642/0 (socket says 172.21.15.7:54642) 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 -- 172.21.15.7:0/1892399876 learned_addr learned my addr 172.21.15.7:0/1892399876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9daffd640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9dc100750 0x7fe9dc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9d3fff640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9dc101940 0x7fe9dc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 -- 172.21.15.7:0/1892399876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9dc101940 msgr2=0x7fe9dc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9dc101940 0x7fe9dc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 -- 172.21.15.7:0/1892399876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9dc100750 msgr2=0x7fe9dc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9dc100750 0x7fe9dc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 -- 172.21.15.7:0/1892399876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc19f710 con 0x7fe9dc102300 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9db7fe640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 0x7fe9dc19f0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400efd0 tx=0x7fe9d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.596+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003b70 con 0x7fe9dc102300 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.597+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9d4003d10 con 0x7fe9dc102300 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.597+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4010980 con 0x7fe9dc102300 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.597+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc19fa00 con 0x7fe9dc102300 2024-03-19T00:40:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.597+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc19fed0 con 0x7fe9dc102300 2024-03-19T00:40:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.597+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc1a0100 con 0x7fe9dc102300 2024-03-19T00:40:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.599+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe9d4004020 con 0x7fe9dc102300 2024-03-19T00:40:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.599+0000 7fe9d8ff9640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9c80a2800 0x7fe9c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.599+0000 7fe9daffd640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9c80a2800 0x7fe9c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.599+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe9d40cf280 con 0x7fe9dc102300 2024-03-19T00:40:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.600+0000 7fe9daffd640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9c80a2800 0x7fe9c80a4cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc1017a0 tx=0x7fe9cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.604+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe9d409bf60 con 0x7fe9dc102300 2024-03-19T00:40:06.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.827+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9dc1a0370 con 0x7fe9dc102300 2024-03-19T00:40:06.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.827+0000 7fe9d8ff9640 1 -- 172.21.15.7:0/1892399876 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11856 (secure 0 0 0) 0x7fe9d409b860 con 0x7fe9dc102300 2024-03-19T00:40:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:40:06.829 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:39:16.038000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:40:06.829 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9c80a2800 msgr2=0x7fe9c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9c80a2800 0x7fe9c80a4cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc1017a0 tx=0x7fe9cc005f90 comp rx=0 tx=0).stop 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 msgr2=0x7fe9dc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 0x7fe9dc19f0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400efd0 tx=0x7fe9d400c6a0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 shutdown_connections 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9c80a2800 0x7fe9c80a4cc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9dc102300 0x7fe9dc19f0a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9dc101940 0x7fe9dc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 --2- 172.21.15.7:0/1892399876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9dc100750 0x7fe9dc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.829+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 >> 172.21.15.7:0/1892399876 conn(0x7fe9dc0fbcc0 msgr2=0x7fe9dc068be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.830+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 shutdown_connections 2024-03-19T00:40:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.830+0000 7fe9e1297640 1 -- 172.21.15.7:0/1892399876 wait complete. 2024-03-19T00:40:06.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 -- 172.21.15.7:0/313340414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 msgr2=0x7f14e4107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 --2- 172.21.15.7:0/313340414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4107040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009900 tx=0x7f14cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 -- 172.21.15.7:0/313340414 shutdown_connections 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 --2- 172.21.15.7:0/313340414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 --2- 172.21.15.7:0/313340414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4107040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 --2- 172.21.15.7:0/313340414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4102320 0x7f14e4104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 -- 172.21.15.7:0/313340414 >> 172.21.15.7:0/313340414 conn(0x7f14e40fbc90 msgr2=0x7f14e40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 -- 172.21.15.7:0/313340414 shutdown_connections 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 -- 172.21.15.7:0/313340414 wait complete. 2024-03-19T00:40:06.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.946+0000 7f14e92ff640 1 Processor -- start 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 -- start start 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4102320 0x7f14e4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14e410d360 con 0x7f14e4107580 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14e410d1e0 con 0x7f14e4104c50 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e92ff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14e410d4e0 con 0x7f14e4102320 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41450/0 (socket says 172.21.15.7:41450) 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54656/0 (socket says 172.21.15.7:54656) 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 -- 172.21.15.7:0/2599269402 learned_addr learned my addr 172.21.15.7:0/2599269402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4102320 0x7f14e4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 -- 172.21.15.7:0/2599269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4102320 msgr2=0x7f14e4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4102320 0x7f14e4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 -- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 msgr2=0x7f14e419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14da7fc640 1 -- 172.21.15.7:0/2599269402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e419f6b0 con 0x7f14e4104c50 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.947+0000 7f14e37fe640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.948+0000 7f14da7fc640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4197fd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14cc000c00 tx=0x7f14cc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.948+0000 7f14e0ff9640 1 -- 172.21.15.7:0/2599269402 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc043020 con 0x7f14e4104c50 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.948+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e419f940 con 0x7f14e4104c50 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.948+0000 7f14e0ff9640 1 -- 172.21.15.7:0/2599269402 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14cc004350 con 0x7f14e4104c50 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.948+0000 7f14e0ff9640 1 -- 172.21.15.7:0/2599269402 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc003b80 con 0x7f14e4104c50 2024-03-19T00:40:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.948+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e419fe90 con 0x7f14e4104c50 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.950+0000 7f14e0ff9640 1 -- 172.21.15.7:0/2599269402 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f14cc046020 con 0x7f14e4104c50 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.950+0000 7f14ebfc9740 1 Processor -- start 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.950+0000 7f14ebfc9740 1 -- start start 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.950+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce72ce0 0x560f4ce777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce77e50 0x560f4ce7a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f4cdd64b0 con 0x560f4ce722f0 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560f4cd7ed70 con 0x560f4ce77e50 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f4cdd2550 con 0x560f4ce72ce0 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce72710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54670/0 (socket says 172.21.15.7:54670) 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14da7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce77e50 0x560f4ce7a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e37fe640 1 -- 172.21.15.7:0/282441636 learned_addr learned my addr 172.21.15.7:0/282441636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce72ce0 0x560f4ce777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e0ff9640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14d00a2800 0x7f14d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 -- 172.21.15.7:0/282441636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce77e50 msgr2=0x560f4ce7a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce77e50 0x560f4ce7a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 -- 172.21.15.7:0/282441636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 msgr2=0x560f4ce72710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce72710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 -- 172.21.15.7:0/282441636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4cda1210 con 0x560f4ce72ce0 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e37fe640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14d00a2800 0x7f14d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e0ff9640 1 -- 172.21.15.7:0/2599269402 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14cc0f0f40 con 0x7f14e4104c50 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce72ce0 0x560f4ce777e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009c00 tx=0x7f14d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c999a7abe777e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14d97fa640 1 -- 172.21.15.7:0/282441636 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4004230 con 0x560f4ce72ce0 2024-03-19T00:40:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14e37fe640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14d00a2800 0x7f14d00a4cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x560f4ce73540 tx=0x7f14dc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.951+0000 7f14d97fa640 1 -- 172.21.15.7:0/282441636 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d40043d0 con 0x560f4ce72ce0 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14d97fa640 1 -- 172.21.15.7:0/282441636 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4004c60 con 0x560f4ce72ce0 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 -- 172.21.15.7:0/282441636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce72ce0 msgr2=0x560f4ce777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce72ce0 0x560f4ce777e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009c00 tx=0x7f14d4011810 comp rx=0 tx=0).stop 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 -- 172.21.15.7:0/282441636 shutdown_connections 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce77e50 0x560f4ce7a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce72ce0 0x560f4ce777e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/282441636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce72710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 -- 172.21.15.7:0/282441636 >> 172.21.15.7:0/282441636 conn(0x560f4ce6d860 msgr2=0x560f4ce6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 -- 172.21.15.7:0/282441636 shutdown_connections 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.952+0000 7f14ebfc9740 1 -- 172.21.15.7:0/282441636 wait complete. 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 -- start start 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 Processor -- start 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce7c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce7cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f4ce7ae00 con 0x560f4ce722f0 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560f4cd92310 con 0x560f4ce72ce0 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f4ce7af80 con 0x560f4ce77e50 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ebfc9740 10 client.0 _ll_register_callbacks cb 0x560f4cd862d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce7c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce7c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54672/0 (socket says 172.21.15.7:54672) 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3994022494 learned_addr learned my addr 172.21.15.7:0/3994022494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14da7fc640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce7cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e37fe640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 msgr2=0x560f4ce83d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce83d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3994022494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 msgr2=0x560f4ce7cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce7cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3994022494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4caea3f0 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce7c7d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009bd0 tx=0x7f14d4019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d20235f9d7f859 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.953+0000 7f14ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3994022494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4028020 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3994022494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14d4024620 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d40043d0 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3994022494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14d4033050 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 10 client.15912 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f4cbb47d0 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3994022494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d40043d0 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3994022494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f14d4018070 con 0x560f4ce722f0 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 10 client.15912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 20 client.15912 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ce88400 0x560f4ce8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 4 client.15912 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f4ce343e0 con 0x560f4ce88400 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.954+0000 7f14da7fc640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ce88400 0x560f4ce8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.955+0000 7f14da7fc640 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ce88400 0x560f4ce8a8a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=372a5f8ea1264693 server_cookie=18655476e783aa13 in_seq=0 out_seq=0 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.955+0000 7f14ca7fc640 10 client.15912 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.955+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3994022494 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560f4ce343e0 con 0x560f4ce88400 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.955+0000 7f14ca7fc640 10 client.15912 handle_command_reply: tid=0 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.955+0000 7f14ebfc9740 1 client.15912 shutdown 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 msgr2=0x560f4ce7c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce7c7d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009bd0 tx=0x7f14d4019ab0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 shutdown_connections 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ce88400 0x560f4ce8a8a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ca7fc640 0 client.15912 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce7cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3994022494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce7c7d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 >> 172.21.15.7:0/3994022494 conn(0x560f4ce6d860 msgr2=0x560f4ce6cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 shutdown_connections 2024-03-19T00:40:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3994022494 wait complete. 2024-03-19T00:40:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.956+0000 7f14ebfc9740 20 client.15912 trim_cache size 0 max 16384 2024-03-19T00:40:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 Processor -- start 2024-03-19T00:40:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 -- start start 2024-03-19T00:40:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce722f0 0x560f4ce711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 0x560f4ce8f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41acb00 con 0x560f4ce77e50 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41aa3d0 con 0x560f4ce72ce0 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41a8220 con 0x560f4ce722f0 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14e37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 0x560f4ce8f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14e37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 0x560f4ce8f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54674/0 (socket says 172.21.15.7:54674) 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14e37fe640 1 -- 172.21.15.7:0/785584532 learned_addr learned my addr 172.21.15.7:0/785584532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14da7fc640 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.969+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce722f0 0x560f4ce711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14e37fe640 1 -- 172.21.15.7:0/785584532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce722f0 msgr2=0x560f4ce711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14e37fe640 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce722f0 0x560f4ce711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14e37fe640 1 -- 172.21.15.7:0/785584532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 msgr2=0x560f4ce83d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14e37fe640 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14e37fe640 1 -- 172.21.15.7:0/785584532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4ce43880 con 0x560f4ce77e50 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14da7fc640 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14e37fe640 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 0x560f4ce8f160 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f14dc011420 tx=0x7f14dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec03627e7464f84b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14d8ff9640 1 -- 172.21.15.7:0/785584532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc02f070 con 0x560f4ce77e50 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14d8ff9640 1 -- 172.21.15.7:0/785584532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14dc01fd80 con 0x560f4ce77e50 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.970+0000 7f14d8ff9640 1 -- 172.21.15.7:0/785584532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc010e10 con 0x560f4ce77e50 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- 172.21.15.7:0/785584532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 msgr2=0x560f4ce8f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 0x560f4ce8f160 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f14dc011420 tx=0x7f14dc01a000 comp rx=0 tx=0).stop 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- 172.21.15.7:0/785584532 shutdown_connections 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce77e50 0x560f4ce8f160 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/785584532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce722f0 0x560f4ce711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- 172.21.15.7:0/785584532 >> 172.21.15.7:0/785584532 conn(0x560f4ce6d860 msgr2=0x560f4ce70ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- 172.21.15.7:0/785584532 shutdown_connections 2024-03-19T00:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- 172.21.15.7:0/785584532 wait complete. 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- start start 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 Processor -- start 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce836a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce81d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41a8890 con 0x560f4ce722f0 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41a9d00 con 0x560f4ce72ce0 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41a8f60 con 0x560f4ce77e50 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14ebfc9740 10 client.0 _ll_register_callbacks cb 0x7f14dc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce836a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce836a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54676/0 (socket says 172.21.15.7:54676) 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3940245699 learned_addr learned my addr 172.21.15.7:0/3940245699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14e37fe640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce81d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.971+0000 7f14da7fc640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 msgr2=0x560f4ce81d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce81d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3940245699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 msgr2=0x560f4ce83be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14e2ffd640 1 -- 172.21.15.7:0/3940245699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4ce6e2d0 con 0x560f4ce722f0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14e2ffd640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce836a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f14d4018680 tx=0x7f14d4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cd175d763df084 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3940245699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d403c060 con 0x560f4ce722f0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3940245699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14d4018870 con 0x560f4ce722f0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc01fd80 con 0x560f4ce722f0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3940245699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14d4043050 con 0x560f4ce722f0 2024-03-19T00:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ebfc9740 10 client.15918 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f4ceef110 con 0x560f4ce722f0 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.972+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3940245699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14dc01fd80 con 0x560f4ce722f0 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3940245699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f14d4018040 con 0x560f4ce722f0 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 10 client.15918 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 10 client.15918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 20 client.15918 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ceefaf0 0x560f4cef1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 4 client.15918 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f4cde39f0 con 0x560f4ceefaf0 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14da7fc640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ceefaf0 0x560f4cef1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ca7fc640 10 client.15918 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14da7fc640 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ceefaf0 0x560f4cef1f10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47522210a52d9a06 server_cookie=680647514853124 in_seq=0 out_seq=0 2024-03-19T00:40:06.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ca7fc640 1 -- 172.21.15.7:0/3940245699 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f4cde39f0 con 0x560f4ceefaf0 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ca7fc640 10 client.15918 handle_command_reply: tid=0 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 client.15918 shutdown 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 msgr2=0x560f4ce836a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce836a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f14d4018680 tx=0x7f14d4005da0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 shutdown_connections 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x560f4ceefaf0 0x560f4cef1f10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f4ce77e50 0x560f4ce81d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f4ce72ce0 0x560f4ce83be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ca7fc640 0 client.15918 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 --2- 172.21.15.7:0/3940245699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f4ce722f0 0x560f4ce836a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.973+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 >> 172.21.15.7:0/3940245699 conn(0x560f4ce6d860 msgr2=0x560f4ce71350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 shutdown_connections 2024-03-19T00:40:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14ebfc9740 1 -- 172.21.15.7:0/3940245699 wait complete. 2024-03-19T00:40:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14ebfc9740 20 client.15918 trim_cache size 0 max 16384 2024-03-19T00:40:06.982 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14d00a2800 msgr2=0x7f14d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14d00a2800 0x7f14d00a4cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x560f4ce73540 tx=0x7f14dc004a40 comp rx=0 tx=0).stop 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 msgr2=0x7f14e4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4197fd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14cc000c00 tx=0x7f14cc005dc0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 shutdown_connections 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14d00a2800 0x7f14d00a4cc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14e4107580 0x7f14e419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14e4104c50 0x7f14e4197fd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 --2- 172.21.15.7:0/2599269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4102320 0x7f14e4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 >> 172.21.15.7:0/2599269402 conn(0x7f14e40fbc90 msgr2=0x7f14e4102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:06.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 shutdown_connections 2024-03-19T00:40:06.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:06.974+0000 7f14e92ff640 1 -- 172.21.15.7:0/2599269402 wait complete. 2024-03-19T00:40:06.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 -- 172.21.15.7:0/2782732722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58101820 msgr2=0x7f4f58101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/2782732722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58101820 0x7f4f58101c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009860 tx=0x7f4f4802bb70 comp rx=0 tx=0).stop 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 -- 172.21.15.7:0/2782732722 shutdown_connections 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/2782732722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581033d0 0x7f4f58109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/2782732722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58102a10 0x7f4f58102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/2782732722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58101820 0x7f4f58101c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 -- 172.21.15.7:0/2782732722 >> 172.21.15.7:0/2782732722 conn(0x7f4f580fcd50 msgr2=0x7f4f580ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 -- 172.21.15.7:0/2782732722 shutdown_connections 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 -- 172.21.15.7:0/2782732722 wait complete. 2024-03-19T00:40:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.090+0000 7f4f5dd59640 1 Processor -- start 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 -- start start 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 0x7f4f58197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102a10 0x7f4f58197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f581033d0 0x7f4f5819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5810d320 con 0x7f4f58101820 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5810d1a0 con 0x7f4f581033d0 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f5dd59640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5810d4a0 con 0x7f4f58102a10 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 0x7f4f58197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102a10 0x7f4f58197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 0x7f4f58197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54700/0 (socket says 172.21.15.7:54700) 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f581033d0 0x7f4f5819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 -- 172.21.15.7:0/58014768 learned_addr learned my addr 172.21.15.7:0/58014768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 -- 172.21.15.7:0/58014768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102a10 msgr2=0x7f4f58197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102a10 0x7f4f58197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 -- 172.21.15.7:0/58014768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f581033d0 msgr2=0x7f4f5819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f581033d0 0x7f4f5819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 -- 172.21.15.7:0/58014768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5819f680 con 0x7f4f58101820 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f57fff640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f581033d0 0x7f4f5819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f577fe640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 0x7f4f58197ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f48005940 tx=0x7f4f480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.091+0000 7f4f54ff9640 1 -- 172.21.15.7:0/58014768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48041020 con 0x7f4f58101820 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.092+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5819f910 con 0x7f4f58101820 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.092+0000 7f4f54ff9640 1 -- 172.21.15.7:0/58014768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f480035f0 con 0x7f4f58101820 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.092+0000 7f4f54ff9640 1 -- 172.21.15.7:0/58014768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48039040 con 0x7f4f58101820 2024-03-19T00:40:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.092+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5819fe50 con 0x7f4f58101820 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f54ff9640 1 -- 172.21.15.7:0/58014768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f4f48044020 con 0x7f4f58101820 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 Processor -- start 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f54ff9640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2800 0x7f4f400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 -- start start 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd66b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f57fff640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2800 0x7f4f400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f54ff9640 1 -- 172.21.15.7:0/58014768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4f480f0740 con 0x7f4f58101820 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd6e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55730fcc64b0 con 0x55730fd6c260 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55730fc6ed70 con 0x55730fd670f0 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.094+0000 7f4f60a11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55730fcc2550 con 0x55730fd66700 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd66b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd6e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd66b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33020/0 (socket says 172.21.15.7:33020) 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd6e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54714/0 (socket says 172.21.15.7:54714) 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd6bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41524/0 (socket says 172.21.15.7:41524) 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 -- 172.21.15.7:0/3548930644 learned_addr learned my addr 172.21.15.7:0/3548930644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 -- 172.21.15.7:0/3548930644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 msgr2=0x55730fd6bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd6bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 -- 172.21.15.7:0/3548930644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 msgr2=0x55730fd6e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd6e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 -- 172.21.15.7:0/3548930644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730fc91210 con 0x55730fd66700 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f57fff640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2800 0x7f4f400a4cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f50005fd0 tx=0x7f4f5000fed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f577fe640 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd66b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f480bcd10 tx=0x7f4f48005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ad708ed8be7fef server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f3dffb640 1 -- 172.21.15.7:0/3548930644 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480bd480 con 0x55730fd66700 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f3dffb640 1 -- 172.21.15.7:0/3548930644 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f480bcd80 con 0x55730fd66700 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.095+0000 7f4f3dffb640 1 -- 172.21.15.7:0/3548930644 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480037a0 con 0x55730fd66700 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 -- 172.21.15.7:0/3548930644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 msgr2=0x55730fd66b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd66b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f480bcd10 tx=0x7f4f48005f90 comp rx=0 tx=0).stop 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 -- 172.21.15.7:0/3548930644 shutdown_connections 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd6e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd6bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3548930644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd66b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 -- 172.21.15.7:0/3548930644 >> 172.21.15.7:0/3548930644 conn(0x55730fd5d860 msgr2=0x55730fd5dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 -- 172.21.15.7:0/3548930644 shutdown_connections 2024-03-19T00:40:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 -- 172.21.15.7:0/3548930644 wait complete. 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 -- start start 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.096+0000 7f4f60a11740 1 Processor -- start 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd66700 0x55730fd70970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd670f0 0x55730fd70eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd77ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55730fd6efd0 con 0x55730fd6c260 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55730fc82310 con 0x55730fd66700 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55730fd6f150 con 0x55730fd670f0 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 10 client.0 _ll_register_callbacks cb 0x55730fc762d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd77ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd670f0 0x55730fd70eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd66700 0x55730fd70970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd77ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54716/0 (socket says 172.21.15.7:54716) 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd670f0 0x55730fd70eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:33030/0 (socket says 172.21.15.7:33030) 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 -- 172.21.15.7:0/2997432382 learned_addr learned my addr 172.21.15.7:0/2997432382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 -- 172.21.15.7:0/2997432382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd670f0 msgr2=0x55730fd70eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd670f0 0x55730fd70eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 -- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd66700 msgr2=0x55730fd70970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd66700 0x55730fd70970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 -- 172.21.15.7:0/2997432382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730f9da3f0 con 0x55730fd6c260 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f577fe640 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd66700 0x55730fd70970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f57fff640 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd77ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4f500042a0 tx=0x7f4f50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3ad9739a49faaa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f36ffd640 1 -- 172.21.15.7:0/2997432382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50012270 con 0x55730fd6c260 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f36ffd640 1 -- 172.21.15.7:0/2997432382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4f500209b0 con 0x55730fd6c260 2024-03-19T00:40:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.097+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f480bcd80 con 0x55730fd6c260 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f36ffd640 1 -- 172.21.15.7:0/2997432382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f50020bb0 con 0x55730fd6c260 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55730faa47d0 con 0x55730fd6c260 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f36ffd640 1 -- 172.21.15.7:0/2997432382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f480bcd80 con 0x55730fd6c260 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f36ffd640 1 -- 172.21.15.7:0/2997432382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4f5001f070 con 0x55730fd6c260 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 10 client.15930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 20 client.15930 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fd7c870 0x55730fd7ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 4 client.15930 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55730fd243e0 con 0x55730fd7c870 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.098+0000 7f4f577fe640 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fd7c870 0x55730fd7ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.099+0000 7f4f577fe640 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fd7c870 0x55730fd7ecc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a93b0f1aa3cfa7f0 server_cookie=b3269dc08be5e019 in_seq=0 out_seq=0 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.099+0000 7f4f36ffd640 10 client.15930 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.099+0000 7f4f36ffd640 1 -- 172.21.15.7:0/2997432382 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55730fd243e0 con 0x55730fd7c870 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.099+0000 7f4f36ffd640 10 client.15930 handle_command_reply: tid=0 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.099+0000 7f4f60a11740 1 client.15930 shutdown 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 msgr2=0x55730fd77ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd77ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4f500042a0 tx=0x7f4f50004ab0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 shutdown_connections 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fd7c870 0x55730fd7ecc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd77ef0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd670f0 0x55730fd70eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f36ffd640 0 client.15930 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 --2- 172.21.15.7:0/2997432382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd66700 0x55730fd70970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 >> 172.21.15.7:0/2997432382 conn(0x55730fd5d860 msgr2=0x55730fd5cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 shutdown_connections 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 1 -- 172.21.15.7:0/2997432382 wait complete. 2024-03-19T00:40:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.100+0000 7f4f60a11740 20 client.15930 trim_cache size 0 max 16384 2024-03-19T00:40:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 Processor -- start 2024-03-19T00:40:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 -- start start 2024-03-19T00:40:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 0x55730fd611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd6c260 0x55730fd83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581aa980 con 0x55730fd66700 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581a8250 con 0x55730fd670f0 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f60a11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581a60a0 con 0x55730fd6c260 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 0x55730fd611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 0x55730fd611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54726/0 (socket says 172.21.15.7:54726) 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f577fe640 1 -- 172.21.15.7:0/3729136772 learned_addr learned my addr 172.21.15.7:0/3729136772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f56ffd640 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.113+0000 7f4f57fff640 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd6c260 0x55730fd83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f577fe640 1 -- 172.21.15.7:0/3729136772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd6c260 msgr2=0x55730fd83580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f577fe640 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd6c260 0x55730fd83580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f577fe640 1 -- 172.21.15.7:0/3729136772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 msgr2=0x55730fd77eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f577fe640 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f577fe640 1 -- 172.21.15.7:0/3729136772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730fd33880 con 0x55730fd66700 2024-03-19T00:40:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f57fff640 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd6c260 0x55730fd83580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f577fe640 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 0x55730fd611d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4f480bca80 tx=0x7f4f48003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c8c427d2698496 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f3d7fa640 1 -- 172.21.15.7:0/3729136772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480baea0 con 0x55730fd66700 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f3d7fa640 1 -- 172.21.15.7:0/3729136772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f48087070 con 0x55730fd66700 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f3d7fa640 1 -- 172.21.15.7:0/3729136772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480037a0 con 0x55730fd66700 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 -- 172.21.15.7:0/3729136772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 msgr2=0x55730fd611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 0x55730fd611d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4f480bca80 tx=0x7f4f48003b10 comp rx=0 tx=0).stop 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 -- 172.21.15.7:0/3729136772 shutdown_connections 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd6c260 0x55730fd83580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 --2- 172.21.15.7:0/3729136772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd66700 0x55730fd611d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.114+0000 7f4f60a11740 1 -- 172.21.15.7:0/3729136772 >> 172.21.15.7:0/3729136772 conn(0x55730fd5d860 msgr2=0x55730fd60ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 -- 172.21.15.7:0/3729136772 shutdown_connections 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 -- 172.21.15.7:0/3729136772 wait complete. 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 -- start start 2024-03-19T00:40:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 Processor -- start 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581a6710 con 0x55730fd6c260 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581a7b80 con 0x55730fd670f0 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f581a6de0 con 0x55730fd66700 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f60a11740 10 client.0 _ll_register_callbacks cb 0x7f4f500098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd75d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54732/0 (socket says 172.21.15.7:54732) 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f57fff640 1 -- 172.21.15.7:0/1203348788 learned_addr learned my addr 172.21.15.7:0/1203348788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f56ffd640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.115+0000 7f4f577fe640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f57fff640 1 -- 172.21.15.7:0/1203348788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 msgr2=0x55730fd77770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f57fff640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd77770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f57fff640 1 -- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 msgr2=0x55730fd77cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f57fff640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f57fff640 1 -- 172.21.15.7:0/1203348788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730fd764c0 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f57fff640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd75d80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f50020ae0 tx=0x7f4f50020b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf0c34f3b8d4591 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f36ffd640 1 -- 172.21.15.7:0/1203348788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50004d20 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f36ffd640 1 -- 172.21.15.7:0/1203348788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4f5000f660 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f48087070 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f36ffd640 1 -- 172.21.15.7:0/1203348788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f50041050 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 10 client.15936 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55730fde35e0 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f36ffd640 1 -- 172.21.15.7:0/1203348788 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f48087070 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f36ffd640 1 -- 172.21.15.7:0/1203348788 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4f5001f030 con 0x55730fd6c260 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 10 client.15936 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 10 client.15936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 20 client.15936 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fde3ff0 0x55730fde6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 4 client.15936 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.116+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55730fcd39f0 con 0x55730fde3ff0 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f577fe640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fde3ff0 0x55730fde6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f577fe640 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fde3ff0 0x55730fde6410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cca356c2bcc112d server_cookie=cc914aff7b63ceae in_seq=0 out_seq=0 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f36ffd640 10 client.15936 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f36ffd640 1 -- 172.21.15.7:0/1203348788 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55730fcd39f0 con 0x55730fde3ff0 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f36ffd640 10 client.15936 handle_command_reply: tid=0 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 client.15936 shutdown 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 msgr2=0x55730fd75d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd75d80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f50020ae0 tx=0x7f4f50020b10 comp rx=0 tx=0).stop 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 shutdown_connections 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55730fde3ff0 0x55730fde6410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730fd6c260 0x55730fd75d80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55730fd670f0 0x55730fd77cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f36ffd640 0 client.15936 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 --2- 172.21.15.7:0/1203348788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730fd66700 0x55730fd77770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 >> 172.21.15.7:0/1203348788 conn(0x55730fd5d860 msgr2=0x55730fd61350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 shutdown_connections 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 1 -- 172.21.15.7:0/1203348788 wait complete. 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.117+0000 7f4f60a11740 20 client.15936 trim_cache size 0 max 16384 2024-03-19T00:40:07.125 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:40:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2800 msgr2=0x7f4f400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2800 0x7f4f400a4cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f50005fd0 tx=0x7f4f5000fed0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 msgr2=0x7f4f58197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 0x7f4f58197ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f48005940 tx=0x7f4f480042f0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 shutdown_connections 2024-03-19T00:40:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4f400a2800 0x7f4f400a4cc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f581033d0 0x7f4f5819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f58102a10 0x7f4f58197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 --2- 172.21.15.7:0/58014768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f58101820 0x7f4f58197ab0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 >> 172.21.15.7:0/58014768 conn(0x7f4f580fcd50 msgr2=0x7f4f58105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 shutdown_connections 2024-03-19T00:40:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.118+0000 7f4f5dd59640 1 -- 172.21.15.7:0/58014768 wait complete. 2024-03-19T00:40:07.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:40:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 -- 172.21.15.7:0/3683440310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb41022e0 msgr2=0x7f6eb4108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/3683440310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb41022e0 0x7f6eb4108b70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009ca0 tx=0x7f6ea402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 -- 172.21.15.7:0/3683440310 shutdown_connections 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/3683440310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb41022e0 0x7f6eb4108b70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/3683440310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb4101920 0x7f6eb4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/3683440310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 0x7f6eb4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 -- 172.21.15.7:0/3683440310 >> 172.21.15.7:0/3683440310 conn(0x7f6eb40fbc80 msgr2=0x7f6eb40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 -- 172.21.15.7:0/3683440310 shutdown_connections 2024-03-19T00:40:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.234+0000 7f6eb8af8640 1 -- 172.21.15.7:0/3683440310 wait complete. 2024-03-19T00:40:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 Processor -- start 2024-03-19T00:40:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 -- start start 2024-03-19T00:40:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 0x7f6eb41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 0x7f6eb4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41022e0 0x7f6eb419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410c240 con 0x7f6eb4101920 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410c0c0 con 0x7f6eb41022e0 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb8af8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410c3c0 con 0x7f6eb4100730 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 0x7f6eb4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 0x7f6eb4197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54748/0 (socket says 172.21.15.7:54748) 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/1433810095 learned_addr learned my addr 172.21.15.7:0/1433810095 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb259c640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 0x7f6eb41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.235+0000 7f6eb2d9d640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41022e0 0x7f6eb419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/1433810095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 msgr2=0x7f6eb41979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 0x7f6eb41979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/1433810095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41022e0 msgr2=0x7f6eb419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41022e0 0x7f6eb419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/1433810095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb419f600 con 0x7f6eb4101920 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb259c640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 0x7f6eb41979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 0x7f6eb4197f20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800ea60 tx=0x7f6ea800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6ea37fe640 1 -- 172.21.15.7:0/1433810095 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800cde0 con 0x7f6eb4101920 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb419f8f0 con 0x7f6eb4101920 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6ea37fe640 1 -- 172.21.15.7:0/1433810095 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea80035f0 con 0x7f6eb4101920 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6ea37fe640 1 -- 172.21.15.7:0/1433810095 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8010690 con 0x7f6eb4101920 2024-03-19T00:40:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.236+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb419fe40 con 0x7f6eb4101920 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.238+0000 7f6ea37fe640 1 -- 172.21.15.7:0/1433810095 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6ea80037a0 con 0x7f6eb4101920 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 Processor -- start 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 -- start start 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd11412f0 0x55cfd1141710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd11467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd1146e50 0x55cfd11492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cfd10a54b0 con 0x55cfd1146e50 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cfd104dd70 con 0x55cfd1141ce0 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cfd10a1550 con 0x55cfd11412f0 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ea37fe640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6e9c0a2800 0x7f6e9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd1146e50 0x55cfd11492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd11467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd11467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41576/0 (socket says 172.21.15.7:41576) 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb259c640 1 -- 172.21.15.7:0/853299432 learned_addr learned my addr 172.21.15.7:0/853299432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb2d9d640 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd11412f0 0x55cfd1141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb2d9d640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6e9c0a2800 0x7f6e9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6ea37fe640 1 -- 172.21.15.7:0/1433810095 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6ea80cf0b0 con 0x7f6eb4101920 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb259c640 1 -- 172.21.15.7:0/853299432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd11412f0 msgr2=0x55cfd1141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb259c640 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd11412f0 0x55cfd1141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.239+0000 7f6eb259c640 1 -- 172.21.15.7:0/853299432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd1146e50 msgr2=0x55cfd11492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6eb259c640 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd1146e50 0x55cfd11492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6eb259c640 1 -- 172.21.15.7:0/853299432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd1070210 con 0x55cfd1141ce0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6eb2d9d640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6e9c0a2800 0x7f6e9c0a4cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4006010 tx=0x7f6ea40324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6eb259c640 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd11467e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0114c0 tx=0x7f6eac011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9496d95b24d6c943 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6ea0ff9640 1 -- 172.21.15.7:0/853299432 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004020 con 0x55cfd1141ce0 2024-03-19T00:40:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6ea0ff9640 1 -- 172.21.15.7:0/853299432 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eac0054d0 con 0x55cfd1141ce0 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.240+0000 7f6ea0ff9640 1 -- 172.21.15.7:0/853299432 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004c30 con 0x55cfd1141ce0 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/853299432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 msgr2=0x55cfd11467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd11467e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0114c0 tx=0x7f6eac011980 comp rx=0 tx=0).stop 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/853299432 shutdown_connections 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd1146e50 0x55cfd11492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd11467e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/853299432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd11412f0 0x55cfd1141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/853299432 >> 172.21.15.7:0/853299432 conn(0x55cfd113c860 msgr2=0x55cfd113cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/853299432 shutdown_connections 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/853299432 wait complete. 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- start start 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 Processor -- start 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd114b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1146e50 0x55cfd1152d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cfd1149e00 con 0x55cfd11412f0 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cfd1061310 con 0x55cfd1141ce0 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.241+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cfd1149f80 con 0x55cfd1146e50 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6ebb7c2740 10 client.0 _ll_register_callbacks cb 0x55cfd10552d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd114b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd114b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54762/0 (socket says 172.21.15.7:54762) 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 -- 172.21.15.7:0/3723711322 learned_addr learned my addr 172.21.15.7:0/3723711322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb2d9d640 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1146e50 0x55cfd1152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 -- 172.21.15.7:0/3723711322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1146e50 msgr2=0x55cfd1152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1146e50 0x55cfd1152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 -- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 msgr2=0x55cfd114bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 -- 172.21.15.7:0/3723711322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd0db93f0 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6eb259c640 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd114b7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009530 tx=0x7f6eac00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b7b9c8d92de1e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6e99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6e99ffb640 1 -- 172.21.15.7:0/3723711322 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac028020 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6e99ffb640 1 -- 172.21.15.7:0/3723711322 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6eac004970 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eac0054d0 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6e99ffb640 1 -- 172.21.15.7:0/3723711322 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6eac033050 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6ebb7c2740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.242+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfd0e837d0 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6e99ffb640 1 -- 172.21.15.7:0/3723711322 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eac0054d0 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6e99ffb640 1 -- 172.21.15.7:0/3723711322 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6eac023470 con 0x55cfd11412f0 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6ebb7c2740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6ebb7c2740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6ebb7c2740 20 client.15948 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd11574c0 0x55cfd1159960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6ebb7c2740 4 client.15948 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfd11033e0 con 0x55cfd11574c0 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd11574c0 0x55cfd1159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6e99ffb640 10 client.15948 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.243+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd11574c0 0x55cfd1159960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21269896556a218 server_cookie=d79ce2ae1b5bfdf3 in_seq=0 out_seq=0 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6e99ffb640 1 -- 172.21.15.7:0/3723711322 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cfd11033e0 con 0x55cfd11574c0 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6e99ffb640 10 client.15948 handle_command_reply: tid=0 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 client.15948 shutdown 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 msgr2=0x55cfd114b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd114b7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009530 tx=0x7f6eac00fcc0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 shutdown_connections 2024-03-19T00:40:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1146e50 0x55cfd1152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd11574c0 0x55cfd1159960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/3723711322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd114b7e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 >> 172.21.15.7:0/3723711322 conn(0x55cfd113c860 msgr2=0x55cfd113bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.244+0000 7f6e99ffb640 0 client.15948 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.245+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 shutdown_connections 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.245+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/3723711322 wait complete. 2024-03-19T00:40:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.245+0000 7f6ebb7c2740 20 client.15948 trim_cache size 0 max 16384 2024-03-19T00:40:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.258+0000 7f6ebb7c2740 1 Processor -- start 2024-03-19T00:40:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.258+0000 7f6ebb7c2740 1 -- start start 2024-03-19T00:40:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.258+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd113c860 0x55cfd11401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.258+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1152d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.258+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.258+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41acb20 con 0x55cfd11412f0 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41aa3f0 con 0x55cfd1141ce0 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a8240 con 0x55cfd113c860 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54778/0 (socket says 172.21.15.7:54778) 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd113c860 0x55cfd11401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/4052351659 learned_addr learned my addr 172.21.15.7:0/4052351659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb2d9d640 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 -- 172.21.15.7:0/4052351659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 msgr2=0x55cfd114ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 -- 172.21.15.7:0/4052351659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 msgr2=0x55cfd1152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 -- 172.21.15.7:0/4052351659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd1112880 con 0x55cfd113c860 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6eb259c640 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd113c860 0x55cfd11401d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0238e0 tx=0x7f6eac011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec20d2388d13722d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6e9bfff640 1 -- 172.21.15.7:0/4052351659 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac028040 con 0x55cfd113c860 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6e9bfff640 1 -- 172.21.15.7:0/4052351659 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eac00fd30 con 0x55cfd113c860 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.259+0000 7f6e9bfff640 1 -- 172.21.15.7:0/4052351659 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac01d040 con 0x55cfd113c860 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/4052351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd113c860 msgr2=0x55cfd11401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd113c860 0x55cfd11401d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0238e0 tx=0x7f6eac011980 comp rx=0 tx=0).stop 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/4052351659 shutdown_connections 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd1141ce0 0x55cfd114ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/4052351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd113c860 0x55cfd11401d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/4052351659 >> 172.21.15.7:0/4052351659 conn(0x55cfd1142ac0 msgr2=0x55cfd113fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/4052351659 shutdown_connections 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/4052351659 wait complete. 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 -- start start 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.260+0000 7f6ebb7c2740 1 Processor -- start 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd113c860 0x55cfd11501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1150720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1141ce0 0x55cfd1151730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a88b0 con 0x55cfd11412f0 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a9d20 con 0x55cfd113c860 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb41a8f80 con 0x55cfd1141ce0 2024-03-19T00:40:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 10 client.0 _ll_register_callbacks cb 0x7f6ea8097210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd113c860 0x55cfd11501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd113c860 0x55cfd11501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41596/0 (socket says 172.21.15.7:41596) 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1141ce0 0x55cfd1151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb259c640 1 -- 172.21.15.7:0/2417879565 learned_addr learned my addr 172.21.15.7:0/2417879565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/2417879565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1141ce0 msgr2=0x55cfd1151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1141ce0 0x55cfd1151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd113c860 msgr2=0x55cfd11501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd113c860 0x55cfd11501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 -- 172.21.15.7:0/2417879565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd114ac00 con 0x55cfd11412f0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6eb1d9b640 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1150720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6ea809b390 tx=0x7f6ea800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958e2bdb7b8c64e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6e99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6e99ffb640 1 -- 172.21.15.7:0/2417879565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea809bd90 con 0x55cfd11412f0 2024-03-19T00:40:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6e99ffb640 1 -- 172.21.15.7:0/2417879565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6ea8097d00 con 0x55cfd11412f0 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eac00fd30 con 0x55cfd11412f0 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6e99ffb640 1 -- 172.21.15.7:0/2417879565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ea80a3e30 con 0x55cfd11412f0 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 10 client.15954 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.261+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfd1152f20 con 0x55cfd11412f0 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6e99ffb640 1 -- 172.21.15.7:0/2417879565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eac00fd30 con 0x55cfd11412f0 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6e99ffb640 1 -- 172.21.15.7:0/2417879565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6ea8099c50 con 0x55cfd11412f0 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6ebb7c2740 10 client.15954 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6ebb7c2740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6ebb7c2740 20 client.15954 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd1146e50 0x55cfd11bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6ebb7c2740 4 client.15954 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfd10b29f0 con 0x55cfd1146e50 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6eb259c640 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd1146e50 0x55cfd11bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6e99ffb640 10 client.15954 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.262+0000 7f6eb259c640 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd1146e50 0x55cfd11bff30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b759ec5a04be43 server_cookie=7d2c7d3372ef5407 in_seq=0 out_seq=0 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6e99ffb640 1 -- 172.21.15.7:0/2417879565 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfd10b29f0 con 0x55cfd1146e50 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6e99ffb640 10 client.15954 handle_command_reply: tid=0 2024-03-19T00:40:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 client.15954 shutdown 2024-03-19T00:40:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 msgr2=0x55cfd1150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1150720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6ea809b390 tx=0x7f6ea800ef90 comp rx=0 tx=0).stop 2024-03-19T00:40:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 shutdown_connections 2024-03-19T00:40:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfd1141ce0 0x55cfd1151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cfd1146e50 0x55cfd11bff30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfd11412f0 0x55cfd1150720 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 --2- 172.21.15.7:0/2417879565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfd113c860 0x55cfd11501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6e99ffb640 0 client.15954 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 >> 172.21.15.7:0/2417879565 conn(0x55cfd1142ac0 msgr2=0x55cfd1140350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 shutdown_connections 2024-03-19T00:40:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 1 -- 172.21.15.7:0/2417879565 wait complete. 2024-03-19T00:40:07.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6ebb7c2740 20 client.15954 trim_cache size 0 max 16384 2024-03-19T00:40:07.274 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:40:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6e9c0a2800 msgr2=0x7f6e9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6e9c0a2800 0x7f6e9c0a4cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4006010 tx=0x7f6ea40324f0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 msgr2=0x7f6eb4197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 0x7f6eb4197f20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800ea60 tx=0x7f6ea800ef20 comp rx=0 tx=0).stop 2024-03-19T00:40:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 shutdown_connections 2024-03-19T00:40:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6e9c0a2800 0x7f6e9c0a4cc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb41022e0 0x7f6eb419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eb4101920 0x7f6eb4197f20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 --2- 172.21.15.7:0/1433810095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6eb4100730 0x7f6eb41979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.263+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 >> 172.21.15.7:0/1433810095 conn(0x7f6eb40fbc80 msgr2=0x7f6eb4069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.264+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 shutdown_connections 2024-03-19T00:40:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:07.264+0000 7f6eb8af8640 1 -- 172.21.15.7:0/1433810095 wait complete. 2024-03-19T00:40:07.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:07 smithi204 ceph-mon[66572]: pgmap v416: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:07.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:07 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1892399876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:40:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:07 smithi044 ceph-mon[66366]: pgmap v416: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:07 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1892399876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:40:07.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:07 smithi007 ceph-mon[62854]: pgmap v416: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:07.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:07 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1892399876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:40:08.279 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:40:08.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:40:08.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 -- 172.21.15.7:0/2553182067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c101930 msgr2=0x7f368c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 --2- 172.21.15.7:0/2553182067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c101930 0x7f368c101db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3674009900 tx=0x7f367402bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 -- 172.21.15.7:0/2553182067 shutdown_connections 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 --2- 172.21.15.7:0/2553182067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c1022f0 0x7f368c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 --2- 172.21.15.7:0/2553182067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c101930 0x7f368c101db0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 --2- 172.21.15.7:0/2553182067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f368c100740 0x7f368c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 -- 172.21.15.7:0/2553182067 >> 172.21.15.7:0/2553182067 conn(0x7f368c0fbcb0 msgr2=0x7f368c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 -- 172.21.15.7:0/2553182067 shutdown_connections 2024-03-19T00:40:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.398+0000 7f36913b3640 1 -- 172.21.15.7:0/2553182067 wait complete. 2024-03-19T00:40:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 Processor -- start 2024-03-19T00:40:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 -- start start 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 0x7f368c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f368c101930 0x7f368c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c1022f0 0x7f368c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f368c10d390 con 0x7f368c1022f0 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f368c10d210 con 0x7f368c100740 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f36913b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f368c10d510 con 0x7f368c101930 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c1022f0 0x7f368c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 0x7f368c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 0x7f368c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41610/0 (socket says 172.21.15.7:41610) 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c1022f0 0x7f368c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54812/0 (socket says 172.21.15.7:54812) 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 -- 172.21.15.7:0/1722416514 learned_addr learned my addr 172.21.15.7:0/1722416514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f3683fff640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f368c101930 0x7f368c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 -- 172.21.15.7:0/1722416514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f368c101930 msgr2=0x7f368c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f368c101930 0x7f368c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 -- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c1022f0 msgr2=0x7f368c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c1022f0 0x7f368c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.399+0000 7f368affd640 1 -- 172.21.15.7:0/1722416514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c19f710 con 0x7f368c100740 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.400+0000 7f368affd640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 0x7f368c197af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f367c00e9c0 tx=0x7f367c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.400+0000 7f3688ff9640 1 -- 172.21.15.7:0/1722416514 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c00ccc0 con 0x7f368c100740 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.400+0000 7f3688ff9640 1 -- 172.21.15.7:0/1722416514 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f367c0035f0 con 0x7f368c100740 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.400+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c19fa00 con 0x7f368c100740 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.400+0000 7f3688ff9640 1 -- 172.21.15.7:0/1722416514 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c00ce60 con 0x7f368c100740 2024-03-19T00:40:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.400+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c19ff20 con 0x7f368c100740 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.402+0000 7f3688ff9640 1 -- 172.21.15.7:0/1722416514 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f367c0037a0 con 0x7f368c100740 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.402+0000 7f3693f99740 1 Processor -- start 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.402+0000 7f3693f99740 1 -- start start 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3693f99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3693f99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3693f99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3693f99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56483c100160 con 0x56483c19b3a0 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3693f99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56483c0a8e10 con 0x56483c19bd90 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3688ff9640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36780a2880 0x7f36780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3693f99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56483c0fc200 con 0x56483c1a0f00 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f368affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c19b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54826/0 (socket says 172.21.15.7:54826) 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f368b7fe640 1 -- 172.21.15.7:0/3038114259 learned_addr learned my addr 172.21.15.7:0/3038114259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f368b7fe640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36780a2880 0x7f36780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3688ff9640 1 -- 172.21.15.7:0/1722416514 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f367c0cf250 con 0x7f368c100740 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 -- 172.21.15.7:0/3038114259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 msgr2=0x56483c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 -- 172.21.15.7:0/3038114259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 msgr2=0x56483c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 -- 172.21.15.7:0/3038114259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483bedc7e0 con 0x56483c1a0f00 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f3683fff640 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1a3390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87b4b9d3b5574861 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.403+0000 7f368b7fe640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36780a2880 0x7f36780a4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3684005c40 tx=0x7f3684009c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f36817fa640 1 -- 172.21.15.7:0/3038114259 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674045020 con 0x56483c1a0f00 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f36817fa640 1 -- 172.21.15.7:0/3038114259 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3674004040 con 0x56483c1a0f00 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f36817fa640 1 -- 172.21.15.7:0/3038114259 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36740054f0 con 0x56483c1a0f00 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f3693f99740 1 -- 172.21.15.7:0/3038114259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 msgr2=0x56483c1a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f3693f99740 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1a3390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674009560 comp rx=0 tx=0).stop 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f3693f99740 1 -- 172.21.15.7:0/3038114259 shutdown_connections 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f3693f99740 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1a3390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f3693f99740 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.404+0000 7f3693f99740 1 --2- 172.21.15.7:0/3038114259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- 172.21.15.7:0/3038114259 >> 172.21.15.7:0/3038114259 conn(0x56483c196910 msgr2=0x56483c196d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- 172.21.15.7:0/3038114259 shutdown_connections 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- 172.21.15.7:0/3038114259 wait complete. 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- start start 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 Processor -- start 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 0x56483c1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 0x56483c1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56483c1a3e50 con 0x56483c19bd90 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56483c0bc1a0 con 0x56483c19b3a0 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56483c1a3fd0 con 0x56483c1a0f00 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3693f99740 10 client.0 _ll_register_callbacks cb 0x56483be13c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3683fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 0x56483c1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f368affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 0x56483c1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3683fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 0x56483c1a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54828/0 (socket says 172.21.15.7:54828) 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f3683fff640 1 -- 172.21.15.7:0/1008792823 learned_addr learned my addr 172.21.15.7:0/1008792823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f368affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 0x56483c1a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:41618/0 (socket says 172.21.15.7:41618) 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.405+0000 7f368b7fe640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3683fff640 1 -- 172.21.15.7:0/1008792823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 msgr2=0x56483c1acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3683fff640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3683fff640 1 -- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 msgr2=0x56483c1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3683fff640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 0x56483c1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3683fff640 1 -- 172.21.15.7:0/1008792823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483bf3cf70 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f368affd640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 0x56483c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3683fff640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 0x56483c1a5c40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3674004620 tx=0x7f3674005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68331308c1828d15 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f36727fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f36727fc640 1 -- 172.21.15.7:0/1008792823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367404e030 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f36727fc640 1 -- 172.21.15.7:0/1008792823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f367402caf0 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3674004040 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f36727fc640 1 -- 172.21.15.7:0/1008792823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3674058050 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3693f99740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56483bdd4170 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f36727fc640 1 -- 172.21.15.7:0/1008792823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3674004040 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f36727fc640 1 -- 172.21.15.7:0/1008792823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f36740038c0 con 0x56483c19bd90 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.406+0000 7f3693f99740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f3693f99740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f3693f99740 20 client.15966 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f3693f99740 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c1b11c0 0x56483c1b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f3693f99740 4 client.15966 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56483c10d6a0 con 0x56483c1b11c0 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f368affd640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c1b11c0 0x56483c1b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f368affd640 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c1b11c0 0x56483c1b3640 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3debd01011a35916 server_cookie=1f0567513b6ef388 in_seq=0 out_seq=0 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.407+0000 7f36727fc640 10 client.15966 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f36727fc640 1 -- 172.21.15.7:0/1008792823 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56483c10d6a0 con 0x56483c1b11c0 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f36727fc640 10 client.15966 handle_command_reply: tid=0 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 client.15966 shutdown 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 msgr2=0x56483c1a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 0x56483c1a5c40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3674004620 tx=0x7f3674005520 comp rx=0 tx=0).stop 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 shutdown_connections 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c1b11c0 0x56483c1b3640 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19bd90 0x56483c1a5c40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 --2- 172.21.15.7:0/1008792823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19b3a0 0x56483c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f36727fc640 0 client.15966 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 >> 172.21.15.7:0/1008792823 conn(0x56483c196910 msgr2=0x56483c195b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 shutdown_connections 2024-03-19T00:40:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 1 -- 172.21.15.7:0/1008792823 wait complete. 2024-03-19T00:40:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.408+0000 7f3693f99740 20 client.15966 trim_cache size 0 max 16384 2024-03-19T00:40:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 Processor -- start 2024-03-19T00:40:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 -- start start 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c19b3a0 0x56483c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 0x56483c1b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1aca00 con 0x56483c1a0f00 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1aa2d0 con 0x56483c19bd90 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3693f99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1a8120 con 0x56483c19b3a0 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 0x56483c1b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 0x56483c1b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54832/0 (socket says 172.21.15.7:54832) 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 -- 172.21.15.7:0/3210270961 learned_addr learned my addr 172.21.15.7:0/3210270961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3683fff640 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368affd640 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c19b3a0 0x56483c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 -- 172.21.15.7:0/3210270961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c19b3a0 msgr2=0x56483c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c19b3a0 0x56483c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 -- 172.21.15.7:0/3210270961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 msgr2=0x56483c1acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f368b7fe640 1 -- 172.21.15.7:0/3210270961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483c197680 con 0x56483c1a0f00 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.426+0000 7f3683fff640 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f368b7fe640 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 0x56483c1b7f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3684009850 tx=0x7f3684016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1fcccf9b011baf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3680ff9640 1 -- 172.21.15.7:0/3210270961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368401f500 con 0x56483c1a0f00 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3680ff9640 1 -- 172.21.15.7:0/3210270961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f368401fb50 con 0x56483c1a0f00 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3680ff9640 1 -- 172.21.15.7:0/3210270961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684029a00 con 0x56483c1a0f00 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 -- 172.21.15.7:0/3210270961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 msgr2=0x56483c1b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 0x56483c1b7f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3684009850 tx=0x7f3684016000 comp rx=0 tx=0).stop 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 -- 172.21.15.7:0/3210270961 shutdown_connections 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c1a0f00 0x56483c1b7f40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 --2- 172.21.15.7:0/3210270961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c19b3a0 0x56483c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.427+0000 7f3693f99740 1 -- 172.21.15.7:0/3210270961 >> 172.21.15.7:0/3210270961 conn(0x56483c196910 msgr2=0x56483c199b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 -- 172.21.15.7:0/3210270961 shutdown_connections 2024-03-19T00:40:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 -- 172.21.15.7:0/3210270961 wait complete. 2024-03-19T00:40:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 -- start start 2024-03-19T00:40:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 Processor -- start 2024-03-19T00:40:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c1ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1a8790 con 0x56483c19b3a0 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1a9c00 con 0x56483c19bd90 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1a8e60 con 0x56483c1a0f00 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3693f99740 10 client.0 _ll_register_callbacks cb 0x7f3684011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f368affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c1ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f368affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c1ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54848/0 (socket says 172.21.15.7:54848) 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f368affd640 1 -- 172.21.15.7:0/1091148051 learned_addr learned my addr 172.21.15.7:0/1091148051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f3683fff640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f368b7fe640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.428+0000 7f368affd640 1 -- 172.21.15.7:0/1091148051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 msgr2=0x56483c1aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f368affd640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f368affd640 1 -- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 msgr2=0x56483c1aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f368affd640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f368affd640 1 -- 172.21.15.7:0/1091148051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483c1ab250 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f368affd640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c1ac550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f367c009fd0 tx=0x7f367c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b973f663f2be036 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f36727fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f36727fc640 1 -- 172.21.15.7:0/1091148051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0b3070 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f36727fc640 1 -- 172.21.15.7:0/1091148051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f367c0860b0 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f368401fb50 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f36727fc640 1 -- 172.21.15.7:0/1091148051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f367c09bf30 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 10 client.15972 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56483c218050 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f36727fc640 1 -- 172.21.15.7:0/1091148051 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f368401fb50 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f36727fc640 1 -- 172.21.15.7:0/1091148051 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f367c0a3b40 con 0x56483c19b3a0 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 10 client.15972 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 20 client.15972 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c2189b0 0x56483c21add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 4 client.15972 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:40:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.429+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56483c1079f0 con 0x56483c2189b0 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.430+0000 7f3683fff640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c2189b0 0x56483c21add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.430+0000 7f36727fc640 10 client.15972 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.430+0000 7f3683fff640 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c2189b0 0x56483c21add0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5789c8cf4f979adf server_cookie=e00e3d960da54752 in_seq=0 out_seq=0 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f36727fc640 1 -- 172.21.15.7:0/1091148051 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56483c1079f0 con 0x56483c2189b0 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f36727fc640 10 client.15972 handle_command_reply: tid=0 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 client.15972 shutdown 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 msgr2=0x56483c1ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c1ac550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f367c009fd0 tx=0x7f367c003d00 comp rx=0 tx=0).stop 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 shutdown_connections 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56483c1a0f00 0x56483c1aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483c19bd90 0x56483c1aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56483c2189b0 0x56483c21add0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 --2- 172.21.15.7:0/1091148051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483c19b3a0 0x56483c1ac550 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f36727fc640 0 client.15972 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 >> 172.21.15.7:0/1091148051 conn(0x56483c196910 msgr2=0x56483c19a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 shutdown_connections 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 1 -- 172.21.15.7:0/1091148051 wait complete. 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.431+0000 7f3693f99740 20 client.15972 trim_cache size 0 max 16384 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "b78f1ee7-4662-4ad5-be93-71503a308cfe", 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:40:08.438 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36780a2880 msgr2=0x7f36780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36780a2880 0x7f36780a4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3684005c40 tx=0x7f3684009c90 comp rx=0 tx=0).stop 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 msgr2=0x7f368c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 0x7f368c197af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f367c00e9c0 tx=0x7f367c00edc0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 shutdown_connections 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f36780a2880 0x7f36780a4d40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c1022f0 0x7f368c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f368c101930 0x7f368c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 --2- 172.21.15.7:0/1722416514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f368c100740 0x7f368c197af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 >> 172.21.15.7:0/1722416514 conn(0x7f368c0fbcb0 msgr2=0x7f368c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 shutdown_connections 2024-03-19T00:40:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.432+0000 7f36913b3640 1 -- 172.21.15.7:0/1722416514 wait complete. 2024-03-19T00:40:08.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:40:08.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 -- 172.21.15.7:0/1959202305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 msgr2=0x7fce90101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 --2- 172.21.15.7:0/1959202305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce90101db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fce78009900 tx=0x7fce7802bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 -- 172.21.15.7:0/1959202305 shutdown_connections 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 --2- 172.21.15.7:0/1959202305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce901022f0 0x7fce90108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 --2- 172.21.15.7:0/1959202305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce90101db0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 --2- 172.21.15.7:0/1959202305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90100740 0x7fce90100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 -- 172.21.15.7:0/1959202305 >> 172.21.15.7:0/1959202305 conn(0x7fce900fbcd0 msgr2=0x7fce900fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 -- 172.21.15.7:0/1959202305 shutdown_connections 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 -- 172.21.15.7:0/1959202305 wait complete. 2024-03-19T00:40:08.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 Processor -- start 2024-03-19T00:40:08.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.548+0000 7fce953e6640 1 -- start start 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90100740 0x7fce90199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce9019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce901022f0 0x7fce901a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010d390 con 0x7fce90101930 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010d210 con 0x7fce901022f0 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010d510 con 0x7fce90100740 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce9019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce9019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54858/0 (socket says 172.21.15.7:54858) 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90100740 0x7fce90199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 -- 172.21.15.7:0/1343335864 learned_addr learned my addr 172.21.15.7:0/1343335864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce901022f0 0x7fce901a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 -- 172.21.15.7:0/1343335864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90100740 msgr2=0x7fce90199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90100740 0x7fce90199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 -- 172.21.15.7:0/1343335864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce901022f0 msgr2=0x7fce901a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce901022f0 0x7fce901a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 -- 172.21.15.7:0/1343335864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce901a18d0 con 0x7fce90101930 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce9019a220 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fce78006010 tx=0x7fce78005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce87fff640 1 -- 172.21.15.7:0/1343335864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78043020 con 0x7fce90101930 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce87fff640 1 -- 172.21.15.7:0/1343335864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce780043d0 con 0x7fce90101930 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a1b60 con 0x7fce90101930 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.549+0000 7fce87fff640 1 -- 172.21.15.7:0/1343335864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78003a80 con 0x7fce90101930 2024-03-19T00:40:08.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.550+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a20b0 con 0x7fce90101930 2024-03-19T00:40:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.552+0000 7fce87fff640 1 -- 172.21.15.7:0/1343335864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fce78046020 con 0x7fce90101930 2024-03-19T00:40:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.552+0000 7fce97fd0740 1 Processor -- start 2024-03-19T00:40:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.552+0000 7fce97fd0740 1 -- start start 2024-03-19T00:40:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.552+0000 7fce87fff640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce7c0a2800 0x7fce7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.552+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a4f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.552+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f3440 0x56243a4f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f85b0 0x56243a4faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce97fd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56243a457500 con 0x56243a4f2a50 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce97fd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56243a3ffbe0 con 0x56243a4f85b0 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce97fd0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56243a4535a0 con 0x56243a4f3440 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce7c0a2800 0x7fce7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a4f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce87fff640 1 -- 172.21.15.7:0/1343335864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fce780f0f90 con 0x7fce90101930 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a4f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54870/0 (socket says 172.21.15.7:54870) 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f3440 0x56243a4f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 -- 172.21.15.7:0/534526820 learned_addr learned my addr 172.21.15.7:0/534526820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f85b0 0x56243a4faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 -- 172.21.15.7:0/534526820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f3440 msgr2=0x56243a4f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f3440 0x56243a4f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 -- 172.21.15.7:0/534526820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f85b0 msgr2=0x56243a4faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f85b0 0x56243a4faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 -- 172.21.15.7:0/534526820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243a421000 con 0x56243a4f2a50 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce7c0a2800 0x7fce7c0a4cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fce80005fd0 tx=0x7fce80009e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce8effd640 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a4f2e70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fce88009c60 tx=0x7fce88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b88a66742dd382 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce857fa640 1 -- 172.21.15.7:0/534526820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88004220 con 0x56243a4f2a50 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce857fa640 1 -- 172.21.15.7:0/534526820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce880043c0 con 0x56243a4f2a50 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.553+0000 7fce857fa640 1 -- 172.21.15.7:0/534526820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88004c50 con 0x56243a4f2a50 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 -- 172.21.15.7:0/534526820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 msgr2=0x56243a4f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a4f2e70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fce88009c60 tx=0x7fce88011810 comp rx=0 tx=0).stop 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 -- 172.21.15.7:0/534526820 shutdown_connections 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f85b0 0x56243a4faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f3440 0x56243a4f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 --2- 172.21.15.7:0/534526820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a4f2e70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 -- 172.21.15.7:0/534526820 >> 172.21.15.7:0/534526820 conn(0x56243a4edfc0 msgr2=0x56243a4ee3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 -- 172.21.15.7:0/534526820 shutdown_connections 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 -- 172.21.15.7:0/534526820 wait complete. 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.554+0000 7fce97fd0740 1 -- start start 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 Processor -- start 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f2a50 0x56243a4fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a4fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 0x56243a5043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56243a4fb560 con 0x56243a4f85b0 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56243a413160 con 0x56243a4f3440 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56243a4fb6e0 con 0x56243a4f2a50 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce97fd0740 10 client.0 _ll_register_callbacks cb 0x56243a44fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 0x56243a5043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 0x56243a5043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54878/0 (socket says 172.21.15.7:54878) 2024-03-19T00:40:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 -- 172.21.15.7:0/2679068815 learned_addr learned my addr 172.21.15.7:0/2679068815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a4fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8effd640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f2a50 0x56243a4fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 -- 172.21.15.7:0/2679068815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f2a50 msgr2=0x56243a4fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f2a50 0x56243a4fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 -- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 msgr2=0x56243a4fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a4fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 -- 172.21.15.7:0/2679068815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243a2357d0 con 0x56243a4f85b0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 0x56243a5043a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fce80013790 tx=0x7fce8001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaefe550abdc3f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.555+0000 7fce767fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce767fc640 1 -- 172.21.15.7:0/2679068815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80004670 con 0x56243a4f85b0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce767fc640 1 -- 172.21.15.7:0/2679068815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fce8002a430 con 0x56243a4f85b0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce880043c0 con 0x56243a4f85b0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce767fc640 1 -- 172.21.15.7:0/2679068815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce80032050 con 0x56243a4f85b0 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56243a16b3f0 con 0x56243a4f85b0 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce767fc640 1 -- 172.21.15.7:0/2679068815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce880043c0 con 0x56243a4f85b0 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce767fc640 1 -- 172.21.15.7:0/2679068815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fce80020050 con 0x56243a4f85b0 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 20 client.15984 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a508c60 0x56243a50b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 4 client.15984 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56243a464a40 con 0x56243a508c60 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.556+0000 7fce8effd640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a508c60 0x56243a50b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.557+0000 7fce767fc640 10 client.15984 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.557+0000 7fce8effd640 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a508c60 0x56243a50b0c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6283ccfb6bff5ed9 server_cookie=856788d98af72fc1 in_seq=0 out_seq=0 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.557+0000 7fce767fc640 1 -- 172.21.15.7:0/2679068815 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56243a464a40 con 0x56243a508c60 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.557+0000 7fce767fc640 10 client.15984 handle_command_reply: tid=0 2024-03-19T00:40:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.557+0000 7fce97fd0740 1 client.15984 shutdown 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 msgr2=0x56243a5043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 0x56243a5043a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fce80013790 tx=0x7fce8001d000 comp rx=0 tx=0).stop 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 shutdown_connections 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a508c60 0x56243a50b0c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f85b0 0x56243a5043a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a4fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce767fc640 0 client.15984 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 --2- 172.21.15.7:0/2679068815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f2a50 0x56243a4fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 >> 172.21.15.7:0/2679068815 conn(0x56243a4edfc0 msgr2=0x56243a4ed1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 shutdown_connections 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 1 -- 172.21.15.7:0/2679068815 wait complete. 2024-03-19T00:40:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.558+0000 7fce97fd0740 20 client.15984 trim_cache size 0 max 16384 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 Processor -- start 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 -- start start 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f2a50 0x56243a4ed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 0x56243a504360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a50f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce901aed40 con 0x56243a4f3440 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.575+0000 7fce97fd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ac610 con 0x56243a4f2a50 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce97fd0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce901aa460 con 0x56243a4f85b0 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 0x56243a504360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 0x56243a504360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54888/0 (socket says 172.21.15.7:54888) 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 -- 172.21.15.7:0/3451240169 learned_addr learned my addr 172.21.15.7:0/3451240169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8effd640 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f2a50 0x56243a4ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a50f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 -- 172.21.15.7:0/3451240169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 msgr2=0x56243a50f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a50f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 -- 172.21.15.7:0/3451240169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f2a50 msgr2=0x56243a4ed060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f2a50 0x56243a4ed060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 -- 172.21.15.7:0/3451240169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243a12b0f0 con 0x56243a4f3440 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a50f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 0x56243a504360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fce78005ab0 tx=0x7fce78041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7c8667cac9941a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce84ff9640 1 -- 172.21.15.7:0/3451240169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78095020 con 0x56243a4f3440 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce84ff9640 1 -- 172.21.15.7:0/3451240169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce780bdcd0 con 0x56243a4f3440 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.576+0000 7fce84ff9640 1 -- 172.21.15.7:0/3451240169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780c5d70 con 0x56243a4f3440 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 -- 172.21.15.7:0/3451240169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 msgr2=0x56243a504360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 0x56243a504360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fce78005ab0 tx=0x7fce78041040 comp rx=0 tx=0).stop 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 -- 172.21.15.7:0/3451240169 shutdown_connections 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a50f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f3440 0x56243a504360 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 --2- 172.21.15.7:0/3451240169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f2a50 0x56243a4ed060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 -- 172.21.15.7:0/3451240169 >> 172.21.15.7:0/3451240169 conn(0x56243a4edfc0 msgr2=0x56243a4ec950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 -- 172.21.15.7:0/3451240169 shutdown_connections 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 -- 172.21.15.7:0/3451240169 wait complete. 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 -- start start 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 Processor -- start 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.577+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a503b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a5040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a502170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce901aaad0 con 0x56243a4f2a50 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce901abf40 con 0x56243a4f3440 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ab1a0 con 0x56243a4f85b0 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 10 client.0 _ll_register_callbacks cb 0x7fce80009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a503b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a503b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54896/0 (socket says 172.21.15.7:54896) 2024-03-19T00:40:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 -- 172.21.15.7:0/1150205335 learned_addr learned my addr 172.21.15.7:0/1150205335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a5040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8f7fe640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a502170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 -- 172.21.15.7:0/1150205335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 msgr2=0x56243a502170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a502170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 -- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 msgr2=0x56243a5040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a5040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 -- 172.21.15.7:0/1150205335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243a5027e0 con 0x56243a4f2a50 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce8effd640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a503b60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fce8801b040 tx=0x7fce88005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbad6922fcf12169 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce767fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce767fc640 1 -- 172.21.15.7:0/1150205335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8801d040 con 0x56243a4f2a50 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce767fc640 1 -- 172.21.15.7:0/1150205335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fce8802ba30 con 0x56243a4f2a50 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce780bdcd0 con 0x56243a4f2a50 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce767fc640 1 -- 172.21.15.7:0/1150205335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce8803c050 con 0x56243a4f2a50 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 10 client.15990 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.578+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56243a56f9e0 con 0x56243a4f2a50 2024-03-19T00:40:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce767fc640 1 -- 172.21.15.7:0/1150205335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce780bdcd0 con 0x56243a4f2a50 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce767fc640 1 -- 172.21.15.7:0/1150205335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fce880245f0 con 0x56243a4f2a50 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce97fd0740 10 client.15990 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce97fd0740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce97fd0740 20 client.15990 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce97fd0740 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a570200 0x56243a572620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce97fd0740 4 client.15990 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56243a45ed90 con 0x56243a570200 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a570200 0x56243a572620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce767fc640 10 client.15990 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.579+0000 7fce8e7fc640 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a570200 0x56243a572620 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7da7365ee4efc2 server_cookie=7f7f9422a3b08625 in_seq=0 out_seq=0 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce767fc640 1 -- 172.21.15.7:0/1150205335 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56243a45ed90 con 0x56243a570200 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce767fc640 10 client.15990 handle_command_reply: tid=0 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 client.15990 shutdown 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 msgr2=0x56243a503b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a503b60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fce8801b040 tx=0x7fce88005db0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 shutdown_connections 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56243a4f85b0 0x56243a502170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56243a4f3440 0x56243a5040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56243a570200 0x56243a572620 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 --2- 172.21.15.7:0/1150205335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56243a4f2a50 0x56243a503b60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce767fc640 0 client.15990 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 >> 172.21.15.7:0/1150205335 conn(0x56243a4edfc0 msgr2=0x56243a4f1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 shutdown_connections 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 1 -- 172.21.15.7:0/1150205335 wait complete. 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce97fd0740 20 client.15990 trim_cache size 0 max 16384 2024-03-19T00:40:08.587 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: "b78f1ee7-4662-4ad5-be93-71503a308cfe": { 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "b78f1ee7-4662-4ad5-be93-71503a308cfe", 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:40:08.588 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce7c0a2800 msgr2=0x7fce7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce7c0a2800 0x7fce7c0a4cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fce80005fd0 tx=0x7fce80009e10 comp rx=0 tx=0).stop 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 msgr2=0x7fce9019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce9019a220 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fce78006010 tx=0x7fce78005e40 comp rx=0 tx=0).stop 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 shutdown_connections 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fce7c0a2800 0x7fce7c0a4cc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce901022f0 0x7fce901a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce90101930 0x7fce9019a220 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 --2- 172.21.15.7:0/1343335864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce90100740 0x7fce90199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 >> 172.21.15.7:0/1343335864 conn(0x7fce900fbcd0 msgr2=0x7fce90109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 shutdown_connections 2024-03-19T00:40:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:08.580+0000 7fce953e6640 1 -- 172.21.15.7:0/1343335864 wait complete. 2024-03-19T00:40:08.592 INFO:tasks.cephfs.filesystem:scrub status for tag:b78f1ee7-4662-4ad5-be93-71503a308cfe - {'path': '/', 'tag': 'b78f1ee7-4662-4ad5-be93-71503a308cfe', 'options': 'recursive,force'} 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: pgmap v417: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:09 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: pgmap v417: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:09 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: pgmap v417: 129 pgs: 129 active+clean; 937 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:40:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:40:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:40:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:09 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:11.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:11 smithi204 ceph-mon[66572]: pgmap v418: 129 pgs: 129 active+clean; 960 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:11.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:11 smithi044 ceph-mon[66366]: pgmap v418: 129 pgs: 129 active+clean; 960 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:11 smithi007 ceph-mon[62854]: pgmap v418: 129 pgs: 129 active+clean; 960 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:13.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:13 smithi204 ceph-mon[66572]: pgmap v419: 129 pgs: 129 active+clean; 960 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:40:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:13 smithi044 ceph-mon[66366]: pgmap v419: 129 pgs: 129 active+clean; 960 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:40:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:13 smithi007 ceph-mon[62854]: pgmap v419: 129 pgs: 129 active+clean; 960 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:40:14.490 INFO:tasks.workunit.client.0.smithi007.stdout:1/600: open fd = 3 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:2/600: open fd = 3 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:3/600: open fd = 3 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:4/600: open fd = 3 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:5/600: open fd = 3 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:6/600: open fd = 3 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:7/600: open fd = 3 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.492 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:8/600: open fd = 3 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:9/600: open fd = 3 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:10/600: open fd = 3 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:11/600: open fd = 3 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:12/600: open fd = 3 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:13/600: open fd = 3 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:14/600: open fd = 3 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:15/600: open fd = 3 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:16/600: open fd = 3 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:17/600: open fd = 3 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:18/600: open fd = 3 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:19/600: open fd = 3 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:20/600: open fd = 3 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:21/600: open fd = 3 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:22/600: open fd = 3 2024-03-19T00:40:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:23/600: open fd = 3 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:24/600: open fd = 3 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:25/600: open fd = 3 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:26/600: open fd = 3 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:27/600: open fd = 3 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:28/600: open fd = 3 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:29/600: open fd = 3 2024-03-19T00:40:14.498 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:30/600: open fd = 3 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:31/600: open fd = 3 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:32/600: open fd = 3 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:33/600: open fd = 3 2024-03-19T00:40:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:34/600: open fd = 3 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:35/600: open fd = 3 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:36/600: open fd = 3 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.500 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:37/600: open fd = 3 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:38/600: open fd = 3 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:39/600: open fd = 3 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:40/600: open fd = 3 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.501 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:41/600: open fd = 3 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:42/600: open fd = 3 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:43/600: open fd = 3 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.502 INFO:tasks.workunit.client.0.smithi007.stdout:44/600: open fd = 3 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:45/600: open fd = 3 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:46/600: open fd = 3 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:47/600: open fd = 3 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.503 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:48/600: open fd = 3 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:49/600: open fd = 3 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:50/600: open fd = 3 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:51/600: open fd = 3 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.504 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:52/600: open fd = 3 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:53/600: open fd = 3 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:54/600: open fd = 3 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:55/600: open fd = 3 2024-03-19T00:40:14.505 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:56/600: open fd = 3 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:57/600: open fd = 3 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:58/600: open fd = 3 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:59/600: open fd = 3 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:60/600: open fd = 3 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:61/600: open fd = 3 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:62/600: open fd = 3 2024-03-19T00:40:14.507 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:63/600: open fd = 3 2024-03-19T00:40:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:40:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:40:15.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:15 smithi204 ceph-mon[66572]: pgmap v420: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:15.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:15 smithi044 ceph-mon[66366]: pgmap v420: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:15.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:15 smithi007 ceph-mon[62854]: pgmap v420: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:16 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:40:16.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:16 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:40:16.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:16 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:40:16.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:16 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:40:16.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:16 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:40:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:16 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:40:17.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:17 smithi204 ceph-mon[66572]: pgmap v421: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:17.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:17 smithi044 ceph-mon[66366]: pgmap v421: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:17.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:17 smithi007 ceph-mon[62854]: pgmap v421: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:19.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:19 smithi204 ceph-mon[66572]: pgmap v422: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:19.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:19 smithi007 ceph-mon[62854]: pgmap v422: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:19.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:19 smithi044 ceph-mon[66366]: pgmap v422: 129 pgs: 129 active+clean; 976 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:21.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:21 smithi204 ceph-mon[66572]: pgmap v423: 129 pgs: 129 active+clean; 999 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:21 smithi044 ceph-mon[66366]: pgmap v423: 129 pgs: 129 active+clean; 999 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:21.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:21 smithi007 ceph-mon[62854]: pgmap v423: 129 pgs: 129 active+clean; 999 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:23.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:23 smithi204 ceph-mon[66572]: pgmap v424: 129 pgs: 129 active+clean; 999 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:23.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:23 smithi044 ceph-mon[66366]: pgmap v424: 129 pgs: 129 active+clean; 999 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:23 smithi007 ceph-mon[62854]: pgmap v424: 129 pgs: 129 active+clean; 999 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:25.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:25 smithi204 ceph-mon[66572]: pgmap v425: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:40:25.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:25 smithi044 ceph-mon[66366]: pgmap v425: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:40:25.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:25 smithi007 ceph-mon[62854]: pgmap v425: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:40:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:27 smithi204 ceph-mon[66572]: pgmap v426: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:27.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:27 smithi044 ceph-mon[66366]: pgmap v426: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:27 smithi007 ceph-mon[62854]: pgmap v426: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:29 smithi204 ceph-mon[66572]: pgmap v427: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:29.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:29 smithi044 ceph-mon[66366]: pgmap v427: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:29 smithi007 ceph-mon[62854]: pgmap v427: 129 pgs: 129 active+clean; 1016 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:31.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:31 smithi044 ceph-mon[66366]: pgmap v428: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:31.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:31 smithi007 ceph-mon[62854]: pgmap v428: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:31 smithi204 ceph-mon[66572]: pgmap v428: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:33.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:33 smithi044 ceph-mon[66366]: pgmap v429: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:33.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:33 smithi007 ceph-mon[62854]: pgmap v429: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:33 smithi204 ceph-mon[66572]: pgmap v429: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:35.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:35 smithi044 ceph-mon[66366]: pgmap v430: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:40:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:35 smithi007 ceph-mon[62854]: pgmap v430: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:40:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:35 smithi204 ceph-mon[66572]: pgmap v430: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:40:37.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:37 smithi044 ceph-mon[66366]: pgmap v431: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:37.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:37 smithi007 ceph-mon[62854]: pgmap v431: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:38.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:37 smithi204 ceph-mon[66572]: pgmap v431: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:38.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:40:38.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:40:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/1212733657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8101930 msgr2=0x7fe4b8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/1212733657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8101930 0x7fe4b8101db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009860 tx=0x7fe4a002bb70 comp rx=0 tx=0).stop 2024-03-19T00:40:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/1212733657 shutdown_connections 2024-03-19T00:40:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/1212733657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4b81022f0 0x7fe4b8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/1212733657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8101930 0x7fe4b8101db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/1212733657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8100740 0x7fe4b8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/1212733657 >> 172.21.15.7:0/1212733657 conn(0x7fe4b80fbcd0 msgr2=0x7fe4b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/1212733657 shutdown_connections 2024-03-19T00:40:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.710+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/1212733657 wait complete. 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 Processor -- start 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 -- start start 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8100740 0x7fe4b8196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 0x7fe4b8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4b81022f0 0x7fe4b819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b810c270 con 0x7fe4b8101930 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b810c0f0 con 0x7fe4b8100740 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.711+0000 7fe4bd1cd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b810c3f0 con 0x7fe4b81022f0 2024-03-19T00:40:38.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 0x7fe4b8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 0x7fe4b8196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57114/0 (socket says 172.21.15.7:57114) 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4ae59c640 1 -- 172.21.15.7:0/740900463 learned_addr learned my addr 172.21.15.7:0/740900463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8100740 0x7fe4b8196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b759e640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4b81022f0 0x7fe4b819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/740900463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4b81022f0 msgr2=0x7fe4b819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4b81022f0 0x7fe4b819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 msgr2=0x7fe4b8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 0x7fe4b8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/740900463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b819e560 con 0x7fe4b8100740 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 0x7fe4b8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8100740 0x7fe4b8196940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8101790 tx=0x7fe4a800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b4d99640 1 -- 172.21.15.7:0/740900463 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8004440 con 0x7fe4b8100740 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b4d99640 1 -- 172.21.15.7:0/740900463 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4a800cd70 con 0x7fe4b8100740 2024-03-19T00:40:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b8069f20 con 0x7fe4b8100740 2024-03-19T00:40:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4b4d99640 1 -- 172.21.15.7:0/740900463 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8019480 con 0x7fe4b8100740 2024-03-19T00:40:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.712+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b806a470 con 0x7fe4b8100740 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4b4d99640 1 -- 172.21.15.7:0/740900463 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe4a8003710 con 0x7fe4b8100740 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 Processor -- start 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 -- start start 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302ba50 0x5626e302be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e3030f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e3033a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626e2f90500 con 0x5626e30315b0 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626e2f38be0 con 0x5626e302c440 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5626e2f8c5a0 con 0x5626e302ba50 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4b4d99640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4a40a2800 0x7fe4a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.715+0000 7fe4ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e3033a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e3033a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57130/0 (socket says 172.21.15.7:57130) 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e3030f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 -- 172.21.15.7:0/1387298036 learned_addr learned my addr 172.21.15.7:0/1387298036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4b759e640 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302ba50 0x5626e302be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4b759e640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4a40a2800 0x7fe4a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4b4d99640 1 -- 172.21.15.7:0/740900463 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe4a80cf140 con 0x7fe4b8100740 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 -- 172.21.15.7:0/1387298036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302ba50 msgr2=0x5626e302be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302ba50 0x5626e302be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 -- 172.21.15.7:0/1387298036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 msgr2=0x5626e3030f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e3030f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 -- 172.21.15.7:0/1387298036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e2f5a000 con 0x5626e30315b0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4b759e640 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302ba50 0x5626e302be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e3033a40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009980 tx=0x7fe4a0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10100a6adc5cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4b759e640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4a40a2800 0x7fe4a40a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5626e302cca0 tx=0x7fe4b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ad59a640 1 -- 172.21.15.7:0/1387298036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0039040 con 0x5626e30315b0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ad59a640 1 -- 172.21.15.7:0/1387298036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4a0034dc0 con 0x5626e30315b0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.716+0000 7fe4ad59a640 1 -- 172.21.15.7:0/1387298036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0003850 con 0x5626e30315b0 2024-03-19T00:40:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1387298036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 msgr2=0x5626e3033a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e3033a40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009980 tx=0x7fe4a0005d90 comp rx=0 tx=0).stop 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1387298036 shutdown_connections 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e3033a40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e3030f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1387298036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302ba50 0x5626e302be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1387298036 >> 172.21.15.7:0/1387298036 conn(0x5626e3026fc0 msgr2=0x5626e30273c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1387298036 shutdown_connections 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1387298036 wait complete. 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.717+0000 7fe4bfdb0740 1 -- start start 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 Processor -- start 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e3035e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 0x5626e30363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626e3034500 con 0x5626e302c440 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626e2f4c160 con 0x5626e302ba50 2024-03-19T00:40:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5626e3034680 con 0x5626e30315b0 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4bfdb0740 10 client.0 _ll_register_callbacks cb 0x5626e2f88a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 0x5626e30363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e3035e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 0x5626e30363c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57136/0 (socket says 172.21.15.7:57136) 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 -- 172.21.15.7:0/2015810423 learned_addr learned my addr 172.21.15.7:0/2015810423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4b759e640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 -- 172.21.15.7:0/2015810423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 msgr2=0x5626e303d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 -- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 msgr2=0x5626e3035e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e3035e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4ae59c640 1 -- 172.21.15.7:0/2015810423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e2d6e7d0 con 0x5626e302c440 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4b759e640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.718+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e3035e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 0x5626e30363c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0037040 tx=0x7fe4a0034bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ff6cddee8e813f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe49e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe49e7fc640 1 -- 172.21.15.7:0/2015810423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0039030 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a0034dc0 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe49e7fc640 1 -- 172.21.15.7:0/2015810423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe4a002c510 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe49e7fc640 1 -- 172.21.15.7:0/2015810423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4a005a050 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 10 client.16002 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626e2ca43f0 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe49e7fc640 1 -- 172.21.15.7:0/2015810423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4a0034dc0 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe49e7fc640 1 -- 172.21.15.7:0/2015810423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe4a00046c0 con 0x5626e302c440 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 10 client.16002 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 20 client.16002 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e3041880 0x5626e3043c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 4 client.16002 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.719+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e2f9da40 con 0x5626e3041880 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.720+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e3041880 0x5626e3043c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.720+0000 7fe49e7fc640 10 client.16002 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.720+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e3041880 0x5626e3043c90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61edca5f7d4d0acd server_cookie=e1d2e970b46fc5cb in_seq=0 out_seq=0 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.720+0000 7fe49e7fc640 1 -- 172.21.15.7:0/2015810423 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5626e2f9da40 con 0x5626e3041880 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.720+0000 7fe49e7fc640 10 client.16002 handle_command_reply: tid=0 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 client.16002 shutdown 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 msgr2=0x5626e30363c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 0x5626e30363c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0037040 tx=0x7fe4a0034bc0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 shutdown_connections 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e3041880 0x5626e3043c90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302c440 0x5626e30363c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/2015810423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e3035e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe49e7fc640 0 client.16002 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:38.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 >> 172.21.15.7:0/2015810423 conn(0x5626e3026fc0 msgr2=0x5626e30261e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 shutdown_connections 2024-03-19T00:40:38.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/2015810423 wait complete. 2024-03-19T00:40:38.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.721+0000 7fe4bfdb0740 20 client.16002 trim_cache size 0 max 16384 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.738+0000 7fe4bfdb0740 1 Processor -- start 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.738+0000 7fe4bfdb0740 1 -- start start 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e302d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302c440 0x5626e303d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e302dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81aa820 con 0x5626e30315b0 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a80f0 con 0x5626e302ba50 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a5f40 con 0x5626e302c440 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e302dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e302d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e302dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57150/0 (socket says 172.21.15.7:57150) 2024-03-19T00:40:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 -- 172.21.15.7:0/1811107561 learned_addr learned my addr 172.21.15.7:0/1811107561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302c440 0x5626e303d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 -- 172.21.15.7:0/1811107561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302c440 msgr2=0x5626e303d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302c440 0x5626e303d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 -- 172.21.15.7:0/1811107561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 msgr2=0x5626e302d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e302d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 -- 172.21.15.7:0/1811107561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e2c640f0 con 0x5626e30315b0 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e302d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4b759e640 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e302dde0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00095a0 tx=0x7fe4b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5e24ef9e1e93bd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4acd99640 1 -- 172.21.15.7:0/1811107561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b002f070 con 0x5626e30315b0 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4acd99640 1 -- 172.21.15.7:0/1811107561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4b001fb90 con 0x5626e30315b0 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.739+0000 7fe4acd99640 1 -- 172.21.15.7:0/1811107561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b001e410 con 0x5626e30315b0 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1811107561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 msgr2=0x5626e302dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e302dde0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00095a0 tx=0x7fe4b0018000 comp rx=0 tx=0).stop 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1811107561 shutdown_connections 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e30315b0 0x5626e302dde0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e302c440 0x5626e303d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1811107561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302ba50 0x5626e302d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1811107561 >> 172.21.15.7:0/1811107561 conn(0x5626e3026fc0 msgr2=0x5626e30273a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1811107561 shutdown_connections 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1811107561 wait complete. 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.740+0000 7fe4bfdb0740 1 -- start start 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 Processor -- start 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 0x5626e303ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e303b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a65b0 con 0x5626e302ba50 2024-03-19T00:40:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a7a20 con 0x5626e302c440 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81a6c80 con 0x5626e30315b0 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4bfdb0740 10 client.0 _ll_register_callbacks cb 0x7fe4a80973e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 0x5626e303ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 0x5626e303ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57156/0 (socket says 172.21.15.7:57156) 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/1295389144 learned_addr learned my addr 172.21.15.7:0/1295389144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e303b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b759e640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/1295389144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 msgr2=0x5626e303c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 msgr2=0x5626e303b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e303b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 -- 172.21.15.7:0/1295389144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e303c820 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e303b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe4b6d9d640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 0x5626e303ace0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8097be0 tx=0x7fe4a806a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e67f02619ef8fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe49e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.741+0000 7fe49e7fc640 1 -- 172.21.15.7:0/1295389144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a809b780 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b001fb90 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe49e7fc640 1 -- 172.21.15.7:0/1295389144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe4a80860b0 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe49e7fc640 1 -- 172.21.15.7:0/1295389144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4a809f8d0 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 10 client.16008 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626e30a7f30 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe49e7fc640 1 -- 172.21.15.7:0/1295389144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4b001fb90 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe49e7fc640 1 -- 172.21.15.7:0/1295389144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe4a8097040 con 0x5626e302ba50 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 10 client.16008 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 20 client.16008 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e30a88e0 0x5626e30aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 4 client.16008 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626e2f97d90 con 0x5626e30a88e0 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.742+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e30a88e0 0x5626e30aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe49e7fc640 10 client.16008 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4ae59c640 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e30a88e0 0x5626e30aad00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6209850c35d74646 server_cookie=2e9f689a8c9bff48 in_seq=0 out_seq=0 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe49e7fc640 1 -- 172.21.15.7:0/1295389144 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5626e2f97d90 con 0x5626e30a88e0 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe49e7fc640 10 client.16008 handle_command_reply: tid=0 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 client.16008 shutdown 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 msgr2=0x5626e303ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 0x5626e303ace0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8097be0 tx=0x7fe4a806a5e0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 shutdown_connections 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626e30315b0 0x5626e303c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626e302c440 0x5626e303b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626e30a88e0 0x5626e30aad00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 --2- 172.21.15.7:0/1295389144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626e302ba50 0x5626e303ace0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe49e7fc640 0 client.16008 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 >> 172.21.15.7:0/1295389144 conn(0x5626e3026fc0 msgr2=0x5626e3035e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 shutdown_connections 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 1 -- 172.21.15.7:0/1295389144 wait complete. 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.743+0000 7fe4bfdb0740 20 client.16008 trim_cache size 0 max 16384 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:40:38.751 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:40:38.752 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:40:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4a40a2800 msgr2=0x7fe4a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4a40a2800 0x7fe4a40a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5626e302cca0 tx=0x7fe4b0009500 comp rx=0 tx=0).stop 2024-03-19T00:40:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8100740 msgr2=0x7fe4b8196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8100740 0x7fe4b8196940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8101790 tx=0x7fe4a800ee50 comp rx=0 tx=0).stop 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 shutdown_connections 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4a40a2800 0x7fe4a40a4cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4b81022f0 0x7fe4b819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b8101930 0x7fe4b8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 --2- 172.21.15.7:0/740900463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b8100740 0x7fe4b8196940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 >> 172.21.15.7:0/740900463 conn(0x7fe4b80fbcd0 msgr2=0x7fe4b8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 shutdown_connections 2024-03-19T00:40:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.744+0000 7fe4bd1cd640 1 -- 172.21.15.7:0/740900463 wait complete. 2024-03-19T00:40:38.755 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:40:38.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 -- 172.21.15.7:0/3041879824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8102aa0 msgr2=0x7fd1e8102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/3041879824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8102f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009860 tx=0x7fd1d002bb70 comp rx=0 tx=0).stop 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 -- 172.21.15.7:0/3041879824 shutdown_connections 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/3041879824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8103460 0x7fd1e8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/3041879824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8102f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/3041879824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1e81018b0 0x7fd1e8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 -- 172.21.15.7:0/3041879824 >> 172.21.15.7:0/3041879824 conn(0x7fd1e80fce40 msgr2=0x7fd1e80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:38.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 -- 172.21.15.7:0/3041879824 shutdown_connections 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.857+0000 7fd1ecba9640 1 -- 172.21.15.7:0/3041879824 wait complete. 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 Processor -- start 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 -- start start 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1e81018b0 0x7fd1e80755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8103460 0x7fd1e8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e810d350 con 0x7fd1e8102aa0 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e810d1d0 con 0x7fd1e8103460 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ecba9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e810d4d0 con 0x7fd1e81018b0 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1e6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8103460 0x7fd1e8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57172/0 (socket says 172.21.15.7:57172) 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 -- 172.21.15.7:0/2965217439 learned_addr learned my addr 172.21.15.7:0/2965217439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:38.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1e659c640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1e81018b0 0x7fd1e80755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 -- 172.21.15.7:0/2965217439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1e81018b0 msgr2=0x7fd1e80755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1e81018b0 0x7fd1e80755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 -- 172.21.15.7:0/2965217439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8103460 msgr2=0x7fd1e8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8103460 0x7fd1e8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1ddd9b640 1 -- 172.21.15.7:0/2965217439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e80749e0 con 0x7fd1e8102aa0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.858+0000 7fd1e6d9d640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8103460 0x7fd1e8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.859+0000 7fd1ddd9b640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8073c10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0005940 tx=0x7fd1d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.859+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0041020 con 0x7fd1e8102aa0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.859+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e81a1b10 con 0x7fd1e8102aa0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.859+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d002c9c0 con 0x7fd1e8102aa0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.859+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e81a20b0 con 0x7fd1e8102aa0 2024-03-19T00:40:38.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.859+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0039040 con 0x7fd1e8102aa0 2024-03-19T00:40:38.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.860+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e8108ca0 con 0x7fd1e8102aa0 2024-03-19T00:40:38.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.861+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fd1d0044020 con 0x7fd1e8102aa0 2024-03-19T00:40:38.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.861+0000 7fd1dffff640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1d40a2800 0x7fd1d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:38.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.862+0000 7fd1e659c640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1d40a2800 0x7fd1d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:38.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.862+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd1d00f0740 con 0x7fd1e8102aa0 2024-03-19T00:40:38.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.862+0000 7fd1e659c640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1d40a2800 0x7fd1d40a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8102900 tx=0x7fd1d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:38.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:38.866+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd1d004b020 con 0x7fd1e8102aa0 2024-03-19T00:40:39.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.089+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1e8062420 con 0x7fd1e8102aa0 2024-03-19T00:40:39.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.090+0000 7fd1dffff640 1 -- 172.21.15.7:0/2965217439 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11856 (secure 0 0 0) 0x7fd1d00bd420 con 0x7fd1e8102aa0 2024-03-19T00:40:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:40:39.091 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:39:16.038000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:40:39.091 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-19T00:40:39.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.091+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1d40a2800 msgr2=0x7fd1d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.091+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1d40a2800 0x7fd1d40a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8102900 tx=0x7fd1d8005f90 comp rx=0 tx=0).stop 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.091+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 msgr2=0x7fd1e8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.091+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8073c10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0005940 tx=0x7fd1d0004270 comp rx=0 tx=0).stop 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 shutdown_connections 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd1d40a2800 0x7fd1d40a4cc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e8103460 0x7fd1e8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1e8102aa0 0x7fd1e8073c10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 --2- 172.21.15.7:0/2965217439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1e81018b0 0x7fd1e80755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 >> 172.21.15.7:0/2965217439 conn(0x7fd1e80fce40 msgr2=0x7fd1e8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 shutdown_connections 2024-03-19T00:40:39.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.092+0000 7fd1ecba9640 1 -- 172.21.15.7:0/2965217439 wait complete. 2024-03-19T00:40:39.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:40:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 -- 172.21.15.7:0/3536980661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c102e30 msgr2=0x7f060c103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 --2- 172.21.15.7:0/3536980661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c102e30 0x7f060c103250 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009900 tx=0x7f05fc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 -- 172.21.15.7:0/3536980661 shutdown_connections 2024-03-19T00:40:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 --2- 172.21.15.7:0/3536980661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c100260 0x7f060c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 --2- 172.21.15.7:0/3536980661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c0ff8a0 0x7f060c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 --2- 172.21.15.7:0/3536980661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c102e30 0x7f060c103250 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 -- 172.21.15.7:0/3536980661 >> 172.21.15.7:0/3536980661 conn(0x7f060c0fb400 msgr2=0x7f060c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 -- 172.21.15.7:0/3536980661 shutdown_connections 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.207+0000 7f0610c0b640 1 -- 172.21.15.7:0/3536980661 wait complete. 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 Processor -- start 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 -- start start 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c0ff8a0 0x7f060c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 0x7f060c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c102e30 0x7f060c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10d340 con 0x7f060c100260 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10d1c0 con 0x7f060c102e30 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0610c0b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10d4c0 con 0x7f060c0ff8a0 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 0x7f060c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 0x7f060c197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57194/0 (socket says 172.21.15.7:57194) 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 -- 172.21.15.7:0/1424360069 learned_addr learned my addr 172.21.15.7:0/1424360069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f060a59c640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c0ff8a0 0x7f060c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f060ad9d640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c102e30 0x7f060c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 -- 172.21.15.7:0/1424360069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c0ff8a0 msgr2=0x7f060c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c0ff8a0 0x7f060c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 -- 172.21.15.7:0/1424360069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c102e30 msgr2=0x7f060c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c102e30 0x7f060c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.208+0000 7f0603fff640 1 -- 172.21.15.7:0/1424360069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c19f680 con 0x7f060c100260 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.209+0000 7f0603fff640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 0x7f060c197fa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f05f400ea60 tx=0x7f05f400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.209+0000 7f06037fe640 1 -- 172.21.15.7:0/1424360069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f400cd20 con 0x7f060c100260 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.209+0000 7f06037fe640 1 -- 172.21.15.7:0/1424360069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f40035f0 con 0x7f060c100260 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.209+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c19f970 con 0x7f060c100260 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.209+0000 7f06037fe640 1 -- 172.21.15.7:0/1424360069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4010690 con 0x7f060c100260 2024-03-19T00:40:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.209+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c19fec0 con 0x7f060c100260 2024-03-19T00:40:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.211+0000 7f06037fe640 1 -- 172.21.15.7:0/1424360069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f05f40037a0 con 0x7f060c100260 2024-03-19T00:40:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.211+0000 7f06137fa740 1 Processor -- start 2024-03-19T00:40:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 -- start start 2024-03-19T00:40:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06037fe640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f80a2800 0x7f05f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 0x55b621e5ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e5fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e60260 0x55b621e626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b621dba4b0 con 0x55b621e5a700 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b621d62d70 con 0x55b621e60260 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06137fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b621db6550 con 0x55b621e5b0f0 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060ad9d640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f80a2800 0x7f05f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 0x55b621e5ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 0x55b621e5ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57210/0 (socket says 172.21.15.7:57210) 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 -- 172.21.15.7:0/3882880307 learned_addr learned my addr 172.21.15.7:0/3882880307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060ad9d640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e60260 0x55b621e626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f06037fe640 1 -- 172.21.15.7:0/1424360069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f05f40cf060 con 0x7f060c100260 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 -- 172.21.15.7:0/3882880307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 msgr2=0x55b621e5fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f0603fff640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e5fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e5fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 -- 172.21.15.7:0/3882880307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e60260 msgr2=0x55b621e626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e60260 0x55b621e626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060a59c640 1 -- 172.21.15.7:0/3882880307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621d85210 con 0x55b621e5a700 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f0603fff640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e5fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060ad9d640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e60260 0x55b621e626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.212+0000 7f060ad9d640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f80a2800 0x7f05f80a4cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0604005fd0 tx=0x7f0604009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f060a59c640 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 0x55b621e5ab20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009900 tx=0x7f05fc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49b3b70844d7ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f0600ff9640 1 -- 172.21.15.7:0/3882880307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc045020 con 0x55b621e5a700 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f0600ff9640 1 -- 172.21.15.7:0/3882880307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05fc004040 con 0x55b621e5a700 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f0600ff9640 1 -- 172.21.15.7:0/3882880307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005540 con 0x55b621e5a700 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 -- 172.21.15.7:0/3882880307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 msgr2=0x55b621e5ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 0x55b621e5ab20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009900 tx=0x7f05fc009560 comp rx=0 tx=0).stop 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 -- 172.21.15.7:0/3882880307 shutdown_connections 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e60260 0x55b621e626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e5fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 --2- 172.21.15.7:0/3882880307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e5a700 0x55b621e5ab20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.213+0000 7f06137fa740 1 -- 172.21.15.7:0/3882880307 >> 172.21.15.7:0/3882880307 conn(0x55b621e51860 msgr2=0x55b621e51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 -- 172.21.15.7:0/3882880307 shutdown_connections 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 -- 172.21.15.7:0/3882880307 wait complete. 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 -- start start 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 Processor -- start 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5a700 0x55b621e64b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5b0f0 0x55b621e650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b621e631b0 con 0x55b621e60260 2024-03-19T00:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b621d76310 con 0x55b621e5b0f0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b621e63330 con 0x55b621e5a700 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f06137fa740 10 client.0 _ll_register_callbacks cb 0x55b621d6a2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57224/0 (socket says 172.21.15.7:57224) 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f060ad9d640 1 -- 172.21.15.7:0/699648838 learned_addr learned my addr 172.21.15.7:0/699648838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f0603fff640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5b0f0 0x55b621e650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.214+0000 7f060a59c640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5a700 0x55b621e64b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f060ad9d640 1 -- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5a700 msgr2=0x55b621e64b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f060ad9d640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5a700 0x55b621e64b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f060ad9d640 1 -- 172.21.15.7:0/699648838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5b0f0 msgr2=0x55b621e650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f060ad9d640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5b0f0 0x55b621e650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f060ad9d640 1 -- 172.21.15.7:0/699648838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b621ace3f0 con 0x55b621e60260 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f0603fff640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5b0f0 0x55b621e650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f060ad9d640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6c100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f060400fcc0 tx=0x7f0604004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0afa4ce541dbf8f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f05f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f05f1ffb640 1 -- 172.21.15.7:0/699648838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604026070 con 0x55b621e60260 2024-03-19T00:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f05f1ffb640 1 -- 172.21.15.7:0/699648838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f060402a600 con 0x55b621e60260 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05fc004040 con 0x55b621e60260 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f05f1ffb640 1 -- 172.21.15.7:0/699648838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0604035050 con 0x55b621e60260 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f06137fa740 10 client.16026 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.215+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b621b987d0 con 0x55b621e60260 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f05f1ffb640 1 -- 172.21.15.7:0/699648838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05fc004040 con 0x55b621e60260 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f05f1ffb640 1 -- 172.21.15.7:0/699648838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f060401f070 con 0x55b621e60260 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f06137fa740 10 client.16026 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f06137fa740 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f06137fa740 20 client.16026 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f06137fa740 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621e708b0 0x55b621e72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f06137fa740 4 client.16026 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b621e183e0 con 0x55b621e708b0 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f060a59c640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621e708b0 0x55b621e72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f05f1ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.216+0000 7f060a59c640 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621e708b0 0x55b621e72d30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bb2336491d1a37 server_cookie=2b3c3947d0b8d4d0 in_seq=0 out_seq=0 2024-03-19T00:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f05f1ffb640 1 -- 172.21.15.7:0/699648838 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b621e183e0 con 0x55b621e708b0 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f05f1ffb640 10 client.16026 handle_command_reply: tid=0 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 client.16026 shutdown 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 msgr2=0x55b621e6c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6c100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f060400fcc0 tx=0x7f0604004ab0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 shutdown_connections 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621e708b0 0x55b621e72d30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6c100 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f05f1ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5b0f0 0x55b621e650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 --2- 172.21.15.7:0/699648838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5a700 0x55b621e64b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 >> 172.21.15.7:0/699648838 conn(0x55b621e51860 msgr2=0x55b621e50ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 shutdown_connections 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 1 -- 172.21.15.7:0/699648838 wait complete. 2024-03-19T00:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.217+0000 7f06137fa740 20 client.16026 trim_cache size 0 max 16384 2024-03-19T00:40:39.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.231+0000 7f06137fa740 1 Processor -- start 2024-03-19T00:40:39.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.231+0000 7f06137fa740 1 -- start start 2024-03-19T00:40:39.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f06137fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f06137fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f06137fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e50850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f06137fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1aa900 con 0x55b621e60260 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f06137fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a81d0 con 0x55b621e5a700 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f06137fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a6020 con 0x55b621e5b0f0 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e50850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e50310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38258/0 (socket says 172.21.15.7:38258) 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e50850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57238/0 (socket says 172.21.15.7:57238) 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 -- 172.21.15.7:0/272169242 learned_addr learned my addr 172.21.15.7:0/272169242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f0603fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 -- 172.21.15.7:0/272169242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 msgr2=0x55b621e6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 -- 172.21.15.7:0/272169242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 msgr2=0x55b621e50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e50310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 -- 172.21.15.7:0/272169242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621e5b980 con 0x55b621e60260 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060a59c640 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f060ad9d640 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e50850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0604004a80 tx=0x7f0604013610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ae9efd8bd70a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f05f3fff640 1 -- 172.21.15.7:0/272169242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604026040 con 0x55b621e60260 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f05f3fff640 1 -- 172.21.15.7:0/272169242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0604009820 con 0x55b621e60260 2024-03-19T00:40:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.232+0000 7f05f3fff640 1 -- 172.21.15.7:0/272169242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604020a70 con 0x55b621e60260 2024-03-19T00:40:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 -- 172.21.15.7:0/272169242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 msgr2=0x55b621e50850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e50850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0604004a80 tx=0x7f0604013610 comp rx=0 tx=0).stop 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 -- 172.21.15.7:0/272169242 shutdown_connections 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e50850 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 --2- 172.21.15.7:0/272169242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 -- 172.21.15.7:0/272169242 >> 172.21.15.7:0/272169242 conn(0x55b621e51860 msgr2=0x55b621e5c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 -- 172.21.15.7:0/272169242 shutdown_connections 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 -- 172.21.15.7:0/272169242 wait complete. 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 -- start start 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.233+0000 7f06137fa740 1 Processor -- start 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e69890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a6690 con 0x55b621e60260 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a7b00 con 0x55b621e5a700 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a6d60 con 0x55b621e5b0f0 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 10 client.0 _ll_register_callbacks cb 0x7f0604013dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e69890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e69890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38260/0 (socket says 172.21.15.7:38260) 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 -- 172.21.15.7:0/2083314228 learned_addr learned my addr 172.21.15.7:0/2083314228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f0603fff640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 -- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 msgr2=0x55b621e69dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e69dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 -- 172.21.15.7:0/2083314228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 msgr2=0x55b621e6ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 -- 172.21.15.7:0/2083314228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b621e6b2f0 con 0x55b621e5a700 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060ad9d640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f060a59c640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e69890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f05fc02c420 tx=0x7f05fc005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3fbf97fd78dcad server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f05f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f05f1ffb640 1 -- 172.21.15.7:0/2083314228 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc045040 con 0x55b621e5a700 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f05f1ffb640 1 -- 172.21.15.7:0/2083314228 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f05fc02cae0 con 0x55b621e5a700 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.234+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0604009820 con 0x55b621e5a700 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f05f1ffb640 1 -- 172.21.15.7:0/2083314228 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05fc041480 con 0x55b621e5a700 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 10 client.25883 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b621ed6ce0 con 0x55b621e5a700 2024-03-19T00:40:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f05f1ffb640 1 -- 172.21.15.7:0/2083314228 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0604009820 con 0x55b621e5a700 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f05f1ffb640 1 -- 172.21.15.7:0/2083314228 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f05fc034560 con 0x55b621e5a700 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 10 client.25883 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 10 client.25883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 20 client.25883 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621ed72b0 0x55b621ed96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 4 client.25883 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b621dc79f0 con 0x55b621ed72b0 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f0603fff640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621ed72b0 0x55b621ed96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f0603fff640 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621ed72b0 0x55b621ed96a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47c7f9d394a6c1dd server_cookie=2007f9e6572f92a3 in_seq=0 out_seq=0 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.235+0000 7f05f1ffb640 10 client.25883 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f05f1ffb640 1 -- 172.21.15.7:0/2083314228 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b621dc79f0 con 0x55b621ed72b0 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f05f1ffb640 10 client.25883 handle_command_reply: tid=0 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 client.25883 shutdown 2024-03-19T00:40:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 msgr2=0x55b621e69890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e69890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f05fc02c420 tx=0x7f05fc005e20 comp rx=0 tx=0).stop 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 shutdown_connections 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55b621ed72b0 0x55b621ed96a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b621e60260 0x55b621e6ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b621e5b0f0 0x55b621e69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 --2- 172.21.15.7:0/2083314228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b621e5a700 0x55b621e69890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f05f1ffb640 0 client.25883 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 >> 172.21.15.7:0/2083314228 conn(0x55b621e51860 msgr2=0x55b621e54ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 shutdown_connections 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 1 -- 172.21.15.7:0/2083314228 wait complete. 2024-03-19T00:40:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f06137fa740 20 client.25883 trim_cache size 0 max 16384 2024-03-19T00:40:39.245 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:40:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f80a2800 msgr2=0x7f05f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f80a2800 0x7f05f80a4cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0604005fd0 tx=0x7f0604009bf0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 msgr2=0x7f060c197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 0x7f060c197fa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f05f400ea60 tx=0x7f05f400ee60 comp rx=0 tx=0).stop 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 shutdown_connections 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f80a2800 0x7f05f80a4cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c102e30 0x7f060c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c100260 0x7f060c197fa0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 --2- 172.21.15.7:0/1424360069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c0ff8a0 0x7f060c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.236+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 >> 172.21.15.7:0/1424360069 conn(0x7f060c0fb400 msgr2=0x7f060c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.237+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 shutdown_connections 2024-03-19T00:40:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.237+0000 7f0610c0b640 1 -- 172.21.15.7:0/1424360069 wait complete. 2024-03-19T00:40:39.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 -- 172.21.15.7:0/2463476488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3288103470 msgr2=0x7f3288109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 --2- 172.21.15.7:0/2463476488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3288103470 0x7f3288109d00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3280009930 tx=0x7f328002bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 -- 172.21.15.7:0/2463476488 shutdown_connections 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 --2- 172.21.15.7:0/2463476488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3288103470 0x7f3288109d00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 --2- 172.21.15.7:0/2463476488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3288102ab0 0x7f3288102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 --2- 172.21.15.7:0/2463476488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32880ffae0 0x7f32880fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 -- 172.21.15.7:0/2463476488 >> 172.21.15.7:0/2463476488 conn(0x7f32880fb3d0 msgr2=0x7f32880fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 -- 172.21.15.7:0/2463476488 shutdown_connections 2024-03-19T00:40:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.351+0000 7f328ced1640 1 -- 172.21.15.7:0/2463476488 wait complete. 2024-03-19T00:40:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 Processor -- start 2024-03-19T00:40:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 -- start start 2024-03-19T00:40:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 0x7f3288197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3288102ab0 0x7f3288197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3288103470 0x7f328819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f328810d340 con 0x7f32880ffae0 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f328810d1c0 con 0x7f3288102ab0 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328ced1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f328810d4c0 con 0x7f3288103470 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 0x7f3288197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 0x7f3288197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57266/0 (socket says 172.21.15.7:57266) 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 -- 172.21.15.7:0/1887800669 learned_addr learned my addr 172.21.15.7:0/1887800669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f3285d9b640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3288102ab0 0x7f3288197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f3286d9d640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3288103470 0x7f328819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 -- 172.21.15.7:0/1887800669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3288103470 msgr2=0x7f328819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3288103470 0x7f328819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 -- 172.21.15.7:0/1887800669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3288102ab0 msgr2=0x7f3288197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3288102ab0 0x7f3288197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 -- 172.21.15.7:0/1887800669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328819f600 con 0x7f32880ffae0 2024-03-19T00:40:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.352+0000 7f328659c640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 0x7f3288197a30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f327800ebc0 tx=0x7f327800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.353+0000 7f32777fe640 1 -- 172.21.15.7:0/1887800669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003b70 con 0x7f32880ffae0 2024-03-19T00:40:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.353+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328819f8f0 con 0x7f32880ffae0 2024-03-19T00:40:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.353+0000 7f32777fe640 1 -- 172.21.15.7:0/1887800669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3278003d10 con 0x7f32880ffae0 2024-03-19T00:40:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.353+0000 7f32777fe640 1 -- 172.21.15.7:0/1887800669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278019840 con 0x7f32880ffae0 2024-03-19T00:40:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.353+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328819fe40 con 0x7f32880ffae0 2024-03-19T00:40:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.355+0000 7f32777fe640 1 -- 172.21.15.7:0/1887800669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f32780199e0 con 0x7f32880ffae0 2024-03-19T00:40:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.355+0000 7f328fb98740 1 Processor -- start 2024-03-19T00:40:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.355+0000 7f328fb98740 1 -- start start 2024-03-19T00:40:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328fb98740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fb2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328fb98740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fb77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f32777fe640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32700a2800 0x7f32700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328fb98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328fb98740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556127f164b0 con 0x556127fb22f0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328fb98740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556127ebed70 con 0x556127fb2ce0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328fb98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556127f12550 con 0x556127fb7e50 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fb2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3285d9b640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32700a2800 0x7f32700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fb2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57282/0 (socket says 172.21.15.7:57282) 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 -- 172.21.15.7:0/357272796 learned_addr learned my addr 172.21.15.7:0/357272796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f32777fe640 1 -- 172.21.15.7:0/1887800669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f32780cf690 con 0x7f32880ffae0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f328659c640 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fb77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 -- 172.21.15.7:0/357272796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 msgr2=0x556127fba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 -- 172.21.15.7:0/357272796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 msgr2=0x556127fb77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fb77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 -- 172.21.15.7:0/357272796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556127ee1210 con 0x556127fb22f0 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3285d9b640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32700a2800 0x7f32700a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3288198ef0 tx=0x7f327c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3286d9d640 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fb2710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f328002c640 tx=0x7f328002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4159d4f9dc5848 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3274ff9640 1 -- 172.21.15.7:0/357272796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328004a070 con 0x556127fb22f0 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3274ff9640 1 -- 172.21.15.7:0/357272796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3280005050 con 0x556127fb22f0 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.356+0000 7f3274ff9640 1 -- 172.21.15.7:0/357272796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280004900 con 0x556127fb22f0 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 -- 172.21.15.7:0/357272796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 msgr2=0x556127fb2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fb2710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f328002c640 tx=0x7f328002c5d0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 -- 172.21.15.7:0/357272796 shutdown_connections 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fb77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 --2- 172.21.15.7:0/357272796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fb2710 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 -- 172.21.15.7:0/357272796 >> 172.21.15.7:0/357272796 conn(0x556127fad860 msgr2=0x556127fadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 -- 172.21.15.7:0/357272796 shutdown_connections 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.357+0000 7f328fb98740 1 -- 172.21.15.7:0/357272796 wait complete. 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 -- start start 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 Processor -- start 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fbcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fc3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556127fbada0 con 0x556127fb22f0 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556127ed2310 con 0x556127fb2ce0 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556127fbaf20 con 0x556127fb7e50 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328fb98740 10 client.0 _ll_register_callbacks cb 0x556127ec62d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57292/0 (socket says 172.21.15.7:57292) 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 -- 172.21.15.7:0/3338533634 learned_addr learned my addr 172.21.15.7:0/3338533634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f3285d9b640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fbcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f3286d9d640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fc3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 -- 172.21.15.7:0/3338533634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 msgr2=0x556127fc3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fc3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 -- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 msgr2=0x556127fbcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fbcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 -- 172.21.15.7:0/3338533634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556127c2a3f0 con 0x556127fb22f0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f328659c640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbc7f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f327809b970 tx=0x7f327809bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaa34d7a2409d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.358+0000 7f326dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f326dffb640 1 -- 172.21.15.7:0/3338533634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780b8030 con 0x556127fb22f0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f326dffb640 1 -- 172.21.15.7:0/3338533634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f32780cf370 con 0x556127fb22f0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3280005050 con 0x556127fb22f0 2024-03-19T00:40:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f326dffb640 1 -- 172.21.15.7:0/3338533634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3278083050 con 0x556127fb22f0 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 10 client.16044 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556127cf47d0 con 0x556127fb22f0 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f326dffb640 1 -- 172.21.15.7:0/3338533634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3280005050 con 0x556127fb22f0 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f326dffb640 1 -- 172.21.15.7:0/3338533634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f32780a3f80 con 0x556127fb22f0 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 10 client.16044 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 20 client.16044 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556127fc8650 0x556127fcaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 4 client.16044 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556127f743e0 con 0x556127fc8650 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.359+0000 7f3285d9b640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556127fc8650 0x556127fcaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.360+0000 7f326dffb640 10 client.16044 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.360+0000 7f3285d9b640 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556127fc8650 0x556127fcaaf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=709f3180045a8ac4 server_cookie=7059a108d751e317 in_seq=0 out_seq=0 2024-03-19T00:40:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f326dffb640 1 -- 172.21.15.7:0/3338533634 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556127f743e0 con 0x556127fc8650 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f326dffb640 10 client.16044 handle_command_reply: tid=0 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 client.16044 shutdown 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 msgr2=0x556127fbc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbc7f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f327809b970 tx=0x7f327809bb70 comp rx=0 tx=0).stop 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 shutdown_connections 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556127fc8650 0x556127fcaaf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb7e50 0x556127fc3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x556127fbcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f326dffb640 0 client.16044 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 --2- 172.21.15.7:0/3338533634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbc7f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 >> 172.21.15.7:0/3338533634 conn(0x556127fad860 msgr2=0x556127facba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 shutdown_connections 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 1 -- 172.21.15.7:0/3338533634 wait complete. 2024-03-19T00:40:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.361+0000 7f328fb98740 20 client.16044 trim_cache size 0 max 16384 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 Processor -- start 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 -- start start 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb2ce0 0x556127fb11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb7e50 0x556127fcf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32881aca40 con 0x556127fb22f0 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32881aa310 con 0x556127fb7e50 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328fb98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32881a8160 con 0x556127fb2ce0 2024-03-19T00:40:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57302/0 (socket says 172.21.15.7:57302) 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f328659c640 1 -- 172.21.15.7:0/3530233779 learned_addr learned my addr 172.21.15.7:0/3530233779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.374+0000 7f3286d9d640 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb7e50 0x556127fcf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f3285d9b640 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb2ce0 0x556127fb11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328659c640 1 -- 172.21.15.7:0/3530233779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb2ce0 msgr2=0x556127fb11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328659c640 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb2ce0 0x556127fb11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328659c640 1 -- 172.21.15.7:0/3530233779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb7e50 msgr2=0x556127fcf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328659c640 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb7e50 0x556127fcf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328659c640 1 -- 172.21.15.7:0/3530233779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556127f83880 con 0x556127fb22f0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328659c640 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbcc90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3278004b10 tx=0x7f3278000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d75fe1ca5250dba server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f326ffff640 1 -- 172.21.15.7:0/3530233779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780b8070 con 0x556127fb22f0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f326ffff640 1 -- 172.21.15.7:0/3530233779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327809bc70 con 0x556127fb22f0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f326ffff640 1 -- 172.21.15.7:0/3530233779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278098480 con 0x556127fb22f0 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328fb98740 1 -- 172.21.15.7:0/3530233779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 msgr2=0x556127fbcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.375+0000 7f328fb98740 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbcc90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3278004b10 tx=0x7f3278000f40 comp rx=0 tx=0).stop 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- 172.21.15.7:0/3530233779 shutdown_connections 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb7e50 0x556127fcf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb2ce0 0x556127fb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 --2- 172.21.15.7:0/3530233779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb22f0 0x556127fbcc90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- 172.21.15.7:0/3530233779 >> 172.21.15.7:0/3530233779 conn(0x556127fad860 msgr2=0x556127fb0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- 172.21.15.7:0/3530233779 shutdown_connections 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- 172.21.15.7:0/3530233779 wait complete. 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- start start 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 Processor -- start 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb22f0 0x55612802a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x55612802a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 0x556128031600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32881a87d0 con 0x556127fb7e50 2024-03-19T00:40:39.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32881a9c40 con 0x556127fb2ce0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32881a8ea0 con 0x556127fb22f0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328fb98740 10 client.0 _ll_register_callbacks cb 0x7f3278097fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 0x556128031600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 0x556128031600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57306/0 (socket says 172.21.15.7:57306) 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 -- 172.21.15.7:0/1537270736 learned_addr learned my addr 172.21.15.7:0/1537270736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3285d9b640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x55612802a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f328659c640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb22f0 0x55612802a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 -- 172.21.15.7:0/1537270736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb22f0 msgr2=0x55612802a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb22f0 0x55612802a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 -- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 msgr2=0x55612802a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.376+0000 7f3286d9d640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x55612802a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f3286d9d640 1 -- 172.21.15.7:0/1537270736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556128031bf0 con 0x556127fb7e50 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f3286d9d640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 0x556128031600 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3280009930 tx=0x7f328002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb1929ae95b79f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f326dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:39.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f326dffb640 1 -- 172.21.15.7:0/1537270736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328004a040 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f326dffb640 1 -- 172.21.15.7:0/1537270736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3280005250 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327809bc70 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f326dffb640 1 -- 172.21.15.7:0/1537270736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3280009520 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 10 client.16050 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556128032ef0 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f326dffb640 1 -- 172.21.15.7:0/1537270736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327809bc70 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f326dffb640 1 -- 172.21.15.7:0/1537270736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f32800035f0 con 0x556127fb7e50 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 10 client.16050 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 20 client.16050 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556128033390 0x5561280357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 4 client.16050 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.377+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556127f239f0 con 0x556128033390 2024-03-19T00:40:39.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328659c640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556128033390 0x5561280357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f326dffb640 10 client.16050 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328659c640 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556128033390 0x5561280357b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bd2d9b5505ca6b server_cookie=7cb1c144014971ce in_seq=0 out_seq=0 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f326dffb640 1 -- 172.21.15.7:0/1537270736 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556127f239f0 con 0x556128033390 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f326dffb640 10 client.16050 handle_command_reply: tid=0 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 client.16050 shutdown 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 msgr2=0x556128031600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 0x556128031600 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3280009930 tx=0x7f328002c7e0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 shutdown_connections 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556128033390 0x5561280357b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556127fb7e50 0x556128031600 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556127fb2ce0 0x55612802a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f326dffb640 0 client.16050 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 --2- 172.21.15.7:0/1537270736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556127fb22f0 0x55612802a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 >> 172.21.15.7:0/1537270736 conn(0x556127fad860 msgr2=0x556127fbca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 shutdown_connections 2024-03-19T00:40:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 1 -- 172.21.15.7:0/1537270736 wait complete. 2024-03-19T00:40:39.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.378+0000 7f328fb98740 20 client.16050 trim_cache size 0 max 16384 2024-03-19T00:40:39.388 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:40:39.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32700a2800 msgr2=0x7f32700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32700a2800 0x7f32700a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3288198ef0 tx=0x7f327c005f90 comp rx=0 tx=0).stop 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 msgr2=0x7f3288197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 0x7f3288197a30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f327800ebc0 tx=0x7f327800c6a0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 shutdown_connections 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32700a2800 0x7f32700a4cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3288103470 0x7f328819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3288102ab0 0x7f3288197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 --2- 172.21.15.7:0/1887800669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880ffae0 0x7f3288197a30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 >> 172.21.15.7:0/1887800669 conn(0x7f32880fb3d0 msgr2=0x7f3288105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 shutdown_connections 2024-03-19T00:40:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.379+0000 7f328ced1640 1 -- 172.21.15.7:0/1887800669 wait complete. 2024-03-19T00:40:39.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 -- 172.21.15.7:0/2070672004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90101910 msgr2=0x7f6b90101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 --2- 172.21.15.7:0/2070672004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90101910 0x7f6b90101d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009900 tx=0x7f6b8002bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 -- 172.21.15.7:0/2070672004 shutdown_connections 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 --2- 172.21.15.7:0/2070672004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b901022d0 0x7f6b90108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 --2- 172.21.15.7:0/2070672004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90101910 0x7f6b90101d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 --2- 172.21.15.7:0/2070672004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b90100720 0x7f6b90100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 -- 172.21.15.7:0/2070672004 >> 172.21.15.7:0/2070672004 conn(0x7f6b900fbc90 msgr2=0x7f6b900fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 -- 172.21.15.7:0/2070672004 shutdown_connections 2024-03-19T00:40:39.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.494+0000 7f6b976e8640 1 -- 172.21.15.7:0/2070672004 wait complete. 2024-03-19T00:40:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 Processor -- start 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 -- start start 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90100720 0x7f6b9006aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 0x7f6b9019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b901022d0 0x7f6b9019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010c250 con 0x7f6b90101910 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010c0d0 con 0x7f6b90100720 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b976e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010c3d0 con 0x7f6b901022d0 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 0x7f6b9019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 0x7f6b9019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57336/0 (socket says 172.21.15.7:57336) 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 -- 172.21.15.7:0/973707484 learned_addr learned my addr 172.21.15.7:0/973707484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b95484640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90100720 0x7f6b9006aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b95c85640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b901022d0 0x7f6b9019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 -- 172.21.15.7:0/973707484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b901022d0 msgr2=0x7f6b9019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b901022d0 0x7f6b9019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 -- 172.21.15.7:0/973707484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90100720 msgr2=0x7f6b9006aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90100720 0x7f6b9006aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 -- 172.21.15.7:0/973707484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9019f730 con 0x7f6b90101910 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.495+0000 7f6b94c83640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 0x7f6b9019c800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b80006010 tx=0x7f6b80005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.496+0000 7f6b867fc640 1 -- 172.21.15.7:0/973707484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80043020 con 0x7f6b90101910 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.496+0000 7f6b867fc640 1 -- 172.21.15.7:0/973707484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b800043d0 con 0x7f6b90101910 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.496+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9019f960 con 0x7f6b90101910 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.496+0000 7f6b867fc640 1 -- 172.21.15.7:0/973707484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80003a80 con 0x7f6b90101910 2024-03-19T00:40:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.496+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9019ff00 con 0x7f6b90101910 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b867fc640 1 -- 172.21.15.7:0/973707484 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6b80046020 con 0x7f6b90101910 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 Processor -- start 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 -- start start 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3af6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3afb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3afbe50 0x5572c3afe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572c3a5a4b0 con 0x5572c3afbe50 2024-03-19T00:40:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5572c3a02d70 con 0x5572c3af6ce0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5572c3a56550 con 0x5572c3af62f0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.498+0000 7f6b867fc640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b7c0a2880 0x7f6b7c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b94c83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3afbe50 0x5572c3afe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3afb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3afb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38340/0 (socket says 172.21.15.7:38340) 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 -- 172.21.15.7:0/3113621814 learned_addr learned my addr 172.21.15.7:0/3113621814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95c85640 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3af6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95c85640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b7c0a2880 0x7f6b7c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b867fc640 1 -- 172.21.15.7:0/973707484 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6b800f0f60 con 0x7f6b90101910 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 -- 172.21.15.7:0/3113621814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 msgr2=0x5572c3af6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3af6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 -- 172.21.15.7:0/3113621814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3afbe50 msgr2=0x5572c3afe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3afbe50 0x5572c3afe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 -- 172.21.15.7:0/3113621814 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c3a25210 con 0x5572c3af6ce0 2024-03-19T00:40:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95c85640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b7c0a2880 0x7f6b7c0a4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5572c3af7540 tx=0x7f6b88009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b95484640 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3afb7e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009ca0 tx=0x7f6b8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75eca2782d8c618a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b7b7fe640 1 -- 172.21.15.7:0/3113621814 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004220 con 0x5572c3af6ce0 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b7b7fe640 1 -- 172.21.15.7:0/3113621814 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b8c0043c0 con 0x5572c3af6ce0 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.499+0000 7f6b7b7fe640 1 -- 172.21.15.7:0/3113621814 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c019840 con 0x5572c3af6ce0 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/3113621814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 msgr2=0x5572c3afb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3afb7e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009ca0 tx=0x7f6b8c011810 comp rx=0 tx=0).stop 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/3113621814 shutdown_connections 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3afbe50 0x5572c3afe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3afb7e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/3113621814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3af6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/3113621814 >> 172.21.15.7:0/3113621814 conn(0x5572c3af1860 msgr2=0x5572c3af1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/3113621814 shutdown_connections 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/3113621814 wait complete. 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 -- start start 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.500+0000 7f6b9a3a9740 1 Processor -- start 2024-03-19T00:40:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3b007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 0x5572c3b00d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3b07d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572c3afee00 con 0x5572c3af6ce0 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5572c3a16310 con 0x5572c3afbe50 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5572c3afef80 con 0x5572c3af62f0 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b9a3a9740 10 client.0 _ll_register_callbacks cb 0x5572c3a0a2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b94c83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 0x5572c3b00d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95484640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3b007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3b07d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b94c83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 0x5572c3b00d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57352/0 (socket says 172.21.15.7:57352) 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95484640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3b007e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:47664/0 (socket says 172.21.15.7:47664) 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b94c83640 1 -- 172.21.15.7:0/1919184568 learned_addr learned my addr 172.21.15.7:0/1919184568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 -- 172.21.15.7:0/1919184568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 msgr2=0x5572c3b007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3b007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 -- 172.21.15.7:0/1919184568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 msgr2=0x5572c3b00d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 0x5572c3b00d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 -- 172.21.15.7:0/1919184568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c376e3f0 con 0x5572c3afbe50 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b94c83640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 0x5572c3b00d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95484640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3b007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.501+0000 7f6b95c85640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3b07d60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b880099a0 tx=0x7f6b88004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ee822b058efd18 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b78ff9640 1 -- 172.21.15.7:0/1919184568 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8802d060 con 0x5572c3afbe50 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c0043c0 con 0x5572c3afbe50 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b78ff9640 1 -- 172.21.15.7:0/1919184568 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6b88004780 con 0x5572c3afbe50 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b78ff9640 1 -- 172.21.15.7:0/1919184568 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b88034050 con 0x5572c3afbe50 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 10 client.25913 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572c38387d0 con 0x5572c3afbe50 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b78ff9640 1 -- 172.21.15.7:0/1919184568 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b8c0043c0 con 0x5572c3afbe50 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b78ff9640 1 -- 172.21.15.7:0/1919184568 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6b88027030 con 0x5572c3afbe50 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 10 client.25913 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 20 client.25913 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b0c660 0x5572c3b0eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 4 client.25913 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c3ab83e0 con 0x5572c3b0c660 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.502+0000 7f6b95484640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b0c660 0x5572c3b0eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.503+0000 7f6b95484640 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b0c660 0x5572c3b0eaa0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fefc2462bba075b server_cookie=d9cb1b1a6397d50d in_seq=0 out_seq=0 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.503+0000 7f6b78ff9640 10 client.25913 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.503+0000 7f6b78ff9640 1 -- 172.21.15.7:0/1919184568 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572c3ab83e0 con 0x5572c3b0c660 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.503+0000 7f6b78ff9640 10 client.25913 handle_command_reply: tid=0 2024-03-19T00:40:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 client.25913 shutdown 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 msgr2=0x5572c3b07d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3b07d60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b880099a0 tx=0x7f6b88004a40 comp rx=0 tx=0).stop 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 shutdown_connections 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3b07d60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b0c660 0x5572c3b0eaa0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af6ce0 0x5572c3b00d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/1919184568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af62f0 0x5572c3b007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b78ff9640 0 client.25913 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 >> 172.21.15.7:0/1919184568 conn(0x5572c3af1860 msgr2=0x5572c3af0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 shutdown_connections 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/1919184568 wait complete. 2024-03-19T00:40:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.504+0000 7f6b9a3a9740 20 client.25913 trim_cache size 0 max 16384 2024-03-19T00:40:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 Processor -- start 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 -- start start 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3af0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af6ce0 0x5572c3b07d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3aff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901acac0 con 0x5572c3af62f0 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901aa390 con 0x5572c3afbe50 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901a81e0 con 0x5572c3af6ce0 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b95484640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3af0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b95484640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3af0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57358/0 (socket says 172.21.15.7:57358) 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b95484640 1 -- 172.21.15.7:0/642790042 learned_addr learned my addr 172.21.15.7:0/642790042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b95c85640 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3aff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.517+0000 7f6b94c83640 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af6ce0 0x5572c3b07d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b95484640 1 -- 172.21.15.7:0/642790042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af6ce0 msgr2=0x5572c3b07d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b95484640 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af6ce0 0x5572c3b07d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b95484640 1 -- 172.21.15.7:0/642790042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 msgr2=0x5572c3aff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b95484640 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3aff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b95484640 1 -- 172.21.15.7:0/642790042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c3ac7880 con 0x5572c3af62f0 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b95484640 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3af0a20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c013010 tx=0x7f6b8c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177f6a127e1c7a9a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b7affd640 1 -- 172.21.15.7:0/642790042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c031020 con 0x5572c3af62f0 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b7affd640 1 -- 172.21.15.7:0/642790042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b8c004310 con 0x5572c3af62f0 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.518+0000 7f6b7affd640 1 -- 172.21.15.7:0/642790042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c02d700 con 0x5572c3af62f0 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/642790042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 msgr2=0x5572c3af0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3af0a20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c013010 tx=0x7f6b8c009680 comp rx=0 tx=0).stop 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/642790042 shutdown_connections 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3afbe50 0x5572c3aff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3af6ce0 0x5572c3b07d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/642790042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3af0a20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/642790042 >> 172.21.15.7:0/642790042 conn(0x5572c3af1860 msgr2=0x5572c3af0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/642790042 shutdown_connections 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/642790042 wait complete. 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- start start 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 Processor -- start 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3b04b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3b05050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3afbe50 0x5572c3b709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901a8850 con 0x5572c3af62f0 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901a9cc0 con 0x5572c3af6ce0 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901a8f20 con 0x5572c3afbe50 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b9a3a9740 10 client.0 _ll_register_callbacks cb 0x7f6b88009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b95484640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3b04b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b95484640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3b04b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57374/0 (socket says 172.21.15.7:57374) 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b95484640 1 -- 172.21.15.7:0/2648095847 learned_addr learned my addr 172.21.15.7:0/2648095847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.519+0000 7f6b94c83640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3b05050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95c85640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3afbe50 0x5572c3b709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95484640 1 -- 172.21.15.7:0/2648095847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3afbe50 msgr2=0x5572c3b709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95484640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3afbe50 0x5572c3b709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95484640 1 -- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 msgr2=0x5572c3b05050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95484640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3b05050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95484640 1 -- 172.21.15.7:0/2648095847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c3b70f10 con 0x5572c3af62f0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95c85640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3afbe50 0x5572c3b709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b95484640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3b04b10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c011850 tx=0x7f6b8c004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1762e2745cb57bc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b78ff9640 1 -- 172.21.15.7:0/2648095847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c02d260 con 0x5572c3af62f0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b78ff9640 1 -- 172.21.15.7:0/2648095847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6b8c005030 con 0x5572c3af62f0 2024-03-19T00:40:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c004310 con 0x5572c3af62f0 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b78ff9640 1 -- 172.21.15.7:0/2648095847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c044050 con 0x5572c3af62f0 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b9a3a9740 10 client.16068 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572c3b72080 con 0x5572c3af62f0 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b78ff9640 1 -- 172.21.15.7:0/2648095847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b8c004310 con 0x5572c3af62f0 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.520+0000 7f6b78ff9640 1 -- 172.21.15.7:0/2648095847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6b8c02d680 con 0x5572c3af62f0 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 10 client.16068 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 20 client.16068 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b72910 0x5572c3b74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 4 client.16068 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572c3a679f0 con 0x5572c3b72910 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b94c83640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b72910 0x5572c3b74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b78ff9640 10 client.16068 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b94c83640 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b72910 0x5572c3b74d30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0167b61a7a5f49e server_cookie=6b7882843ba0fca3 in_seq=0 out_seq=0 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b78ff9640 1 -- 172.21.15.7:0/2648095847 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572c3a679f0 con 0x5572c3b72910 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b78ff9640 10 client.16068 handle_command_reply: tid=0 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 client.16068 shutdown 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 msgr2=0x5572c3b04b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3b04b10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c011850 tx=0x7f6b8c004550 comp rx=0 tx=0).stop 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 shutdown_connections 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572c3afbe50 0x5572c3b709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572c3af6ce0 0x5572c3b05050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572c3b72910 0x5572c3b74d30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 --2- 172.21.15.7:0/2648095847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572c3af62f0 0x5572c3b04b10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b78ff9640 0 client.16068 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.521+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 >> 172.21.15.7:0/2648095847 conn(0x5572c3af1860 msgr2=0x5572c3af5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 shutdown_connections 2024-03-19T00:40:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b9a3a9740 1 -- 172.21.15.7:0/2648095847 wait complete. 2024-03-19T00:40:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b9a3a9740 20 client.16068 trim_cache size 0 max 16384 2024-03-19T00:40:39.530 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b7c0a2880 msgr2=0x7f6b7c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b7c0a2880 0x7f6b7c0a4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5572c3af7540 tx=0x7f6b88009500 comp rx=0 tx=0).stop 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 msgr2=0x7f6b9019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 0x7f6b9019c800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b80006010 tx=0x7f6b80005e40 comp rx=0 tx=0).stop 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 shutdown_connections 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6b7c0a2880 0x7f6b7c0a4d40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b901022d0 0x7f6b9019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b90101910 0x7f6b9019c800 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 --2- 172.21.15.7:0/973707484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b90100720 0x7f6b9006aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 >> 172.21.15.7:0/973707484 conn(0x7f6b900fbc90 msgr2=0x7f6b90104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 shutdown_connections 2024-03-19T00:40:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:39.522+0000 7f6b976e8640 1 -- 172.21.15.7:0/973707484 wait complete. 2024-03-19T00:40:39.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:39 smithi007 ceph-mon[62854]: pgmap v432: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:39 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2965217439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:40:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:39 smithi044 ceph-mon[66366]: pgmap v432: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:39.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:39 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2965217439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:40:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:39 smithi204 ceph-mon[66572]: pgmap v432: 129 pgs: 129 active+clean; 1.0 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:39 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2965217439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:40:40.535 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:40:40.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:40:40.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 -- 172.21.15.7:0/2807484237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c101930 msgr2=0x7f6f5c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 --2- 172.21.15.7:0/2807484237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c101930 0x7f6f5c101db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009900 tx=0x7f6f4c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 -- 172.21.15.7:0/2807484237 shutdown_connections 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 --2- 172.21.15.7:0/2807484237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c1022f0 0x7f6f5c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 --2- 172.21.15.7:0/2807484237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c101930 0x7f6f5c101db0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 --2- 172.21.15.7:0/2807484237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c100740 0x7f6f5c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 -- 172.21.15.7:0/2807484237 >> 172.21.15.7:0/2807484237 conn(0x7f6f5c0fbcb0 msgr2=0x7f6f5c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 -- 172.21.15.7:0/2807484237 shutdown_connections 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.653+0000 7f6f639f4640 1 -- 172.21.15.7:0/2807484237 wait complete. 2024-03-19T00:40:40.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 Processor -- start 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 -- start start 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c100740 0x7f6f5c06aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 0x7f6f5c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c1022f0 0x7f6f5c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10c250 con 0x7f6f5c1022f0 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10c0d0 con 0x7f6f5c101930 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f639f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c10c3d0 con 0x7f6f5c100740 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 0x7f6f5c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f61f91640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c1022f0 0x7f6f5c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 0x7f6f5c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38376/0 (socket says 172.21.15.7:38376) 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 -- 172.21.15.7:0/261525267 learned_addr learned my addr 172.21.15.7:0/261525267 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f61790640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c100740 0x7f6f5c06aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 -- 172.21.15.7:0/261525267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c100740 msgr2=0x7f6f5c06aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c100740 0x7f6f5c06aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 -- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c1022f0 msgr2=0x7f6f5c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c1022f0 0x7f6f5c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f60f8f640 1 -- 172.21.15.7:0/261525267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c19f680 con 0x7f6f5c101930 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.654+0000 7f6f61f91640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c1022f0 0x7f6f5c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.655+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 0x7f6f5c19c780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c006010 tx=0x7f6f4c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.655+0000 7f6f527fc640 1 -- 172.21.15.7:0/261525267 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c043020 con 0x7f6f5c101930 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.655+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c19f910 con 0x7f6f5c101930 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.655+0000 7f6f527fc640 1 -- 172.21.15.7:0/261525267 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f4c0043d0 con 0x7f6f5c101930 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.655+0000 7f6f527fc640 1 -- 172.21.15.7:0/261525267 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c003b80 con 0x7f6f5c101930 2024-03-19T00:40:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.655+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c19feb0 con 0x7f6f5c101930 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.657+0000 7f6f527fc640 1 -- 172.21.15.7:0/261525267 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6f4c046020 con 0x7f6f5c101930 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.657+0000 7f6f665e7740 1 Processor -- start 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.657+0000 7f6f665e7740 1 -- start start 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f527fc640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f480a2800 0x7f6f480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 0x561dea4477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea44c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61f91640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f480a2800 0x7f6f480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea44cf00 0x561dea44f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f665e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561dea3ac160 con 0x561dea44cf00 2024-03-19T00:40:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f665e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561dea354e10 con 0x561dea4473a0 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f665e7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561dea3a8200 con 0x561dea447d90 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f527fc640 1 -- 172.21.15.7:0/261525267 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f4c0f1030 con 0x7f6f5c101930 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 0x561dea4477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 0x561dea4477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38386/0 (socket says 172.21.15.7:38386) 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 -- 172.21.15.7:0/586861489 learned_addr learned my addr 172.21.15.7:0/586861489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61f91640 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea44cf00 0x561dea44f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea44c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 -- 172.21.15.7:0/586861489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 msgr2=0x561dea44c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea44c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 -- 172.21.15.7:0/586861489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea44cf00 msgr2=0x561dea44f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea44cf00 0x561dea44f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 -- 172.21.15.7:0/586861489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dea1887e0 con 0x561dea4473a0 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea44c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61f91640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f480a2800 0x7f6f480a4cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f54005fd0 tx=0x7f6f5400feb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.658+0000 7f6f61790640 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 0x561dea4477c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009ca0 tx=0x7f6f58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d5da7f80a214fb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f477fe640 1 -- 172.21.15.7:0/586861489 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58009e50 con 0x561dea4473a0 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f477fe640 1 -- 172.21.15.7:0/586861489 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f580035f0 con 0x561dea4473a0 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f477fe640 1 -- 172.21.15.7:0/586861489 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58019610 con 0x561dea4473a0 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 -- 172.21.15.7:0/586861489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 msgr2=0x561dea4477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 0x561dea4477c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009ca0 tx=0x7f6f58011810 comp rx=0 tx=0).stop 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 -- 172.21.15.7:0/586861489 shutdown_connections 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea44cf00 0x561dea44f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea44c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 --2- 172.21.15.7:0/586861489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea4473a0 0x561dea4477c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 -- 172.21.15.7:0/586861489 >> 172.21.15.7:0/586861489 conn(0x561dea442910 msgr2=0x561dea442d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 -- 172.21.15.7:0/586861489 shutdown_connections 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.659+0000 7f6f665e7740 1 -- 172.21.15.7:0/586861489 wait complete. 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 -- start start 2024-03-19T00:40:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 Processor -- start 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea4473a0 0x561dea451770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 0x561dea451cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea458cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561dea44feb0 con 0x561dea447d90 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561dea3681a0 con 0x561dea44cf00 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561dea450030 con 0x561dea4473a0 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f60f8f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 0x561dea451cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f665e7740 10 client.0 _ll_register_callbacks cb 0x561dea0bfc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f60f8f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 0x561dea451cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57404/0 (socket says 172.21.15.7:57404) 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f60f8f640 1 -- 172.21.15.7:0/2079819248 learned_addr learned my addr 172.21.15.7:0/2079819248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f61790640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea4473a0 0x561dea451770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.660+0000 7f6f61f91640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea458cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f60f8f640 1 -- 172.21.15.7:0/2079819248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea4473a0 msgr2=0x561dea451770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea4473a0 0x561dea451770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f60f8f640 1 -- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 msgr2=0x561dea458cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea458cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f60f8f640 1 -- 172.21.15.7:0/2079819248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dea1e8f70 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 0x561dea451cb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c005ab0 tx=0x7f6f4c0bd030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bd3588931a0ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f44ff9640 1 -- 172.21.15.7:0/2079819248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0d6060 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f44ff9640 1 -- 172.21.15.7:0/2079819248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f4c0054b0 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f580035f0 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f44ff9640 1 -- 172.21.15.7:0/2079819248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f4c0056b0 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f665e7740 10 client.16080 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561dea080170 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.661+0000 7f6f44ff9640 1 -- 172.21.15.7:0/2079819248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f580035f0 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f44ff9640 1 -- 172.21.15.7:0/2079819248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6f4c0a6ba0 con 0x561dea447d90 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f665e7740 10 client.16080 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f665e7740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f665e7740 20 client.16080 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f665e7740 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea45d770 0x561dea45fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f665e7740 4 client.16080 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dea3b96a0 con 0x561dea45d770 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f61790640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea45d770 0x561dea45fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f61790640 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea45d770 0x561dea45fbd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd57c4bb253a3ef server_cookie=20331b51f5c13e62 in_seq=0 out_seq=0 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.662+0000 7f6f44ff9640 10 client.16080 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f44ff9640 1 -- 172.21.15.7:0/2079819248 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561dea3b96a0 con 0x561dea45d770 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f44ff9640 10 client.16080 handle_command_reply: tid=0 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 client.16080 shutdown 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 msgr2=0x561dea451cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 0x561dea451cb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c005ab0 tx=0x7f6f4c0bd030 comp rx=0 tx=0).stop 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 shutdown_connections 2024-03-19T00:40:40.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea458cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea45d770 0x561dea45fbd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea447d90 0x561dea451cb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f44ff9640 0 client.16080 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 --2- 172.21.15.7:0/2079819248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea4473a0 0x561dea451770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 >> 172.21.15.7:0/2079819248 conn(0x561dea442910 msgr2=0x561dea441b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 shutdown_connections 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 1 -- 172.21.15.7:0/2079819248 wait complete. 2024-03-19T00:40:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.663+0000 7f6f665e7740 20 client.16080 trim_cache size 0 max 16384 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 Processor -- start 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 -- start start 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea446280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea447d90 0x561dea458cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea44cf00 0x561dea464490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1aca70 con 0x561dea4473a0 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1aa340 con 0x561dea447d90 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.681+0000 7f6f665e7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1a8190 con 0x561dea44cf00 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea446280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea446280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57406/0 (socket says 172.21.15.7:57406) 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 -- 172.21.15.7:0/1462035160 learned_addr learned my addr 172.21.15.7:0/1462035160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea447d90 0x561dea458cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61f91640 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea44cf00 0x561dea464490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 -- 172.21.15.7:0/1462035160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea44cf00 msgr2=0x561dea464490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea44cf00 0x561dea464490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 -- 172.21.15.7:0/1462035160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea447d90 msgr2=0x561dea458cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea447d90 0x561dea458cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 -- 172.21.15.7:0/1462035160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dea448170 con 0x561dea4473a0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f61790640 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea446280 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6f58003880 tx=0x7f6f58005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfad9cb5df8d5f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f46ffd640 1 -- 172.21.15.7:0/1462035160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580035f0 con 0x561dea4473a0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f46ffd640 1 -- 172.21.15.7:0/1462035160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f580054f0 con 0x561dea4473a0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.682+0000 7f6f46ffd640 1 -- 172.21.15.7:0/1462035160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58026480 con 0x561dea4473a0 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 -- 172.21.15.7:0/1462035160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 msgr2=0x561dea446280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea446280 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6f58003880 tx=0x7f6f58005850 comp rx=0 tx=0).stop 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 -- 172.21.15.7:0/1462035160 shutdown_connections 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea44cf00 0x561dea464490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea447d90 0x561dea458cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 --2- 172.21.15.7:0/1462035160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea446280 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 -- 172.21.15.7:0/1462035160 >> 172.21.15.7:0/1462035160 conn(0x561dea442910 msgr2=0x561dea445b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 -- 172.21.15.7:0/1462035160 shutdown_connections 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 -- 172.21.15.7:0/1462035160 wait complete. 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 -- start start 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.683+0000 7f6f665e7740 1 Processor -- start 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea4587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea456e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea457340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1a8800 con 0x561dea4473a0 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1a9c70 con 0x561dea44cf00 2024-03-19T00:40:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1a8ed0 con 0x561dea447d90 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 10 client.0 _ll_register_callbacks cb 0x7f6f4c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea4587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea4587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57408/0 (socket says 172.21.15.7:57408) 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 -- 172.21.15.7:0/3957876145 learned_addr learned my addr 172.21.15.7:0/3957876145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61f91640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea457340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea456e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 -- 172.21.15.7:0/3957876145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 msgr2=0x561dea456e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea456e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 -- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 msgr2=0x561dea457340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea457340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 -- 172.21.15.7:0/3957876145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dea457880 con 0x561dea4473a0 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f61790640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea4587f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6f58011850 tx=0x7f6f58011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7432002a2c8b4992 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f44ff9640 1 -- 172.21.15.7:0/3957876145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5803a060 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f44ff9640 1 -- 172.21.15.7:0/3957876145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f580050a0 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.684+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f580054f0 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f44ff9640 1 -- 172.21.15.7:0/3957876145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f580052a0 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 10 client.16086 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561dea4c47a0 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f44ff9640 1 -- 172.21.15.7:0/3957876145 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f580054f0 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f44ff9640 1 -- 172.21.15.7:0/3957876145 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6f5802d040 con 0x561dea4473a0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 10 client.16086 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 20 client.16086 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea4c4f60 0x561dea4c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 4 client.16086 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561dea3b39f0 con 0x561dea4c4f60 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea4c4f60 0x561dea4c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f60f8f640 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea4c4f60 0x561dea4c7380 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747c16b0ab7ca03f server_cookie=f1fb9ff1b5989aa8 in_seq=0 out_seq=0 2024-03-19T00:40:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.685+0000 7f6f44ff9640 10 client.16086 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f44ff9640 1 -- 172.21.15.7:0/3957876145 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561dea3b39f0 con 0x561dea4c4f60 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f44ff9640 10 client.16086 handle_command_reply: tid=0 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 client.16086 shutdown 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 msgr2=0x561dea4587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea4587f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6f58011850 tx=0x7f6f58011da0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 shutdown_connections 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dea44cf00 0x561dea457340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561dea447d90 0x561dea456e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561dea4c4f60 0x561dea4c7380 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 --2- 172.21.15.7:0/3957876145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dea4473a0 0x561dea4587f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f44ff9640 0 client.16086 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 >> 172.21.15.7:0/3957876145 conn(0x561dea442910 msgr2=0x561dea446400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 shutdown_connections 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 1 -- 172.21.15.7:0/3957876145 wait complete. 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.687+0000 7f6f665e7740 20 client.16086 trim_cache size 0 max 16384 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "e24618c0-2ae1-475c-a444-7a5a45128e05", 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:40:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:40:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f480a2800 msgr2=0x7f6f480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f480a2800 0x7f6f480a4cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f54005fd0 tx=0x7f6f5400feb0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 msgr2=0x7f6f5c19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 0x7f6f5c19c780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c006010 tx=0x7f6f4c005e40 comp rx=0 tx=0).stop 2024-03-19T00:40:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 shutdown_connections 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f480a2800 0x7f6f480a4cc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5c1022f0 0x7f6f5c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c101930 0x7f6f5c19c780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 --2- 172.21.15.7:0/261525267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c100740 0x7f6f5c06aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 >> 172.21.15.7:0/261525267 conn(0x7f6f5c0fbcb0 msgr2=0x7f6f5c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 shutdown_connections 2024-03-19T00:40:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.688+0000 7f6f639f4640 1 -- 172.21.15.7:0/261525267 wait complete. 2024-03-19T00:40:40.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:40:40.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:40:40.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 -- 172.21.15.7:0/2876177421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0101830 msgr2=0x7fadd0101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 --2- 172.21.15.7:0/2876177421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0101830 0x7fadd0101c50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009900 tx=0x7fadc002bc30 comp rx=0 tx=0).stop 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 -- 172.21.15.7:0/2876177421 shutdown_connections 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 --2- 172.21.15.7:0/2876177421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd01033e0 0x7fadd0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 --2- 172.21.15.7:0/2876177421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadd0102a20 0x7fadd0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 --2- 172.21.15.7:0/2876177421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0101830 0x7fadd0101c50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 -- 172.21.15.7:0/2876177421 >> 172.21.15.7:0/2876177421 conn(0x7fadd00fcda0 msgr2=0x7fadd00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 -- 172.21.15.7:0/2876177421 shutdown_connections 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.803+0000 7fadd5709640 1 -- 172.21.15.7:0/2876177421 wait complete. 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 Processor -- start 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 -- start start 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadd0101830 0x7fadd0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 0x7fadd0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd01033e0 0x7fadd019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010d360 con 0x7fadd0102a20 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010d1e0 con 0x7fadd0101830 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadd5709640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010d4e0 con 0x7fadd01033e0 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 0x7fadd0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadd0101830 0x7fadd0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 0x7fadd0197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57432/0 (socket says 172.21.15.7:57432) 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 -- 172.21.15.7:0/802244246 learned_addr learned my addr 172.21.15.7:0/802244246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd01033e0 0x7fadd019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 -- 172.21.15.7:0/802244246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd01033e0 msgr2=0x7fadd019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd01033e0 0x7fadd019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 -- 172.21.15.7:0/802244246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadd0101830 msgr2=0x7fadd0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadd0101830 0x7fadd0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.804+0000 7fadc7fff640 1 -- 172.21.15.7:0/802244246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd019f6b0 con 0x7fadd0102a20 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.805+0000 7fadc7fff640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 0x7fadd0197fd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e9f0 tx=0x7fadb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.805+0000 7fadccff9640 1 -- 172.21.15.7:0/802244246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800cd80 con 0x7fadd0102a20 2024-03-19T00:40:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.805+0000 7fadccff9640 1 -- 172.21.15.7:0/802244246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadb80108e0 con 0x7fadd0102a20 2024-03-19T00:40:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.805+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd019f9a0 con 0x7fadd0102a20 2024-03-19T00:40:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.805+0000 7fadccff9640 1 -- 172.21.15.7:0/802244246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8018920 con 0x7fadd0102a20 2024-03-19T00:40:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.805+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd019fef0 con 0x7fadd0102a20 2024-03-19T00:40:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadccff9640 1 -- 172.21.15.7:0/802244246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fadb8010a90 con 0x7fadd0102a20 2024-03-19T00:40:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadd83d4740 1 Processor -- start 2024-03-19T00:40:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadd83d4740 1 -- start start 2024-03-19T00:40:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadccff9640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fadbc0a2800 0x7fadbc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadccff9640 1 -- 172.21.15.7:0/802244246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fadb80cf0b0 con 0x7fadd0102a20 2024-03-19T00:40:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fadbc0a2800 0x7fadbc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.807+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5888e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c588df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c5890a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadd83d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b3c57ed500 con 0x55b3c588e5b0 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadd83d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b3c5795be0 con 0x55b3c5888a50 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadd83d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3c57e95a0 con 0x55b3c5889440 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5888e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5888e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38428/0 (socket says 172.21.15.7:38428) 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c588df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadceffd640 1 -- 172.21.15.7:0/2845158647 learned_addr learned my addr 172.21.15.7:0/2845158647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c5890a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fadbc0a2800 0x7fadbc0a4cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005fd0 tx=0x7fadc8011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 -- 172.21.15.7:0/2845158647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 msgr2=0x55b3c5888e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5888e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 -- 172.21.15.7:0/2845158647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 msgr2=0x55b3c5890a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c5890a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 -- 172.21.15.7:0/2845158647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c57b7000 con 0x55b3c5889440 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadceffd640 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5888e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c588df40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fadb80ccbc0 tx=0x7fadb8010ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf990ce0b411a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc57fa640 1 -- 172.21.15.7:0/2845158647 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80a3e90 con 0x55b3c5889440 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc57fa640 1 -- 172.21.15.7:0/2845158647 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadb8099aa0 con 0x55b3c5889440 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.808+0000 7fadc57fa640 1 -- 172.21.15.7:0/2845158647 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80974f0 con 0x55b3c5889440 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 -- 172.21.15.7:0/2845158647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 msgr2=0x55b3c588df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c588df40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fadb80ccbc0 tx=0x7fadb8010ea0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 -- 172.21.15.7:0/2845158647 shutdown_connections 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c5890a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c588df40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2845158647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5888e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 -- 172.21.15.7:0/2845158647 >> 172.21.15.7:0/2845158647 conn(0x55b3c5883fc0 msgr2=0x55b3c58843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 -- 172.21.15.7:0/2845158647 shutdown_connections 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 -- 172.21.15.7:0/2845158647 wait complete. 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.809+0000 7fadd83d4740 1 -- start start 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 Processor -- start 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5892ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5893410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c589a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b3c5891500 con 0x55b3c588e5b0 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b3c57a9160 con 0x55b3c5888a50 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3c5891680 con 0x55b3c5889440 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadd83d4740 10 client.0 _ll_register_callbacks cb 0x55b3c57e5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadcf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c589a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5892ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5893410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadcf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c589a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57454/0 (socket says 172.21.15.7:57454) 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadceffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5892ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38430/0 (socket says 172.21.15.7:38430) 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadcf7fe640 1 -- 172.21.15.7:0/156196484 learned_addr learned my addr 172.21.15.7:0/156196484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 -- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 msgr2=0x55b3c5892ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5892ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 -- 172.21.15.7:0/156196484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 msgr2=0x55b3c589a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c589a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 -- 172.21.15.7:0/156196484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c55cb7d0 con 0x55b3c5889440 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c589a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadceffd640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5892ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadc7fff640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5893410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e9c0 tx=0x7fadb80973d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6e564b420df531 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.810+0000 7fadb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadb67fc640 1 -- 172.21.15.7:0/156196484 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80b9060 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadb67fc640 1 -- 172.21.15.7:0/156196484 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fadb80b42b0 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb8099aa0 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadb67fc640 1 -- 172.21.15.7:0/156196484 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadb80ab050 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 10 client.25858 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3c55013f0 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadb67fc640 1 -- 172.21.15.7:0/156196484 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadb8099aa0 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadb67fc640 1 -- 172.21.15.7:0/156196484 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fadb80afb60 con 0x55b3c5889440 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 10 client.25858 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 20 client.25858 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c589e940 0x55b3c58a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 4 client.25858 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3c57faa40 con 0x55b3c589e940 2024-03-19T00:40:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.811+0000 7fadceffd640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c589e940 0x55b3c58a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.812+0000 7fadb67fc640 10 client.25858 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.812+0000 7fadceffd640 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c589e940 0x55b3c58a0d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f50757b398991c server_cookie=5e2d80da02183437 in_seq=0 out_seq=0 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadb67fc640 1 -- 172.21.15.7:0/156196484 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b3c57faa40 con 0x55b3c589e940 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadb67fc640 10 client.25858 handle_command_reply: tid=0 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 client.25858 shutdown 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 msgr2=0x55b3c5893410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5893410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e9c0 tx=0x7fadb80973d0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 shutdown_connections 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c589e940 0x55b3c58a0d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c589a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5893410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 --2- 172.21.15.7:0/156196484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5888a50 0x55b3c5892ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadb67fc640 0 client.25858 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 >> 172.21.15.7:0/156196484 conn(0x55b3c5883fc0 msgr2=0x55b3c58831e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 shutdown_connections 2024-03-19T00:40:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 1 -- 172.21.15.7:0/156196484 wait complete. 2024-03-19T00:40:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.813+0000 7fadd83d4740 20 client.25858 trim_cache size 0 max 16384 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 Processor -- start 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 -- start start 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5888a50 0x55b3c5891a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5889440 0x55b3c5883060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c58a5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01acbe0 con 0x55b3c588e5b0 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01aa4b0 con 0x55b3c5889440 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadd83d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01a8300 con 0x55b3c5888a50 2024-03-19T00:40:40.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.830+0000 7fadcf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c58a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c58a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57458/0 (socket says 172.21.15.7:57458) 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 -- 172.21.15.7:0/3904347445 learned_addr learned my addr 172.21.15.7:0/3904347445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadc7fff640 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5889440 0x55b3c5883060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadceffd640 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5888a50 0x55b3c5891a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 -- 172.21.15.7:0/3904347445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5888a50 msgr2=0x55b3c5891a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5888a50 0x55b3c5891a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 -- 172.21.15.7:0/3904347445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5889440 msgr2=0x55b3c5883060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5889440 0x55b3c5883060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 -- 172.21.15.7:0/3904347445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c588afc0 con 0x55b3c588e5b0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c58a5550 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fadc801dc50 tx=0x7fadc801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b71465ec5ede0b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadc4ff9640 1 -- 172.21.15.7:0/3904347445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc802d070 con 0x55b3c588e5b0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadc4ff9640 1 -- 172.21.15.7:0/3904347445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadc8019050 con 0x55b3c588e5b0 2024-03-19T00:40:40.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.831+0000 7fadc4ff9640 1 -- 172.21.15.7:0/3904347445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8017040 con 0x55b3c588e5b0 2024-03-19T00:40:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- 172.21.15.7:0/3904347445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 msgr2=0x55b3c58a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c58a5550 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fadc801dc50 tx=0x7fadc801d000 comp rx=0 tx=0).stop 2024-03-19T00:40:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- 172.21.15.7:0/3904347445 shutdown_connections 2024-03-19T00:40:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c588e5b0 0x55b3c58a5550 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c5889440 0x55b3c5883060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- 172.21.15.7:0/3904347445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5888a50 0x55b3c5891a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- 172.21.15.7:0/3904347445 >> 172.21.15.7:0/3904347445 conn(0x55b3c5883fc0 msgr2=0x55b3c5882950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- 172.21.15.7:0/3904347445 shutdown_connections 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- 172.21.15.7:0/3904347445 wait complete. 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- start start 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 Processor -- start 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 0x55b3c5900220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5900760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c588e5b0 0x55b3c59077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01a8970 con 0x55b3c5888a50 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01a9de0 con 0x55b3c588e5b0 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01a9040 con 0x55b3c5889440 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 0x55b3c5900220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadd83d4740 10 client.0 _ll_register_callbacks cb 0x7fadc8009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.832+0000 7fadceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 0x55b3c5900220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57462/0 (socket says 172.21.15.7:57462) 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 -- 172.21.15.7:0/2304928367 learned_addr learned my addr 172.21.15.7:0/2304928367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadcf7fe640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c588e5b0 0x55b3c59077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5900760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 -- 172.21.15.7:0/2304928367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 msgr2=0x55b3c5900760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5900760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 -- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c588e5b0 msgr2=0x55b3c59077a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c588e5b0 0x55b3c59077a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 -- 172.21.15.7:0/2304928367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c54c10f0 con 0x55b3c5888a50 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5900760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadceffd640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 0x55b3c5900220 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009340 tx=0x7fadc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89541651b9bc8077 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadb67fc640 1 -- 172.21.15.7:0/2304928367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0050020 con 0x55b3c5888a50 2024-03-19T00:40:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadb67fc640 1 -- 172.21.15.7:0/2304928367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fadc0005040 con 0x55b3c5888a50 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc8019050 con 0x55b3c5888a50 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadb67fc640 1 -- 172.21.15.7:0/2304928367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadc0058050 con 0x55b3c5888a50 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadd83d4740 10 client.16104 fetch_fsmap learned FSMap version 26 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3c5908f40 con 0x55b3c5888a50 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.833+0000 7fadb67fc640 1 -- 172.21.15.7:0/2304928367 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadc8019050 con 0x55b3c5888a50 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadb67fc640 1 -- 172.21.15.7:0/2304928367 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fadc00035f0 con 0x55b3c5888a50 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 10 client.16104 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 20 client.16104 populate_metadata read hostname 'smithi007' 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c5909930 0x55b3c590bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 4 client.16104 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3c57f4d90 con 0x55b3c5909930 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c5909930 0x55b3c590bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:40:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadc7fff640 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c5909930 0x55b3c590bd50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5821e76d533b4f27 server_cookie=c28d2e05fb5fb7c9 in_seq=0 out_seq=0 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadb67fc640 10 client.16104 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadb67fc640 1 -- 172.21.15.7:0/2304928367 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b3c57f4d90 con 0x55b3c5909930 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadb67fc640 10 client.16104 handle_command_reply: tid=0 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 1 client.16104 shutdown 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 msgr2=0x55b3c5900220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.834+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 0x55b3c5900220 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009340 tx=0x7fadc0004270 comp rx=0 tx=0).stop 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 shutdown_connections 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c588e5b0 0x55b3c59077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3c5889440 0x55b3c5900760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3c5909930 0x55b3c590bd50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 --2- 172.21.15.7:0/2304928367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c5888a50 0x55b3c5900220 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadb67fc640 0 client.16104 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 >> 172.21.15.7:0/2304928367 conn(0x55b3c5883fc0 msgr2=0x55b3c5887ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 shutdown_connections 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 1 -- 172.21.15.7:0/2304928367 wait complete. 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd83d4740 20 client.16104 trim_cache size 0 max 16384 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "e24618c0-2ae1-475c-a444-7a5a45128e05": { 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "e24618c0-2ae1-475c-a444-7a5a45128e05", 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:40:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:40:40.844 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:40:40.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fadbc0a2800 msgr2=0x7fadbc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fadbc0a2800 0x7fadbc0a4cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005fd0 tx=0x7fadc8011810 comp rx=0 tx=0).stop 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 msgr2=0x7fadd0197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 0x7fadd0197fd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e9f0 tx=0x7fadb800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 shutdown_connections 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fadbc0a2800 0x7fadbc0a4cc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadd01033e0 0x7fadd019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd0102a20 0x7fadd0197fd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 --2- 172.21.15.7:0/802244246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadd0101830 0x7fadd0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 >> 172.21.15.7:0/802244246 conn(0x7fadd00fcda0 msgr2=0x7fadd0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 shutdown_connections 2024-03-19T00:40:40.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:40:40.835+0000 7fadd5709640 1 -- 172.21.15.7:0/802244246 wait complete. 2024-03-19T00:40:40.847 INFO:tasks.cephfs.filesystem:scrub status for tag:e24618c0-2ae1-475c-a444-7a5a45128e05 - {'path': '/', 'tag': 'e24618c0-2ae1-475c-a444-7a5a45128e05', 'options': 'recursive,force'} 2024-03-19T00:40:41.817 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: pgmap v433: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:41.817 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:40:41.817 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.817 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:40:41.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:40:41.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.818 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:41 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: pgmap v433: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:41 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: pgmap v433: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:41 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:40:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:43 smithi044 ceph-mon[66366]: pgmap v434: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:43.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:43 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:40:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:43 smithi007 ceph-mon[62854]: pgmap v434: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:43 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:40:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:43 smithi204 ceph-mon[66572]: pgmap v434: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:40:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:43 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:40:44.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:40:44.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:40:44.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:40:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:40:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:40:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:40:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:40:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:40:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:40:45.859 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:45 smithi044 ceph-mon[66366]: pgmap v435: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:40:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:45 smithi007 ceph-mon[62854]: pgmap v435: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:40:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:45 smithi204 ceph-mon[66572]: pgmap v435: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:40:46.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:40:46.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:40:46.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:40:46.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:40:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:40:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:40:47.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:47 smithi044 ceph-mon[66366]: pgmap v436: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:40:47.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:47 smithi007 ceph-mon[62854]: pgmap v436: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:40:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:47 smithi204 ceph-mon[66572]: pgmap v436: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:40:49.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:49 smithi044 ceph-mon[66366]: pgmap v437: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:40:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:49 smithi007 ceph-mon[62854]: pgmap v437: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:40:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:49 smithi204 ceph-mon[66572]: pgmap v437: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:40:51.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:51 smithi044 ceph-mon[66366]: pgmap v438: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:40:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:51 smithi007 ceph-mon[62854]: pgmap v438: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:40:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:51 smithi204 ceph-mon[66572]: pgmap v438: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:40:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:53 smithi044 ceph-mon[66366]: pgmap v439: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:40:53.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:53 smithi007 ceph-mon[62854]: pgmap v439: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:40:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:53 smithi204 ceph-mon[66572]: pgmap v439: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:40:55.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:55 smithi044 ceph-mon[66366]: pgmap v440: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:55.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:55 smithi007 ceph-mon[62854]: pgmap v440: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:56.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:55 smithi204 ceph-mon[66572]: pgmap v440: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:40:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:57 smithi044 ceph-mon[66366]: pgmap v441: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:57.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:57 smithi007 ceph-mon[62854]: pgmap v441: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:58.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:57 smithi204 ceph-mon[66572]: pgmap v441: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:59.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:40:59 smithi044 ceph-mon[66366]: pgmap v442: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:40:59.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:40:59 smithi007 ceph-mon[62854]: pgmap v442: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:00.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:40:59 smithi204 ceph-mon[66572]: pgmap v442: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:01.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:01 smithi044 ceph-mon[66366]: pgmap v443: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:01.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:01 smithi007 ceph-mon[62854]: pgmap v443: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:02.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:01 smithi204 ceph-mon[66572]: pgmap v443: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:03 smithi044 ceph-mon[66366]: pgmap v444: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:41:03.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:03 smithi007 ceph-mon[62854]: pgmap v444: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:41:04.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:03 smithi204 ceph-mon[66572]: pgmap v444: 129 pgs: 129 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:41:05.859 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:05 smithi044 ceph-mon[66366]: pgmap v445: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:41:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:05 smithi007 ceph-mon[62854]: pgmap v445: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:41:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:05 smithi204 ceph-mon[66572]: pgmap v445: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:41:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:07 smithi044 ceph-mon[66366]: pgmap v446: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:07.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:07 smithi007 ceph-mon[62854]: pgmap v446: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:07 smithi204 ceph-mon[66572]: pgmap v446: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:09 smithi044 ceph-mon[66366]: pgmap v447: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:09 smithi007 ceph-mon[62854]: pgmap v447: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:09 smithi204 ceph-mon[66572]: pgmap v447: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:10.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:41:10.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:41:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 -- 172.21.15.7:0/2093033483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101050 msgr2=0x7f57501014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 --2- 172.21.15.7:0/2093033483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101050 0x7f57501014d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5744009900 tx=0x7f574402bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 -- 172.21.15.7:0/2093033483 shutdown_connections 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 --2- 172.21.15.7:0/2093033483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5750101a10 0x7f5750108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 --2- 172.21.15.7:0/2093033483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101050 0x7f57501014d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 --2- 172.21.15.7:0/2093033483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57500ffe70 0x7f5750100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 -- 172.21.15.7:0/2093033483 >> 172.21.15.7:0/2093033483 conn(0x7f57500fb430 msgr2=0x7f57500fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 -- 172.21.15.7:0/2093033483 shutdown_connections 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.965+0000 7f5757634640 1 -- 172.21.15.7:0/2093033483 wait complete. 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 Processor -- start 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 -- start start 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57500ffe70 0x7f5750197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750101050 0x7f5750197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 0x7f575019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f575010b170 con 0x7f5750101a10 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f575010aff0 con 0x7f5750101050 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5757634640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f575010b2f0 con 0x7f57500ffe70 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5755bd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 0x7f575019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5755bd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 0x7f575019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53760/0 (socket says 172.21.15.7:53760) 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5755bd1640 1 -- 172.21.15.7:0/3669476380 learned_addr learned my addr 172.21.15.7:0/3669476380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f5754bcf640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750101050 0x7f5750197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.966+0000 7f57553d0640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57500ffe70 0x7f5750197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5755bd1640 1 -- 172.21.15.7:0/3669476380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57500ffe70 msgr2=0x7f5750197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5755bd1640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57500ffe70 0x7f5750197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5755bd1640 1 -- 172.21.15.7:0/3669476380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750101050 msgr2=0x7f5750197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5755bd1640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750101050 0x7f5750197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5755bd1640 1 -- 172.21.15.7:0/3669476380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575019f6a0 con 0x7f5750101a10 2024-03-19T00:41:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5755bd1640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 0x7f575019f030 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f574800eb10 tx=0x7f574800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f574e7fc640 1 -- 172.21.15.7:0/3669476380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748004b20 con 0x7f5750101a10 2024-03-19T00:41:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f574e7fc640 1 -- 172.21.15.7:0/3669476380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5748004cc0 con 0x7f5750101a10 2024-03-19T00:41:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575019f990 con 0x7f5750101a10 2024-03-19T00:41:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f574e7fc640 1 -- 172.21.15.7:0/3669476380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748010690 con 0x7f5750101a10 2024-03-19T00:41:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.967+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575019fe60 con 0x7f5750101a10 2024-03-19T00:41:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.969+0000 7f574e7fc640 1 -- 172.21.15.7:0/3669476380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5748005020 con 0x7f5750101a10 2024-03-19T00:41:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 Processor -- start 2024-03-19T00:41:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 -- start start 2024-03-19T00:41:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57380a2800 0x7f57380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f1991e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f1996f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f1999a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576f18f6500 con 0x5576f1992440 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5576f189ebe0 con 0x5576f1991a50 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f575a219740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5576f18f25a0 con 0x5576f19975b0 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f5754bcf640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57380a2800 0x7f57380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f57553d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f1991e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f57553d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f1991e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57372/0 (socket says 172.21.15.7:57372) 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f57553d0640 1 -- 172.21.15.7:0/2512046756 learned_addr learned my addr 172.21.15.7:0/2512046756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f5755bd1640 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f1999a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f5754bcf640 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f1996f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.970+0000 7f574e7fc640 1 -- 172.21.15.7:0/3669476380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f57480cf3b0 con 0x7f5750101a10 2024-03-19T00:41:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57553d0640 1 -- 172.21.15.7:0/2512046756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 msgr2=0x5576f1999a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57553d0640 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f1999a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57553d0640 1 -- 172.21.15.7:0/2512046756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 msgr2=0x5576f1996f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57553d0640 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f1996f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57553d0640 1 -- 172.21.15.7:0/2512046756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f18c0000 con 0x5576f1991a50 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f5754bcf640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57380a2800 0x7f57380a4cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5744009230 tx=0x7f5744005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57553d0640 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f1991e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f574000ad00 tx=0x7f5740005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5db90c84c90e921 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57377fe640 1 -- 172.21.15.7:0/2512046756 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740005020 con 0x5576f1991a50 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57377fe640 1 -- 172.21.15.7:0/2512046756 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57400047a0 con 0x5576f1991a50 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.971+0000 7f57377fe640 1 -- 172.21.15.7:0/2512046756 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57400186c0 con 0x5576f1991a50 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 -- 172.21.15.7:0/2512046756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 msgr2=0x5576f1991e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f1991e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f574000ad00 tx=0x7f5740005f90 comp rx=0 tx=0).stop 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 -- 172.21.15.7:0/2512046756 shutdown_connections 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f1999a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f1996f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 --2- 172.21.15.7:0/2512046756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f1991e70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 -- 172.21.15.7:0/2512046756 >> 172.21.15.7:0/2512046756 conn(0x5576f198cfc0 msgr2=0x5576f198d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 -- 172.21.15.7:0/2512046756 shutdown_connections 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 -- 172.21.15.7:0/2512046756 wait complete. 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 -- start start 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.972+0000 7f575a219740 1 Processor -- start 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f199be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f199c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576f199a560 con 0x5576f1992440 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5576f18b2160 con 0x5576f1991a50 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5576f199a6e0 con 0x5576f19975b0 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f199c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 10 client.0 _ll_register_callbacks cb 0x5576f18eea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:10.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f199c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53782/0 (socket says 172.21.15.7:53782) 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 -- 172.21.15.7:0/813439011 learned_addr learned my addr 172.21.15.7:0/813439011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f57553d0640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f199be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5755bd1640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 -- 172.21.15.7:0/813439011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 msgr2=0x5576f19a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 -- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 msgr2=0x5576f199be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f199be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 -- 172.21.15.7:0/813439011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f16d47d0 con 0x5576f1992440 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5755bd1640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f57553d0640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f199be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5754bcf640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f199c390 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5744035000 tx=0x7f5744005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff4f858d7e619a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5734ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5734ff9640 1 -- 172.21.15.7:0/813439011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744055060 con 0x5576f1992440 2024-03-19T00:41:10.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f5734ff9640 1 -- 172.21.15.7:0/813439011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5744004990 con 0x5576f1992440 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.973+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57400047a0 con 0x5576f1992440 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f5734ff9640 1 -- 172.21.15.7:0/813439011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f574405c050 con 0x5576f1992440 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 10 client.16116 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576f160a3f0 con 0x5576f1992440 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f5734ff9640 1 -- 172.21.15.7:0/813439011 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57400047a0 con 0x5576f1992440 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f5734ff9640 1 -- 172.21.15.7:0/813439011 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f5744003cd0 con 0x5576f1992440 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 20 client.16116 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f19a7d70 0x5576f19aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 4 client.16116 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f1903a40 con 0x5576f19a7d70 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.974+0000 7f57553d0640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f19a7d70 0x5576f19aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f57553d0640 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f19a7d70 0x5576f19aa1f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5efc0cda6faa1e5 server_cookie=e1797a64d6c287f7 in_seq=0 out_seq=0 2024-03-19T00:41:10.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f5734ff9640 10 client.16116 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f5734ff9640 1 -- 172.21.15.7:0/813439011 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5576f1903a40 con 0x5576f19a7d70 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f5734ff9640 10 client.16116 handle_command_reply: tid=0 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f575a219740 1 client.16116 shutdown 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 msgr2=0x5576f199c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.975+0000 7f575a219740 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f199c390 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5744035000 tx=0x7f5744005fd0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 shutdown_connections 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f19a7d70 0x5576f19aa1f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f199c390 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 --2- 172.21.15.7:0/813439011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1991a50 0x5576f199be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f5734ff9640 0 client.16116 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 >> 172.21.15.7:0/813439011 conn(0x5576f198cfc0 msgr2=0x5576f198c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 shutdown_connections 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 1 -- 172.21.15.7:0/813439011 wait complete. 2024-03-19T00:41:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.976+0000 7f575a219740 20 client.16116 trim_cache size 0 max 16384 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.993+0000 7f575a219740 1 Processor -- start 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.993+0000 7f575a219740 1 -- start start 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f575a219740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f1991a50 0x5576f1990930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f575a219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f19a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f575a219740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f19975b0 0x5576f19aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f575a219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57501acaf0 con 0x5576f1992440 2024-03-19T00:41:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f575a219740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57501aa3c0 con 0x5576f19975b0 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f575a219740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a8210 con 0x5576f1991a50 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f19a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f19a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53798/0 (socket says 172.21.15.7:53798) 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 -- 172.21.15.7:0/889465543 learned_addr learned my addr 172.21.15.7:0/889465543 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5755bd1640 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f19975b0 0x5576f19aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f57553d0640 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f1991a50 0x5576f1990930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 -- 172.21.15.7:0/889465543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f1991a50 msgr2=0x5576f1990930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f1991a50 0x5576f1990930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 -- 172.21.15.7:0/889465543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f19975b0 msgr2=0x5576f19aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f19975b0 0x5576f19aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 -- 172.21.15.7:0/889465543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f15ca0f0 con 0x5576f1992440 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5754bcf640 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f19a3390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5744005770 tx=0x7f5744005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b69b683050d089 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5736ffd640 1 -- 172.21.15.7:0/889465543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744055040 con 0x5576f1992440 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5736ffd640 1 -- 172.21.15.7:0/889465543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57440043d0 con 0x5576f1992440 2024-03-19T00:41:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.994+0000 7f5736ffd640 1 -- 172.21.15.7:0/889465543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744034440 con 0x5576f1992440 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 -- 172.21.15.7:0/889465543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 msgr2=0x5576f19a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f19a3390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5744005770 tx=0x7f5744005950 comp rx=0 tx=0).stop 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 -- 172.21.15.7:0/889465543 shutdown_connections 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f19975b0 0x5576f19aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1992440 0x5576f19a3390 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 --2- 172.21.15.7:0/889465543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f1991a50 0x5576f1990930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 -- 172.21.15.7:0/889465543 >> 172.21.15.7:0/889465543 conn(0x5576f198cfc0 msgr2=0x5576f1990220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 -- 172.21.15.7:0/889465543 shutdown_connections 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 -- 172.21.15.7:0/889465543 wait complete. 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 -- start start 2024-03-19T00:41:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.995+0000 7f575a219740 1 Processor -- start 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 0x5576f19a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1992440 0x5576f19a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a8880 con 0x5576f1991a50 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a9cf0 con 0x5576f1992440 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a8f50 con 0x5576f19975b0 2024-03-19T00:41:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f575a219740 10 client.0 _ll_register_callbacks cb 0x7f5744009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 0x5576f19a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 0x5576f19a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53814/0 (socket says 172.21.15.7:53814) 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 -- 172.21.15.7:0/1955623018 learned_addr learned my addr 172.21.15.7:0/1955623018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f5755bd1640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f5754bcf640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1992440 0x5576f19a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 -- 172.21.15.7:0/1955623018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 msgr2=0x5576f19a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 -- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1992440 msgr2=0x5576f19a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1992440 0x5576f19a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 -- 172.21.15.7:0/1955623018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f19a29b0 con 0x5576f1991a50 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f5754bcf640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1992440 0x5576f19a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f57553d0640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 0x5576f19a0ec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f574000a810 tx=0x7f5740004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a123d4d3eb17f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f5734ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.996+0000 7f5734ff9640 1 -- 172.21.15.7:0/1955623018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574000adf0 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f5734ff9640 1 -- 172.21.15.7:0/1955623018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f574002c640 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57440043d0 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f5734ff9640 1 -- 172.21.15.7:0/1955623018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f574003b050 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 10 client.16122 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576f1a0e510 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f5734ff9640 1 -- 172.21.15.7:0/1955623018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57440043d0 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f5734ff9640 1 -- 172.21.15.7:0/1955623018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f574402ca00 con 0x5576f1991a50 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 10 client.16122 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 20 client.16122 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f1a0ece0 0x5576f1a11100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 4 client.16122 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576f18fdd90 con 0x5576f1a0ece0 2024-03-19T00:41:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.997+0000 7f5754bcf640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f1a0ece0 0x5576f1a11100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f5734ff9640 10 client.16122 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f5754bcf640 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f1a0ece0 0x5576f1a11100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3efae8989a3371 server_cookie=aa5ca3cc269b78e8 in_seq=0 out_seq=0 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f5734ff9640 1 -- 172.21.15.7:0/1955623018 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5576f18fdd90 con 0x5576f1a0ece0 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f5734ff9640 10 client.16122 handle_command_reply: tid=0 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 client.16122 shutdown 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 msgr2=0x5576f19a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 0x5576f19a0ec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f574000a810 tx=0x7f5740004700 comp rx=0 tx=0).stop 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 shutdown_connections 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576f19975b0 0x5576f19a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576f1992440 0x5576f19a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5576f1a0ece0 0x5576f1a11100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 --2- 172.21.15.7:0/1955623018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576f1991a50 0x5576f19a0ec0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f5734ff9640 0 client.16122 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 >> 172.21.15.7:0/1955623018 conn(0x5576f198cfc0 msgr2=0x5576f1990ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 shutdown_connections 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 1 -- 172.21.15.7:0/1955623018 wait complete. 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.998+0000 7f575a219740 20 client.16122 trim_cache size 0 max 16384 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:41:11.006 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:41:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57380a2800 msgr2=0x7f57380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57380a2800 0x7f57380a4cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5744009230 tx=0x7f5744005e40 comp rx=0 tx=0).stop 2024-03-19T00:41:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 msgr2=0x7f575019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 0x7f575019f030 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f574800eb10 tx=0x7f574800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 shutdown_connections 2024-03-19T00:41:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f57380a2800 0x7f57380a4cc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101a10 0x7f575019f030 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750101050 0x7f5750197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 --2- 172.21.15.7:0/3669476380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57500ffe70 0x7f5750197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 >> 172.21.15.7:0/3669476380 conn(0x7f57500fb430 msgr2=0x7f57501065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 shutdown_connections 2024-03-19T00:41:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:10.999+0000 7f5757634640 1 -- 172.21.15.7:0/3669476380 wait complete. 2024-03-19T00:41:11.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:41:11.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:41:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 -- 172.21.15.7:0/4022811971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c101940 msgr2=0x7fe62c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 --2- 172.21.15.7:0/4022811971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c101940 0x7fe62c101dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe614009860 tx=0x7fe61402bb70 comp rx=0 tx=0).stop 2024-03-19T00:41:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 -- 172.21.15.7:0/4022811971 shutdown_connections 2024-03-19T00:41:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 --2- 172.21.15.7:0/4022811971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 --2- 172.21.15.7:0/4022811971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c101940 0x7fe62c101dc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 --2- 172.21.15.7:0/4022811971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe62c100750 0x7fe62c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 -- 172.21.15.7:0/4022811971 >> 172.21.15.7:0/4022811971 conn(0x7fe62c0fbcc0 msgr2=0x7fe62c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 -- 172.21.15.7:0/4022811971 shutdown_connections 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.112+0000 7fe6313f4640 1 -- 172.21.15.7:0/4022811971 wait complete. 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 Processor -- start 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 -- start start 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c100750 0x7fe62c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe62c101940 0x7fe62c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10c260 con 0x7fe62c102300 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10c0e0 con 0x7fe62c101940 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe6313f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10c3e0 con 0x7fe62c100750 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe62b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe62b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c1a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53832/0 (socket says 172.21.15.7:53832) 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe62b7fe640 1 -- 172.21.15.7:0/3345422600 learned_addr learned my addr 172.21.15.7:0/3345422600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe62affd640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c100750 0x7fe62c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.113+0000 7fe62a7fc640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe62c101940 0x7fe62c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62b7fe640 1 -- 172.21.15.7:0/3345422600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c100750 msgr2=0x7fe62c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62b7fe640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c100750 0x7fe62c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62b7fe640 1 -- 172.21.15.7:0/3345422600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe62c101940 msgr2=0x7fe62c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62b7fe640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe62c101940 0x7fe62c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62b7fe640 1 -- 172.21.15.7:0/3345422600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c1a1900 con 0x7fe62c102300 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62affd640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c100750 0x7fe62c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe62b7fe640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c1a1260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00efc0 tx=0x7fe61c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c003970 con 0x7fe62c102300 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c1a1bf0 con 0x7fe62c102300 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe61c004020 con 0x7fe62c102300 2024-03-19T00:41:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00b940 con 0x7fe62c102300 2024-03-19T00:41:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.114+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1a20f0 con 0x7fe62c102300 2024-03-19T00:41:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.115+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c1a22c0 con 0x7fe62c102300 2024-03-19T00:41:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.116+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe61c01d020 con 0x7fe62c102300 2024-03-19T00:41:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.117+0000 7fe623fff640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.117+0000 7fe62affd640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.117+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe61c0cf300 con 0x7fe62c102300 2024-03-19T00:41:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.117+0000 7fe62affd640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe62c19b1c0 tx=0x7fe624005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.120+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe61c09c040 con 0x7fe62c102300 2024-03-19T00:41:11.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.346+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe62c1a2490 con 0x7fe62c102300 2024-03-19T00:41:11.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.347+0000 7fe623fff640 1 -- 172.21.15.7:0/3345422600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11856 (secure 0 0 0) 0x7fe61c09b940 con 0x7fe62c102300 2024-03-19T00:41:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:41:11.348 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:39:16.038000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:41:11.348 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-19T00:41:11.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 msgr2=0x7fe6180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe62c19b1c0 tx=0x7fe624005f90 comp rx=0 tx=0).stop 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 msgr2=0x7fe62c1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c1a1260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00efc0 tx=0x7fe61c00c800 comp rx=0 tx=0).stop 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 shutdown_connections 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62c102300 0x7fe62c1a1260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe62c101940 0x7fe62c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 --2- 172.21.15.7:0/3345422600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe62c100750 0x7fe62c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.349+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 >> 172.21.15.7:0/3345422600 conn(0x7fe62c0fbcc0 msgr2=0x7fe62c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.350+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 shutdown_connections 2024-03-19T00:41:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.350+0000 7fe6313f4640 1 -- 172.21.15.7:0/3345422600 wait complete. 2024-03-19T00:41:11.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:41:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 -- 172.21.15.7:0/265670454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 msgr2=0x7fa898107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 --2- 172.21.15.7:0/265670454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa898107060 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009900 tx=0x7fa88c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 -- 172.21.15.7:0/265670454 shutdown_connections 2024-03-19T00:41:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 --2- 172.21.15.7:0/265670454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8981075a0 0x7fa898109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 --2- 172.21.15.7:0/265670454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa898107060 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 --2- 172.21.15.7:0/265670454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa898102340 0x7fa898104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.465+0000 7fa89ec81640 1 -- 172.21.15.7:0/265670454 >> 172.21.15.7:0/265670454 conn(0x7fa8980fbc50 msgr2=0x7fa8980fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 -- 172.21.15.7:0/265670454 shutdown_connections 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 -- 172.21.15.7:0/265670454 wait complete. 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 Processor -- start 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 -- start start 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa898102340 0x7fa898199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa89819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8981075a0 0x7fa8981a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa89810d350 con 0x7fa898104c70 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa89810d1d0 con 0x7fa8981075a0 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ec81640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa89810d4d0 con 0x7fa898102340 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa897fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa89819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa897fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa89819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53844/0 (socket says 172.21.15.7:53844) 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa897fff640 1 -- 172.21.15.7:0/3528344332 learned_addr learned my addr 172.21.15.7:0/3528344332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89d21e640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8981075a0 0x7fa8981a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.466+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa898102340 0x7fa898199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa897fff640 1 -- 172.21.15.7:0/3528344332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa898102340 msgr2=0x7fa898199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa897fff640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa898102340 0x7fa898199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa897fff640 1 -- 172.21.15.7:0/3528344332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8981075a0 msgr2=0x7fa8981a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa897fff640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8981075a0 0x7fa8981a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa897fff640 1 -- 172.21.15.7:0/3528344332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8981a1880 con 0x7fa898104c70 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa89d21e640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8981075a0 0x7fa8981a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa897fff640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa89819a1b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa88c000c00 tx=0x7fa88c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa895ffb640 1 -- 172.21.15.7:0/3528344332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c043020 con 0x7fa898104c70 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa895ffb640 1 -- 172.21.15.7:0/3528344332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa88c004350 con 0x7fa898104c70 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8981a1b10 con 0x7fa898104c70 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa895ffb640 1 -- 172.21.15.7:0/3528344332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c003b80 con 0x7fa898104c70 2024-03-19T00:41:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.467+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8981a2060 con 0x7fa898104c70 2024-03-19T00:41:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.469+0000 7fa895ffb640 1 -- 172.21.15.7:0/3528344332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa88c046020 con 0x7fa898104c70 2024-03-19T00:41:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.469+0000 7fa8a186e740 1 Processor -- start 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 -- start start 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0eb67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5b0e154b0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e5b0dbdd70 con 0x55e5b0eb6e50 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa8a186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e5b0e11550 con 0x55e5b0eb12f0 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa895ffb640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa8800a2800 0x7fa8800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89ca1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0eb67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa897fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eb1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:34650/0 (socket says 172.21.15.7:34650) 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89ca1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0eb67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53848/0 (socket says 172.21.15.7:53848) 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 -- 172.21.15.7:0/4248243232 learned_addr learned my addr 172.21.15.7:0/4248243232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa8800a2800 0x7fa8800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa895ffb640 1 -- 172.21.15.7:0/3528344332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa88c0f0f70 con 0x7fa898104c70 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 -- 172.21.15.7:0/4248243232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 msgr2=0x55e5b0eb92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eb92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 -- 172.21.15.7:0/4248243232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 msgr2=0x55e5b0eb67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0eb67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 -- 172.21.15.7:0/4248243232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b0de0210 con 0x55e5b0eb12f0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0eb67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa8800a2800 0x7fa8800a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa88800fc90 tx=0x7fa88801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.470+0000 7fa89d21e640 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eb1710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa888011e90 tx=0x7fa88800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d671eac4481832d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa87effd640 1 -- 172.21.15.7:0/4248243232 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88801dbf0 con 0x55e5b0eb12f0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa87effd640 1 -- 172.21.15.7:0/4248243232 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa88801dd90 con 0x55e5b0eb12f0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa87effd640 1 -- 172.21.15.7:0/4248243232 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888010d40 con 0x55e5b0eb12f0 2024-03-19T00:41:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 -- 172.21.15.7:0/4248243232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 msgr2=0x55e5b0eb1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eb1710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa888011e90 tx=0x7fa88800f790 comp rx=0 tx=0).stop 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 -- 172.21.15.7:0/4248243232 shutdown_connections 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eb92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0eb67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4248243232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eb1710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 -- 172.21.15.7:0/4248243232 >> 172.21.15.7:0/4248243232 conn(0x55e5b0eac860 msgr2=0x55e5b0eacc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 -- 172.21.15.7:0/4248243232 shutdown_connections 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.471+0000 7fa8a186e740 1 -- 172.21.15.7:0/4248243232 wait complete. 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 -- start start 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 Processor -- start 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ebb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ebbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5b0eb9da0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e5b0dd1310 con 0x55e5b0eb12f0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e5b0eb9f20 con 0x55e5b0eb6e50 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa8a186e740 10 client.0 _ll_register_callbacks cb 0x55e5b0dc52d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa897fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ebbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa897fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ebbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53858/0 (socket says 172.21.15.7:53858) 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa897fff640 1 -- 172.21.15.7:0/3666066991 learned_addr learned my addr 172.21.15.7:0/3666066991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ebb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.472+0000 7fa89d21e640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa897fff640 1 -- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb6e50 msgr2=0x55e5b0ec2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa897fff640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa897fff640 1 -- 172.21.15.7:0/3666066991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb12f0 msgr2=0x55e5b0ebb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa897fff640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ebb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa897fff640 1 -- 172.21.15.7:0/3666066991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b0b293f0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa897fff640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ebbd10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0b9ef0 tx=0x7fa88c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d87147748b19cf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa877fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa877fff640 1 -- 172.21.15.7:0/3666066991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0bd0f0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa877fff640 1 -- 172.21.15.7:0/3666066991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa88c0f30f0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa88801dd90 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa877fff640 1 -- 172.21.15.7:0/3666066991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa88c0bb6f0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa8a186e740 10 client.16140 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5b0bf37d0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa877fff640 1 -- 172.21.15.7:0/3666066991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa88801dd90 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa877fff640 1 -- 172.21.15.7:0/3666066991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa88c0eebd0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa8a186e740 10 client.16140 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.473+0000 7fa8a186e740 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa8a186e740 20 client.16140 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa8a186e740 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0ec75a0 0x55e5b0ec9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa8a186e740 4 client.16140 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5b0e733e0 con 0x55e5b0ec75a0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0ec75a0 0x55e5b0ec9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa877fff640 10 client.16140 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.474+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0ec75a0 0x55e5b0ec9a20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24ddc31d22f2e67f server_cookie=5efefae1f017935e in_seq=0 out_seq=0 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa877fff640 1 -- 172.21.15.7:0/3666066991 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5b0e733e0 con 0x55e5b0ec75a0 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa877fff640 10 client.16140 handle_command_reply: tid=0 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 client.16140 shutdown 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 msgr2=0x55e5b0ebbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ebbd10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0b9ef0 tx=0x7fa88c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 shutdown_connections 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0ec75a0 0x55e5b0ec9a20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ebbd10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 --2- 172.21.15.7:0/3666066991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ebb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa877fff640 0 client.16140 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 >> 172.21.15.7:0/3666066991 conn(0x55e5b0eac860 msgr2=0x55e5b0eabba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 shutdown_connections 2024-03-19T00:41:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.475+0000 7fa8a186e740 1 -- 172.21.15.7:0/3666066991 wait complete. 2024-03-19T00:41:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.476+0000 7fa8a186e740 20 client.16140 trim_cache size 0 max 16384 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 Processor -- start 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 -- start start 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981aece0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981ac5b0 con 0x55e5b0eb6e50 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa8a186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981aa400 con 0x55e5b0eb12f0 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53864/0 (socket says 172.21.15.7:53864) 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 -- 172.21.15.7:0/884592194 learned_addr learned my addr 172.21.15.7:0/884592194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa89d21e640 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 -- 172.21.15.7:0/884592194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 msgr2=0x55e5b0eab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 -- 172.21.15.7:0/884592194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 msgr2=0x55e5b0eab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 -- 172.21.15.7:0/884592194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b0e82880 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa897fff640 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec2d10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa88c005ab0 tx=0x7fa88c0f12d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da80ba9388b2d9c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.489+0000 7fa87e7fc640 1 -- 172.21.15.7:0/884592194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0bcf10 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa87e7fc640 1 -- 172.21.15.7:0/884592194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa88c0bdcb0 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa87e7fc640 1 -- 172.21.15.7:0/884592194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0d6210 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 -- 172.21.15.7:0/884592194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 msgr2=0x55e5b0ec2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec2d10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa88c005ab0 tx=0x7fa88c0f12d0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 -- 172.21.15.7:0/884592194 shutdown_connections 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0eab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec2d10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 --2- 172.21.15.7:0/884592194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb12f0 0x55e5b0eab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 -- 172.21.15.7:0/884592194 >> 172.21.15.7:0/884592194 conn(0x55e5b0eac860 msgr2=0x55e5b0eacc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 -- 172.21.15.7:0/884592194 shutdown_connections 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 -- 172.21.15.7:0/884592194 wait complete. 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.490+0000 7fa8a186e740 1 -- start start 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 Processor -- start 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ec09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981aaa70 con 0x55e5b0eb12f0 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981abee0 con 0x55e5b0eb6e50 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981ab140 con 0x55e5b0eb1ce0 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa8a186e740 10 client.0 _ll_register_callbacks cb 0x7fa88c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ec09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ec09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53870/0 (socket says 172.21.15.7:53870) 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 -- 172.21.15.7:0/4156910202 learned_addr learned my addr 172.21.15.7:0/4156910202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89d21e640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa897fff640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 -- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb1ce0 msgr2=0x55e5b0ec0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 -- 172.21.15.7:0/4156910202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 msgr2=0x55e5b0ec1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.491+0000 7fa89ca1d640 1 -- 172.21.15.7:0/4156910202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b0ebc080 con 0x55e5b0eb12f0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa89ca1d640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ec09a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa89819b140 tx=0x7fa890003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f619c8bc554af60 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa877fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa877fff640 1 -- 172.21.15.7:0/4156910202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900205b0 con 0x55e5b0eb12f0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa877fff640 1 -- 172.21.15.7:0/4156910202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa890020750 con 0x55e5b0eb12f0 2024-03-19T00:41:11.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa88c0bdcb0 con 0x55e5b0eb12f0 2024-03-19T00:41:11.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa877fff640 1 -- 172.21.15.7:0/4156910202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa890031050 con 0x55e5b0eb12f0 2024-03-19T00:41:11.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 10 client.16146 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5b0f2ddf0 con 0x55e5b0eb12f0 2024-03-19T00:41:11.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa877fff640 1 -- 172.21.15.7:0/4156910202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa88c0bdcb0 con 0x55e5b0eb12f0 2024-03-19T00:41:11.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa877fff640 1 -- 172.21.15.7:0/4156910202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa890029800 con 0x55e5b0eb12f0 2024-03-19T00:41:11.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 10 client.16146 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 20 client.16146 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0f2e820 0x55e5b0f30c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 4 client.16146 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5b0e229f0 con 0x55e5b0f2e820 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.492+0000 7fa897fff640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0f2e820 0x55e5b0f30c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa877fff640 10 client.16146 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa897fff640 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0f2e820 0x55e5b0f30c40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c08788a50220588 server_cookie=b14bede51ab1330 in_seq=0 out_seq=0 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa877fff640 1 -- 172.21.15.7:0/4156910202 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5b0e229f0 con 0x55e5b0f2e820 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa877fff640 10 client.16146 handle_command_reply: tid=0 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 client.16146 shutdown 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 msgr2=0x55e5b0ec09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ec09a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa89819b140 tx=0x7fa890003f60 comp rx=0 tx=0).stop 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 shutdown_connections 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55e5b0f2e820 0x55e5b0f30c40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5b0eb6e50 0x55e5b0ec1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5b0eb1ce0 0x55e5b0ec0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 --2- 172.21.15.7:0/4156910202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5b0eb12f0 0x55e5b0ec09a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa877fff640 0 client.16146 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 >> 172.21.15.7:0/4156910202 conn(0x55e5b0eac860 msgr2=0x55e5b0eafac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 shutdown_connections 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 1 -- 172.21.15.7:0/4156910202 wait complete. 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.493+0000 7fa8a186e740 20 client.16146 trim_cache size 0 max 16384 2024-03-19T00:41:11.503 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:41:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa8800a2800 msgr2=0x7fa8800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa8800a2800 0x7fa8800a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa88800fc90 tx=0x7fa88801e000 comp rx=0 tx=0).stop 2024-03-19T00:41:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 msgr2=0x7fa89819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa89819a1b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa88c000c00 tx=0x7fa88c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 shutdown_connections 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa8800a2800 0x7fa8800a4cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8981075a0 0x7fa8981a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa898104c70 0x7fa89819a1b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 --2- 172.21.15.7:0/3528344332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa898102340 0x7fa898199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 >> 172.21.15.7:0/3528344332 conn(0x7fa8980fbc50 msgr2=0x7fa898101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 shutdown_connections 2024-03-19T00:41:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.494+0000 7fa89ec81640 1 -- 172.21.15.7:0/3528344332 wait complete. 2024-03-19T00:41:11.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:41:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 -- 172.21.15.7:0/2295214507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54104c50 msgr2=0x7f5e54107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 --2- 172.21.15.7:0/2295214507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54104c50 0x7f5e54107040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009900 tx=0x7f5e4802bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 -- 172.21.15.7:0/2295214507 shutdown_connections 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 --2- 172.21.15.7:0/2295214507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e54107580 0x7f5e54109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 --2- 172.21.15.7:0/2295214507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54104c50 0x7f5e54107040 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 --2- 172.21.15.7:0/2295214507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 0x7f5e54104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 -- 172.21.15.7:0/2295214507 >> 172.21.15.7:0/2295214507 conn(0x7f5e540fbc90 msgr2=0x7f5e540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 -- 172.21.15.7:0/2295214507 shutdown_connections 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.610+0000 7f5e597c4640 1 -- 172.21.15.7:0/2295214507 wait complete. 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 Processor -- start 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 -- start start 2024-03-19T00:41:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 0x7f5e54197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e54104c50 0x7f5e54197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 0x7f5e5419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d340 con 0x7f5e54107580 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d1c0 con 0x7f5e54104c50 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e597c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d4c0 con 0x7f5e54102320 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 0x7f5e5419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.611+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 0x7f5e5419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53900/0 (socket says 172.21.15.7:53900) 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 -- 172.21.15.7:0/3247845961 learned_addr learned my addr 172.21.15.7:0/3247845961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 0x7f5e54197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e527fc640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e54104c50 0x7f5e54197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 -- 172.21.15.7:0/3247845961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 msgr2=0x7f5e54197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 0x7f5e54197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 -- 172.21.15.7:0/3247845961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e54104c50 msgr2=0x7f5e54197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e54104c50 0x7f5e54197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 -- 172.21.15.7:0/3247845961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5419f6d0 con 0x7f5e54107580 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 0x7f5e54197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e537fe640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 0x7f5e5419f060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00efd0 tx=0x7f5e4c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e3bfff640 1 -- 172.21.15.7:0/3247845961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c003b70 con 0x7f5e54107580 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5419f9c0 con 0x7f5e54107580 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e3bfff640 1 -- 172.21.15.7:0/3247845961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c003d10 con 0x7f5e54107580 2024-03-19T00:41:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e3bfff640 1 -- 172.21.15.7:0/3247845961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c010980 con 0x7f5e54107580 2024-03-19T00:41:11.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.612+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5419fe90 con 0x7f5e54107580 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e3bfff640 1 -- 172.21.15.7:0/3247845961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5e4c004020 con 0x7f5e54107580 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 Processor -- start 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 -- start start 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be73710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be73ce0 0x558a2be787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be78e50 0x558a2be7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558a2bdd74b0 con 0x558a2be732f0 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558a2bd7fd70 con 0x558a2be78e50 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558a2bdd3550 con 0x558a2be73ce0 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be73710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be73710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53916/0 (socket says 172.21.15.7:53916) 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e527fc640 1 -- 172.21.15.7:0/268726600 learned_addr learned my addr 172.21.15.7:0/268726600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be73ce0 0x558a2be787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e3bfff640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e3c0a2800 0x7f5e3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e527fc640 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be78e50 0x558a2be7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e537fe640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e3c0a2800 0x7f5e3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e52ffd640 1 -- 172.21.15.7:0/268726600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be78e50 msgr2=0x558a2be7b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be78e50 0x558a2be7b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e52ffd640 1 -- 172.21.15.7:0/268726600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 msgr2=0x558a2be73710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be73710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e52ffd640 1 -- 172.21.15.7:0/268726600 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2bda2210 con 0x558a2be73ce0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.615+0000 7f5e3bfff640 1 -- 172.21.15.7:0/3247845961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e4c0cf220 con 0x7f5e54107580 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be73ce0 0x558a2be787e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009fd0 tx=0x7f5e440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d796f87392ca5984 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e537fe640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e3c0a2800 0x7f5e3c0a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0ccd30 tx=0x7f5e4c066ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e397fa640 1 -- 172.21.15.7:0/268726600 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44004030 con 0x558a2be73ce0 2024-03-19T00:41:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e397fa640 1 -- 172.21.15.7:0/268726600 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e440054d0 con 0x558a2be73ce0 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e397fa640 1 -- 172.21.15.7:0/268726600 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44004ce0 con 0x558a2be73ce0 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e5c3af740 1 -- 172.21.15.7:0/268726600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be73ce0 msgr2=0x558a2be787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.616+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be73ce0 0x558a2be787e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009fd0 tx=0x7f5e440118d0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- 172.21.15.7:0/268726600 shutdown_connections 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be78e50 0x558a2be7b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be73ce0 0x558a2be787e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/268726600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be73710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- 172.21.15.7:0/268726600 >> 172.21.15.7:0/268726600 conn(0x558a2be6e860 msgr2=0x558a2be6ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- 172.21.15.7:0/268726600 shutdown_connections 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- 172.21.15.7:0/268726600 wait complete. 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- start start 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 Processor -- start 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be7d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be78e50 0x558a2be84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558a2be7be00 con 0x558a2be732f0 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558a2bd93310 con 0x558a2be73ce0 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558a2be7bf80 con 0x558a2be78e50 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e5c3af740 10 client.0 _ll_register_callbacks cb 0x558a2bd872d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be7d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be7d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53932/0 (socket says 172.21.15.7:53932) 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e527fc640 1 -- 172.21.15.7:0/1854818723 learned_addr learned my addr 172.21.15.7:0/1854818723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.617+0000 7f5e527fc640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be78e50 0x558a2be84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e527fc640 1 -- 172.21.15.7:0/1854818723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be78e50 msgr2=0x558a2be84d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e527fc640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be78e50 0x558a2be84d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e527fc640 1 -- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 msgr2=0x558a2be7dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e527fc640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be7dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e527fc640 1 -- 172.21.15.7:0/1854818723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2baeb3f0 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e527fc640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be7d7f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e48036040 tx=0x7f5e48005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f15409ffc4b842 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e36ff5640 1 -- 172.21.15.7:0/1854818723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48005040 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e36ff5640 1 -- 172.21.15.7:0/1854818723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e4803a040 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e440054d0 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e36ff5640 1 -- 172.21.15.7:0/1854818723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e480051e0 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 10 client.16158 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a2bbb57d0 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e36ff5640 1 -- 172.21.15.7:0/1854818723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e440054d0 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e36ff5640 1 -- 172.21.15.7:0/1854818723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f5e48040040 con 0x558a2be732f0 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 10 client.16158 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 20 client.16158 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2be8ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 4 client.16158 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.618+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a2be353e0 con 0x558a2be895a0 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.619+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2be8ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.619+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2be8ba30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed904d090325a56b server_cookie=d825f235e5ed13d0 in_seq=0 out_seq=0 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.619+0000 7f5e36ff5640 10 client.16158 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e36ff5640 1 -- 172.21.15.7:0/1854818723 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558a2be353e0 con 0x558a2be895a0 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e36ff5640 10 client.16158 handle_command_reply: tid=0 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 client.16158 shutdown 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 msgr2=0x558a2be7d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be7d7f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e48036040 tx=0x7f5e48005bd0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 shutdown_connections 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2be8ba30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be78e50 0x558a2be84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be7dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e36ff5640 0 client.16158 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/1854818723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be732f0 0x558a2be7d7f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 >> 172.21.15.7:0/1854818723 conn(0x558a2be6e860 msgr2=0x558a2be6dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 shutdown_connections 2024-03-19T00:41:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 1 -- 172.21.15.7:0/1854818723 wait complete. 2024-03-19T00:41:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.620+0000 7f5e5c3af740 20 client.16158 trim_cache size 0 max 16384 2024-03-19T00:41:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 Processor -- start 2024-03-19T00:41:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 -- start start 2024-03-19T00:41:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be7dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be84d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541acbe0 con 0x558a2be78e50 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541aa4b0 con 0x558a2be73ce0 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a8300 con 0x558a2be732f0 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53936/0 (socket says 172.21.15.7:53936) 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e527fc640 1 -- 172.21.15.7:0/584465932 learned_addr learned my addr 172.21.15.7:0/584465932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e527fc640 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be7dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.633+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be84d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e527fc640 1 -- 172.21.15.7:0/584465932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 msgr2=0x558a2be7dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e527fc640 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be7dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e527fc640 1 -- 172.21.15.7:0/584465932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 msgr2=0x558a2be84d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e527fc640 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be84d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e527fc640 1 -- 172.21.15.7:0/584465932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2be44880 con 0x558a2be78e50 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e527fc640 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be721d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x558a2be7e880 tx=0x7f5e48005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96def597b4247b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e38ff9640 1 -- 172.21.15.7:0/584465932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48005040 con 0x558a2be78e50 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e38ff9640 1 -- 172.21.15.7:0/584465932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e48003710 con 0x558a2be78e50 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e38ff9640 1 -- 172.21.15.7:0/584465932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480046c0 con 0x558a2be78e50 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 -- 172.21.15.7:0/584465932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 msgr2=0x558a2be721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be721d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x558a2be7e880 tx=0x7f5e48005fd0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 -- 172.21.15.7:0/584465932 shutdown_connections 2024-03-19T00:41:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be721d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be84d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/584465932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be7dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.634+0000 7f5e5c3af740 1 -- 172.21.15.7:0/584465932 >> 172.21.15.7:0/584465932 conn(0x558a2be6e860 msgr2=0x558a2be71ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 -- 172.21.15.7:0/584465932 shutdown_connections 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 -- 172.21.15.7:0/584465932 wait complete. 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 -- start start 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 Processor -- start 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be82a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be82f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a8970 con 0x558a2be78e50 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a9de0 con 0x558a2be73ce0 2024-03-19T00:41:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a9040 con 0x558a2be732f0 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e5c3af740 10 client.0 _ll_register_callbacks cb 0x7f5e48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be83f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53940/0 (socket says 172.21.15.7:53940) 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 -- 172.21.15.7:0/2277261414 learned_addr learned my addr 172.21.15.7:0/2277261414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be82f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be82a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 -- 172.21.15.7:0/2277261414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 msgr2=0x558a2be82a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be82a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 -- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 msgr2=0x558a2be82f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be82f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.635+0000 7f5e527fc640 1 -- 172.21.15.7:0/2277261414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2be84580 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e527fc640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be83f90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5e48005700 tx=0x7f5e48005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627b1ea8f849051d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e36ff5640 1 -- 172.21.15.7:0/2277261414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480048c0 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e36ff5640 1 -- 172.21.15.7:0/2277261414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e48004a60 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e48003710 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e36ff5640 1 -- 172.21.15.7:0/2277261414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e480038c0 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 10 client.16164 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a2beeff50 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e36ff5640 1 -- 172.21.15.7:0/2277261414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e48003710 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e36ff5640 1 -- 172.21.15.7:0/2277261414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f5e4804e410 con 0x558a2be78e50 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 10 client.16164 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:41:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 20 client.16164 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2bef26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 4 client.16164 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a2bde49f0 con 0x558a2be895a0 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.636+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2bef26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e52ffd640 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2bef26e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc488a178ba5c635 server_cookie=a26997ccfb367c0b in_seq=0 out_seq=0 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e36ff5640 10 client.16164 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e36ff5640 1 -- 172.21.15.7:0/2277261414 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a2bde49f0 con 0x558a2be895a0 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e36ff5640 10 client.16164 handle_command_reply: tid=0 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 client.16164 shutdown 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 msgr2=0x558a2be83f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be83f90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5e48005700 tx=0x7f5e48005950 comp rx=0 tx=0).stop 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 shutdown_connections 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558a2be895a0 0x558a2bef26e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558a2be78e50 0x558a2be83f90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a2be73ce0 0x558a2be82f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e36ff5640 0 client.16164 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 --2- 172.21.15.7:0/2277261414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a2be732f0 0x558a2be82a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 >> 172.21.15.7:0/2277261414 conn(0x558a2be6e860 msgr2=0x558a2be72350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 shutdown_connections 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 1 -- 172.21.15.7:0/2277261414 wait complete. 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.637+0000 7f5e5c3af740 20 client.16164 trim_cache size 0 max 16384 2024-03-19T00:41:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e3c0a2800 msgr2=0x7f5e3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e3c0a2800 0x7f5e3c0a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0ccd30 tx=0x7f5e4c066ac0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 msgr2=0x7f5e5419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 0x7f5e5419f060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00efd0 tx=0x7f5e4c00c6a0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 shutdown_connections 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e3c0a2800 0x7f5e3c0a4cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e54107580 0x7f5e5419f060 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e54104c50 0x7f5e54197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 --2- 172.21.15.7:0/3247845961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54102320 0x7f5e54197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 >> 172.21.15.7:0/3247845961 conn(0x7f5e540fbc90 msgr2=0x7f5e54102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 shutdown_connections 2024-03-19T00:41:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.638+0000 7f5e597c4640 1 -- 172.21.15.7:0/3247845961 wait complete. 2024-03-19T00:41:11.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:11 smithi007 ceph-mon[62854]: pgmap v448: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-03-19T00:41:11.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:11 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3345422600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:41:11.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:41:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 -- 172.21.15.7:0/3650336253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 msgr2=0x7f8760101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 --2- 172.21.15.7:0/3650336253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f8760101da0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 -- 172.21.15.7:0/3650336253 shutdown_connections 2024-03-19T00:41:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 --2- 172.21.15.7:0/3650336253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87601022e0 0x7f8760108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 --2- 172.21.15.7:0/3650336253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f8760101da0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 --2- 172.21.15.7:0/3650336253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8760100730 0x7f8760100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 -- 172.21.15.7:0/3650336253 >> 172.21.15.7:0/3650336253 conn(0x7f87600fbc80 msgr2=0x7f87600fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 -- 172.21.15.7:0/3650336253 shutdown_connections 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 -- 172.21.15.7:0/3650336253 wait complete. 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.754+0000 7f876520c640 1 Processor -- start 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 -- start start 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 0x7f876006aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f876019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87601022e0 0x7f876019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f876010c240 con 0x7f8760101920 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f876010c0c0 con 0x7f87601022e0 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f876520c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f876010c3c0 con 0x7f8760100730 2024-03-19T00:41:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f876019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 0x7f876006aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f876019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53962/0 (socket says 172.21.15.7:53962) 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 0x7f876006aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:34742/0 (socket says 172.21.15.7:34742) 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 -- 172.21.15.7:0/599229703 learned_addr learned my addr 172.21.15.7:0/599229703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875f59e640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87601022e0 0x7f876019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 -- 172.21.15.7:0/599229703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 msgr2=0x7f876006aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 0x7f876006aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 -- 172.21.15.7:0/599229703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87601022e0 msgr2=0x7f876019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87601022e0 0x7f876019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 -- 172.21.15.7:0/599229703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876019f660 con 0x7f8760101920 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875ed9d640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 0x7f876006aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875f59e640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87601022e0 0x7f876019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.755+0000 7f875e59c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f876019c730 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8748006010 tx=0x7f8748005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.756+0000 7f8757fff640 1 -- 172.21.15.7:0/599229703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748043020 con 0x7f8760101920 2024-03-19T00:41:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.756+0000 7f8757fff640 1 -- 172.21.15.7:0/599229703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87480043d0 con 0x7f8760101920 2024-03-19T00:41:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.756+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876019f8f0 con 0x7f8760101920 2024-03-19T00:41:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.756+0000 7f8757fff640 1 -- 172.21.15.7:0/599229703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748003b80 con 0x7f8760101920 2024-03-19T00:41:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.756+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876019fe40 con 0x7f8760101920 2024-03-19T00:41:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.758+0000 7f8757fff640 1 -- 172.21.15.7:0/599229703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f8748046020 con 0x7f8760101920 2024-03-19T00:41:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.758+0000 7f8767df0740 1 Processor -- start 2024-03-19T00:41:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.758+0000 7f8767df0740 1 -- start start 2024-03-19T00:41:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.758+0000 7f8767df0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x558561180710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8767df0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x5585611857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8767df0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x5585611882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8767df0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5585610e44b0 con 0x5585611802f0 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8767df0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55856108cd70 con 0x558561185e50 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8767df0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5585610e0550 con 0x558561180ce0 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8757fff640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f874c0a2800 0x7f874c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x558561180710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x5585611857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x5585611857e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:34752/0 (socket says 172.21.15.7:34752) 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x558561180710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53974/0 (socket says 172.21.15.7:53974) 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 -- 172.21.15.7:0/514463097 learned_addr learned my addr 172.21.15.7:0/514463097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875e59c640 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x5585611882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875f59e640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f874c0a2800 0x7f874c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f8757fff640 1 -- 172.21.15.7:0/599229703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f87480f0fd0 con 0x7f8760101920 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 -- 172.21.15.7:0/514463097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 msgr2=0x5585611882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x5585611882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 -- 172.21.15.7:0/514463097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 msgr2=0x558561180710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x558561180710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 -- 172.21.15.7:0/514463097 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585610af210 con 0x558561180ce0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875f59e640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f874c0a2800 0x7f874c0a4cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f87500099e0 tx=0x7f8750009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f875ed9d640 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x5585611857e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f87580114f0 tx=0x7f87580119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5297e6bf9c2ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f87557fa640 1 -- 172.21.15.7:0/514463097 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875801e070 con 0x558561180ce0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f87557fa640 1 -- 172.21.15.7:0/514463097 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f875800fa70 con 0x558561180ce0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.759+0000 7f87557fa640 1 -- 172.21.15.7:0/514463097 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758019b00 con 0x558561180ce0 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 -- 172.21.15.7:0/514463097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 msgr2=0x5585611857e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x5585611857e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f87580114f0 tx=0x7f87580119b0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 -- 172.21.15.7:0/514463097 shutdown_connections 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x5585611882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x5585611857e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 --2- 172.21.15.7:0/514463097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x558561180710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 -- 172.21.15.7:0/514463097 >> 172.21.15.7:0/514463097 conn(0x55856117b860 msgr2=0x55856117bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 -- 172.21.15.7:0/514463097 shutdown_connections 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.760+0000 7f8767df0740 1 -- 172.21.15.7:0/514463097 wait complete. 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 -- start start 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 Processor -- start 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x55856118a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x55856118ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x558561191db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558561188e00 con 0x5585611802f0 2024-03-19T00:41:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585610a0310 con 0x558561185e50 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558561188f80 con 0x558561180ce0 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f8767df0740 10 client.0 _ll_register_callbacks cb 0x5585610942d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x55856118a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x55856118a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53984/0 (socket says 172.21.15.7:53984) 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 -- 172.21.15.7:0/785317016 learned_addr learned my addr 172.21.15.7:0/785317016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875f59e640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x558561191db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875e59c640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x55856118ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 -- 172.21.15.7:0/785317016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 msgr2=0x55856118ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x55856118ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 -- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 msgr2=0x558561191db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x558561191db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 -- 172.21.15.7:0/785317016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558560df83f0 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875e59c640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x55856118ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f875ed9d640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x55856118a830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f875801c040 tx=0x7f875800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2158766fea941d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f87467fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.761+0000 7f87467fc640 1 -- 172.21.15.7:0/785317016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875801e040 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f87467fc640 1 -- 172.21.15.7:0/785317016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8758004040 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875800fa70 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f87467fc640 1 -- 172.21.15.7:0/785317016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8758031050 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 10 client.16176 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558560ec27d0 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f87467fc640 1 -- 172.21.15.7:0/785317016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f875800fa70 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f87467fc640 1 -- 172.21.15.7:0/785317016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f875802c030 con 0x5585611802f0 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 10 client.16176 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 20 client.16176 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561196410 0x558561198880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 4 client.16176 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585611423e0 con 0x558561196410 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.762+0000 7f875e59c640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561196410 0x558561198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f87467fc640 10 client.16176 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f875e59c640 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561196410 0x558561198880 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115b2abbab38a027 server_cookie=81008ca81224f06e in_seq=0 out_seq=0 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f87467fc640 1 -- 172.21.15.7:0/785317016 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5585611423e0 con 0x558561196410 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f87467fc640 10 client.16176 handle_command_reply: tid=0 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 client.16176 shutdown 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 msgr2=0x55856118a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x55856118a830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f875801c040 tx=0x7f875800f870 comp rx=0 tx=0).stop 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 shutdown_connections 2024-03-19T00:41:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561185e50 0x558561191db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558561180ce0 0x55856118ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561196410 0x558561198880 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 --2- 172.21.15.7:0/785317016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5585611802f0 0x55856118a830 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f87467fc640 0 client.16176 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.763+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 >> 172.21.15.7:0/785317016 conn(0x55856117b860 msgr2=0x55856117aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.764+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 shutdown_connections 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.764+0000 7f8767df0740 1 -- 172.21.15.7:0/785317016 wait complete. 2024-03-19T00:41:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.764+0000 7f8767df0740 20 client.16176 trim_cache size 0 max 16384 2024-03-19T00:41:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 Processor -- start 2024-03-19T00:41:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 -- start start 2024-03-19T00:41:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 0x55856117f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x558561191d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561180ce0 0x558561189e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87601acbb0 con 0x55856117b860 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87601aa480 con 0x558561180ce0 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f8767df0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a82d0 con 0x5585611802f0 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 0x55856117f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f875ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 0x55856117f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53986/0 (socket says 172.21.15.7:53986) 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f875ed9d640 1 -- 172.21.15.7:0/309917845 learned_addr learned my addr 172.21.15.7:0/309917845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f875f59e640 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561180ce0 0x558561189e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.777+0000 7f875e59c640 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x558561191d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875ed9d640 1 -- 172.21.15.7:0/309917845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 msgr2=0x558561191d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875ed9d640 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x558561191d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875ed9d640 1 -- 172.21.15.7:0/309917845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561180ce0 msgr2=0x558561189e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875ed9d640 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561180ce0 0x558561189e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875ed9d640 1 -- 172.21.15.7:0/309917845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558561151880 con 0x55856117b860 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875e59c640 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x558561191d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f875ed9d640 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 0x55856117f1d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8758000c00 tx=0x7f8758005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc59ec7576024f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f8754ff9640 1 -- 172.21.15.7:0/309917845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875801e030 con 0x55856117b860 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f8754ff9640 1 -- 172.21.15.7:0/309917845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8758016020 con 0x55856117b860 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.778+0000 7f8754ff9640 1 -- 172.21.15.7:0/309917845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758004aa0 con 0x55856117b860 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- 172.21.15.7:0/309917845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 msgr2=0x55856117f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 0x55856117f1d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8758000c00 tx=0x7f8758005f90 comp rx=0 tx=0).stop 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- 172.21.15.7:0/309917845 shutdown_connections 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558561180ce0 0x558561189e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x558561191d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- 172.21.15.7:0/309917845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55856117b860 0x55856117f1d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- 172.21.15.7:0/309917845 >> 172.21.15.7:0/309917845 conn(0x558561181ac0 msgr2=0x55856117eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- 172.21.15.7:0/309917845 shutdown_connections 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- 172.21.15.7:0/309917845 wait complete. 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- start start 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 Processor -- start 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55856117b860 0x55856118f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x55856118f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 0x5585611907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a8940 con 0x558561180ce0 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a9db0 con 0x55856117b860 2024-03-19T00:41:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a9010 con 0x5585611802f0 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f8767df0740 10 client.0 _ll_register_callbacks cb 0x7f875801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f875f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 0x5585611907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f875f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 0x5585611907b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53996/0 (socket says 172.21.15.7:53996) 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f875f59e640 1 -- 172.21.15.7:0/2233057412 learned_addr learned my addr 172.21.15.7:0/2233057412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f875ed9d640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55856117b860 0x55856118f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.779+0000 7f875e59c640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x55856118f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f875f59e640 1 -- 172.21.15.7:0/2233057412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 msgr2=0x55856118f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f875f59e640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x55856118f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f875f59e640 1 -- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55856117b860 msgr2=0x55856118f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f875f59e640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55856117b860 0x55856118f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f875f59e640 1 -- 172.21.15.7:0/2233057412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558561189120 con 0x558561180ce0 2024-03-19T00:41:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f875f59e640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 0x5585611907b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8750003ec0 tx=0x7f8750004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930d4e2a50daba2b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f87467fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f87467fc640 1 -- 172.21.15.7:0/2233057412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750011220 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f87467fc640 1 -- 172.21.15.7:0/2233057412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8750004b20 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8758016020 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f87467fc640 1 -- 172.21.15.7:0/2233057412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8750035050 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f8767df0740 10 client.16182 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558561191f40 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f87467fc640 1 -- 172.21.15.7:0/2233057412 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8758016020 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.780+0000 7f87467fc640 1 -- 172.21.15.7:0/2233057412 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f875001d040 con 0x558561180ce0 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 10 client.16182 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 20 client.16182 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561185e50 0x5585611fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 4 client.16182 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585610f19f0 con 0x558561185e50 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f875ed9d640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561185e50 0x5585611fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f875ed9d640 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561185e50 0x5585611fef20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc4ad5873912691 server_cookie=f80c7dd61804b26e in_seq=0 out_seq=0 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f87467fc640 10 client.16182 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f87467fc640 1 -- 172.21.15.7:0/2233057412 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585610f19f0 con 0x558561185e50 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f87467fc640 10 client.16182 handle_command_reply: tid=0 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 client.16182 shutdown 2024-03-19T00:41:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 msgr2=0x5585611907b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 0x5585611907b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8750003ec0 tx=0x7f8750004ab0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 shutdown_connections 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558561185e50 0x5585611fef20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558561180ce0 0x5585611907b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f87467fc640 0 client.16182 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585611802f0 0x55856118f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 --2- 172.21.15.7:0/2233057412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55856117b860 0x55856118f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 >> 172.21.15.7:0/2233057412 conn(0x558561181ac0 msgr2=0x55856117f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.781+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 shutdown_connections 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f8767df0740 1 -- 172.21.15.7:0/2233057412 wait complete. 2024-03-19T00:41:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f8767df0740 20 client.16182 trim_cache size 0 max 16384 2024-03-19T00:41:11.792 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:41:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f874c0a2800 msgr2=0x7f874c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f874c0a2800 0x7f874c0a4cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f87500099e0 tx=0x7f8750009500 comp rx=0 tx=0).stop 2024-03-19T00:41:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 msgr2=0x7f876019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f876019c730 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8748006010 tx=0x7f8748005e40 comp rx=0 tx=0).stop 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 shutdown_connections 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f874c0a2800 0x7f874c0a4cc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87601022e0 0x7f876019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760101920 0x7f876019c730 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 --2- 172.21.15.7:0/599229703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8760100730 0x7f876006aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 >> 172.21.15.7:0/599229703 conn(0x7f87600fbc80 msgr2=0x7f8760104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 shutdown_connections 2024-03-19T00:41:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:11.782+0000 7f876520c640 1 -- 172.21.15.7:0/599229703 wait complete. 2024-03-19T00:41:11.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:11 smithi044 ceph-mon[66366]: pgmap v448: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-03-19T00:41:11.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:11 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3345422600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:41:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:11 smithi204 ceph-mon[66572]: pgmap v448: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-03-19T00:41:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:11 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3345422600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:41:12.795 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:41:12.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:41:12.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:41:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 -- 172.21.15.7:0/2923871520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 msgr2=0x7fea20100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 --2- 172.21.15.7:0/2923871520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20100b60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009900 tx=0x7fea1c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 -- 172.21.15.7:0/2923871520 shutdown_connections 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 --2- 172.21.15.7:0/2923871520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea201022f0 0x7fea20108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 --2- 172.21.15.7:0/2923871520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea20101930 0x7fea20101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 --2- 172.21.15.7:0/2923871520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20100b60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 -- 172.21.15.7:0/2923871520 >> 172.21.15.7:0/2923871520 conn(0x7fea200fbcb0 msgr2=0x7fea200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 -- 172.21.15.7:0/2923871520 shutdown_connections 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.917+0000 7fea277a9640 1 -- 172.21.15.7:0/2923871520 wait complete. 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 Processor -- start 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 -- start start 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea20101930 0x7fea20196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea201022f0 0x7fea2019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea2010c270 con 0x7fea20100740 2024-03-19T00:41:12.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fea2010c0f0 con 0x7fea201022f0 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea277a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea2010c3f0 con 0x7fea20101930 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54022/0 (socket says 172.21.15.7:54022) 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 -- 172.21.15.7:0/2942972402 learned_addr learned my addr 172.21.15.7:0/2942972402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea24d44640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea20101930 0x7fea20196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25d46640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea201022f0 0x7fea2019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 -- 172.21.15.7:0/2942972402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea20101930 msgr2=0x7fea20196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea20101930 0x7fea20196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 -- 172.21.15.7:0/2942972402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea201022f0 msgr2=0x7fea2019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea201022f0 0x7fea2019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.918+0000 7fea25545640 1 -- 172.21.15.7:0/2942972402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2019e5b0 con 0x7fea20100740 2024-03-19T00:41:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.919+0000 7fea25545640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20196990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.919+0000 7fea167fc640 1 -- 172.21.15.7:0/2942972402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c043020 con 0x7fea20100740 2024-03-19T00:41:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.919+0000 7fea167fc640 1 -- 172.21.15.7:0/2942972402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea1c004350 con 0x7fea20100740 2024-03-19T00:41:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.919+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2019e840 con 0x7fea20100740 2024-03-19T00:41:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.919+0000 7fea167fc640 1 -- 172.21.15.7:0/2942972402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c003b80 con 0x7fea20100740 2024-03-19T00:41:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.919+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2019ed90 con 0x7fea20100740 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.921+0000 7fea167fc640 1 -- 172.21.15.7:0/2942972402 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fea1c046020 con 0x7fea20100740 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 Processor -- start 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea167fc640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fea0c0a2800 0x7fea0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 -- start start 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea167fc640 1 -- 172.21.15.7:0/2942972402 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fea1c0f0ec0 con 0x7fea20100740 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25d46640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fea0c0a2800 0x7fea0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6acb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6acbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6adca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615c6a35160 con 0x5615c6acb9f0 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615c69dde10 con 0x5615c6accd60 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea2a38c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615c6a31200 con 0x5615c6ada680 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea24d44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6acbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25545640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6acb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea24d44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6acbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54032/0 (socket says 172.21.15.7:54032) 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea24d44640 1 -- 172.21.15.7:0/486071051 learned_addr learned my addr 172.21.15.7:0/486071051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25d46640 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6adca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25545640 1 -- 172.21.15.7:0/486071051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 msgr2=0x5615c6adca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25545640 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6adca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25545640 1 -- 172.21.15.7:0/486071051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 msgr2=0x5615c6acbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25545640 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6acbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25545640 1 -- 172.21.15.7:0/486071051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c68117e0 con 0x5615c6accd60 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea25d46640 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6adca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.922+0000 7fea24d44640 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6acbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea25d46640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fea0c0a2800 0x7fea0c0a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fea18005fd0 tx=0x7fea18011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea25545640 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6acb4b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0ee6b0 tx=0x7fea1c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f860105d7428646f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea0b7fe640 1 -- 172.21.15.7:0/486071051 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0bce60 con 0x5615c6accd60 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea0b7fe640 1 -- 172.21.15.7:0/486071051 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea1c0bdc00 con 0x5615c6accd60 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea0b7fe640 1 -- 172.21.15.7:0/486071051 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c003d20 con 0x5615c6accd60 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea2a38c740 1 -- 172.21.15.7:0/486071051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 msgr2=0x5615c6acb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.923+0000 7fea2a38c740 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6acb4b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0ee6b0 tx=0x7fea1c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- 172.21.15.7:0/486071051 shutdown_connections 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6adca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6acbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 --2- 172.21.15.7:0/486071051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6acb4b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- 172.21.15.7:0/486071051 >> 172.21.15.7:0/486071051 conn(0x5615c6acdb20 msgr2=0x5615c6acdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- 172.21.15.7:0/486071051 shutdown_connections 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- 172.21.15.7:0/486071051 wait complete. 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- start start 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 Processor -- start 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6ade970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6adeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5615c6add070 con 0x5615c6acb9f0 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615c69f11a0 con 0x5615c6accd60 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615c6add1f0 con 0x5615c6ada680 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea25545640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6ade970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.924+0000 7fea2a38c740 10 client.0 _ll_register_callbacks cb 0x5615c6748c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6ade970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54042/0 (socket says 172.21.15.7:54042) 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 -- 172.21.15.7:0/3316745989 learned_addr learned my addr 172.21.15.7:0/3316745989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea24d44640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6adeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25d46640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 -- 172.21.15.7:0/3316745989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 msgr2=0x5615c6ae5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 -- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 msgr2=0x5615c6adeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6adeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 -- 172.21.15.7:0/3316745989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c6871f70 con 0x5615c6acb9f0 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea24d44640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6adeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:12.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea25545640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6ade970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b9ed0 tx=0x7fea1c005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230fb77dad70ea0a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea08ff9640 1 -- 172.21.15.7:0/3316745989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0da070 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea08ff9640 1 -- 172.21.15.7:0/3316745989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fea1c0d6260 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c0bdc00 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea08ff9640 1 -- 172.21.15.7:0/3316745989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea1c0bce60 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea2a38c740 10 client.16194 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.925+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615c6709170 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea08ff9640 1 -- 172.21.15.7:0/3316745989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea1c0bdc00 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea08ff9640 1 -- 172.21.15.7:0/3316745989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fea1c0bc9e0 con 0x5615c6acb9f0 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea2a38c740 10 client.16194 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea2a38c740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea2a38c740 20 client.16194 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea2a38c740 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6aea430 0x5615c6aec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea2a38c740 4 client.16194 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615c6a426a0 con 0x5615c6aea430 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea24d44640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6aea430 0x5615c6aec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea08ff9640 10 client.16194 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.926+0000 7fea24d44640 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6aea430 0x5615c6aec890 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46a5e1174e4db66 server_cookie=aa7bf841b7e2037f in_seq=0 out_seq=0 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea08ff9640 1 -- 172.21.15.7:0/3316745989 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5615c6a426a0 con 0x5615c6aea430 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea08ff9640 10 client.16194 handle_command_reply: tid=0 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 client.16194 shutdown 2024-03-19T00:41:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 msgr2=0x5615c6ade970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6ade970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b9ed0 tx=0x7fea1c005700 comp rx=0 tx=0).stop 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 shutdown_connections 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6accd60 0x5615c6adeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6aea430 0x5615c6aec890 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 --2- 172.21.15.7:0/3316745989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6acb9f0 0x5615c6ade970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea08ff9640 0 client.16194 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 >> 172.21.15.7:0/3316745989 conn(0x5615c6acdb20 msgr2=0x5615c6acab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 shutdown_connections 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 1 -- 172.21.15.7:0/3316745989 wait complete. 2024-03-19T00:41:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.927+0000 7fea2a38c740 20 client.16194 trim_cache size 0 max 16384 2024-03-19T00:41:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 Processor -- start 2024-03-19T00:41:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 -- start start 2024-03-19T00:41:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6adee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6accd60 0x5615c6ae5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 0x5615c6acb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea201aa9f0 con 0x5615c6ada680 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fea201a82c0 con 0x5615c6acb9f0 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea2a38c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea201a6110 con 0x5615c6accd60 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea25d46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 0x5615c6acb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea25545640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6adee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea25d46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 0x5615c6acb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54054/0 (socket says 172.21.15.7:54054) 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea25d46640 1 -- 172.21.15.7:0/1363668965 learned_addr learned my addr 172.21.15.7:0/1363668965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.945+0000 7fea24d44640 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6accd60 0x5615c6ae5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25d46640 1 -- 172.21.15.7:0/1363668965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6accd60 msgr2=0x5615c6ae5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25d46640 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6accd60 0x5615c6ae5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25d46640 1 -- 172.21.15.7:0/1363668965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 msgr2=0x5615c6adee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25d46640 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6adee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25d46640 1 -- 172.21.15.7:0/1363668965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c6ace890 con 0x5615c6ada680 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25545640 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6adee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea25d46640 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 0x5615c6acb2b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea1800fe40 tx=0x7fea18009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878e1bed6760f1be server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea0affd640 1 -- 172.21.15.7:0/1363668965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1801a040 con 0x5615c6ada680 2024-03-19T00:41:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea0affd640 1 -- 172.21.15.7:0/1363668965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea18018040 con 0x5615c6ada680 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.946+0000 7fea0affd640 1 -- 172.21.15.7:0/1363668965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18021430 con 0x5615c6ada680 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- 172.21.15.7:0/1363668965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 msgr2=0x5615c6acb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 0x5615c6acb2b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea1800fe40 tx=0x7fea18009500 comp rx=0 tx=0).stop 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- 172.21.15.7:0/1363668965 shutdown_connections 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6ada680 0x5615c6acb2b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6accd60 0x5615c6ae5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- 172.21.15.7:0/1363668965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6adee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- 172.21.15.7:0/1363668965 >> 172.21.15.7:0/1363668965 conn(0x5615c6acdb20 msgr2=0x5615c6ad0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- 172.21.15.7:0/1363668965 shutdown_connections 2024-03-19T00:41:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- 172.21.15.7:0/1363668965 wait complete. 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- start start 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 Processor -- start 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6ae3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 0x5615c6ae3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea201a6780 con 0x5615c6accd60 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fea201a7bf0 con 0x5615c6acb9f0 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea201a6e50 con 0x5615c6ada680 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea2a38c740 10 client.0 _ll_register_callbacks cb 0x7fea10009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.947+0000 7fea24d44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 0x5615c6ae3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 0x5615c6ae3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54068/0 (socket says 172.21.15.7:54068) 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 -- 172.21.15.7:0/2432155276 learned_addr learned my addr 172.21.15.7:0/2432155276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea25545640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6ae3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea25d46640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 -- 172.21.15.7:0/2432155276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 msgr2=0x5615c6ae5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 -- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 msgr2=0x5615c6ae3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6ae3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 -- 172.21.15.7:0/2432155276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c6ae55f0 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea24d44640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 0x5615c6ae3ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea10005890 tx=0x7fea10012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da224f4a9c81bcec server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea08ff9640 1 -- 172.21.15.7:0/2432155276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10004250 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea08ff9640 1 -- 172.21.15.7:0/2432155276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fea100043f0 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea18018040 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea08ff9640 1 -- 172.21.15.7:0/2432155276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea10031050 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea2a38c740 10 client.16200 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.948+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615c6b50db0 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea08ff9640 1 -- 172.21.15.7:0/2432155276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea18018040 con 0x5615c6accd60 2024-03-19T00:41:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea08ff9640 1 -- 172.21.15.7:0/2432155276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fea100204e0 con 0x5615c6accd60 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea2a38c740 10 client.16200 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea2a38c740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea2a38c740 20 client.16200 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea2a38c740 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6b51860 0x5615c6b53c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea2a38c740 4 client.16200 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5615c6a3c9f0 con 0x5615c6b51860 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea25545640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6b51860 0x5615c6b53c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea25545640 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6b51860 0x5615c6b53c80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b1ebaa3d07bb3c server_cookie=586442905b0c9089 in_seq=0 out_seq=0 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.949+0000 7fea08ff9640 10 client.16200 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea08ff9640 1 -- 172.21.15.7:0/2432155276 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5615c6a3c9f0 con 0x5615c6b51860 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea08ff9640 10 client.16200 handle_command_reply: tid=0 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 client.16200 shutdown 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 msgr2=0x5615c6ae3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 0x5615c6ae3ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea10005890 tx=0x7fea10012b40 comp rx=0 tx=0).stop 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 shutdown_connections 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615c6ada680 0x5615c6ae5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5615c6b51860 0x5615c6b53c80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615c6accd60 0x5615c6ae3ff0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 --2- 172.21.15.7:0/2432155276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615c6acb9f0 0x5615c6ae3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 >> 172.21.15.7:0/2432155276 conn(0x5615c6acdb20 msgr2=0x5615c6adec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea08ff9640 0 client.16200 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 shutdown_connections 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 1 -- 172.21.15.7:0/2432155276 wait complete. 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.951+0000 7fea2a38c740 20 client.16200 trim_cache size 0 max 16384 2024-03-19T00:41:12.959 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:41:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:41:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7cdb768a-206d-48b3-ac8d-291d4d8413ea", 2024-03-19T00:41:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:41:12.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:41:12.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fea0c0a2800 msgr2=0x7fea0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fea0c0a2800 0x7fea0c0a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fea18005fd0 tx=0x7fea18011bc0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 msgr2=0x7fea20196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20196990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 shutdown_connections 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fea0c0a2800 0x7fea0c0a4cc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea201022f0 0x7fea2019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea20101930 0x7fea20196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 --2- 172.21.15.7:0/2942972402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea20100740 0x7fea20196990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 >> 172.21.15.7:0/2942972402 conn(0x7fea200fbcb0 msgr2=0x7fea20104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 shutdown_connections 2024-03-19T00:41:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:12.952+0000 7fea277a9640 1 -- 172.21.15.7:0/2942972402 wait complete. 2024-03-19T00:41:12.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:41:12.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:41:13.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 -- 172.21.15.7:0/2892180203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe481022f0 msgr2=0x7efe48108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2892180203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe481022f0 0x7efe48108b80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efe40009ca0 tx=0x7efe4002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 -- 172.21.15.7:0/2892180203 shutdown_connections 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2892180203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe481022f0 0x7efe48108b80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2892180203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe48101930 0x7efe48101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2892180203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe48100740 0x7efe48100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 -- 172.21.15.7:0/2892180203 >> 172.21.15.7:0/2892180203 conn(0x7efe480fbcb0 msgr2=0x7efe480fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 -- 172.21.15.7:0/2892180203 shutdown_connections 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 -- 172.21.15.7:0/2892180203 wait complete. 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 Processor -- start 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 -- start start 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.068+0000 7efe4fe44640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe48100740 0x7efe48069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 0x7efe4819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe481022f0 0x7efe4819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe4810c270 con 0x7efe48101930 2024-03-19T00:41:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe4810c0f0 con 0x7efe481022f0 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe4810c3f0 con 0x7efe48100740 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 0x7efe4819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 0x7efe4819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54090/0 (socket says 172.21.15.7:54090) 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 -- 172.21.15.7:0/2278396168 learned_addr learned my addr 172.21.15.7:0/2278396168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe48100740 0x7efe48069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe481022f0 0x7efe4819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 -- 172.21.15.7:0/2278396168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe48100740 msgr2=0x7efe48069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe48100740 0x7efe48069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 -- 172.21.15.7:0/2278396168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe481022f0 msgr2=0x7efe4819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe481022f0 0x7efe4819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 -- 172.21.15.7:0/2278396168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4819f640 con 0x7efe48101930 2024-03-19T00:41:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4d3df640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 0x7efe4819c740 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efe3800ea60 tx=0x7efe3800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe3effd640 1 -- 172.21.15.7:0/2278396168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3800cde0 con 0x7efe48101930 2024-03-19T00:41:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4819f930 con 0x7efe48101930 2024-03-19T00:41:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe3effd640 1 -- 172.21.15.7:0/2278396168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe380035f0 con 0x7efe48101930 2024-03-19T00:41:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe3effd640 1 -- 172.21.15.7:0/2278396168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38010690 con 0x7efe48101930 2024-03-19T00:41:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.069+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4819fed0 con 0x7efe48101930 2024-03-19T00:41:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe3effd640 1 -- 172.21.15.7:0/2278396168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7efe380037a0 con 0x7efe48101930 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 Processor -- start 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 -- start start 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad45f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad464350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad466e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3ad3bf500 con 0x55f3ad4649c0 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f3ad367be0 con 0x55f3ad45f850 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe52a2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3ad3bb5a0 con 0x55f3ad45ee60 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe4d3df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad466e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe4dbe0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad464350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe4dbe0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad464350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57656/0 (socket says 172.21.15.7:57656) 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe4dbe0640 1 -- 172.21.15.7:0/96419118 learned_addr learned my addr 172.21.15.7:0/96419118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.072+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad45f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe3effd640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe340a2800 0x7efe340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4dbe0640 1 -- 172.21.15.7:0/96419118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 msgr2=0x55f3ad45f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad45f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4dbe0640 1 -- 172.21.15.7:0/96419118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 msgr2=0x55f3ad466e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad466e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4dbe0640 1 -- 172.21.15.7:0/96419118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ad389000 con 0x55f3ad45f850 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe340a2800 0x7efe340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe3effd640 1 -- 172.21.15.7:0/2278396168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efe380cf050 con 0x7efe48101930 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad464350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe44009c80 tx=0x7efe44011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a652f38d869e8733 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe340a2800 0x7efe340a4cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe40006010 tx=0x7efe4002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe33fff640 1 -- 172.21.15.7:0/96419118 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44004220 con 0x55f3ad45f850 2024-03-19T00:41:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe33fff640 1 -- 172.21.15.7:0/96419118 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe440043c0 con 0x55f3ad45f850 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.073+0000 7efe33fff640 1 -- 172.21.15.7:0/96419118 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe440197f0 con 0x55f3ad45f850 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 -- 172.21.15.7:0/96419118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 msgr2=0x55f3ad464350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad464350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe44009c80 tx=0x7efe44011810 comp rx=0 tx=0).stop 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 -- 172.21.15.7:0/96419118 shutdown_connections 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad466e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad464350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 --2- 172.21.15.7:0/96419118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad45f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 -- 172.21.15.7:0/96419118 >> 172.21.15.7:0/96419118 conn(0x55f3ad455fc0 msgr2=0x55f3ad4563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 -- 172.21.15.7:0/96419118 shutdown_connections 2024-03-19T00:41:13.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 -- 172.21.15.7:0/96419118 wait complete. 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 -- start start 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.074+0000 7efe52a2d740 1 Processor -- start 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 0x55f3ad469310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45f850 0x55f3ad469850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad470820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3ad467970 con 0x55f3ad45ee60 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f3ad37b160 con 0x55f3ad4649c0 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3ad467af0 con 0x55f3ad45f850 2024-03-19T00:41:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe52a2d740 10 client.0 _ll_register_callbacks cb 0x55f3ad3b7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 0x55f3ad469310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 0x55f3ad469310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54112/0 (socket says 172.21.15.7:54112) 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 -- 172.21.15.7:0/4174975190 learned_addr learned my addr 172.21.15.7:0/4174975190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad470820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4d3df640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45f850 0x55f3ad469850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 -- 172.21.15.7:0/4174975190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45f850 msgr2=0x55f3ad469850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45f850 0x55f3ad469850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 -- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 msgr2=0x55f3ad470820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad470820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 -- 172.21.15.7:0/4174975190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ad19d7d0 con 0x55f3ad45ee60 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad470820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 0x55f3ad469310 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efe44003d80 tx=0x7efe44003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980c3e1bb754af9a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.075+0000 7efe317fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe317fa640 1 -- 172.21.15.7:0/4174975190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44028070 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe440043c0 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe317fa640 1 -- 172.21.15.7:0/4174975190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efe44005040 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe317fa640 1 -- 172.21.15.7:0/4174975190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe44034050 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 10 client.16212 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3ad0d33f0 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe317fa640 1 -- 172.21.15.7:0/4174975190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe440043c0 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe317fa640 1 -- 172.21.15.7:0/4174975190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7efe44004ad0 con 0x55f3ad45ee60 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 10 client.16212 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:13.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 20 client.16212 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad474e80 0x55f3ad477300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 4 client.16212 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ad3cca40 con 0x55f3ad474e80 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.076+0000 7efe4d3df640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad474e80 0x55f3ad477300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.077+0000 7efe4d3df640 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad474e80 0x55f3ad477300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b821e6ea9310df6 server_cookie=133a64cff077e980 in_seq=0 out_seq=0 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.077+0000 7efe317fa640 10 client.16212 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.077+0000 7efe317fa640 1 -- 172.21.15.7:0/4174975190 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f3ad3cca40 con 0x55f3ad474e80 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.077+0000 7efe317fa640 10 client.16212 handle_command_reply: tid=0 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.077+0000 7efe52a2d740 1 client.16212 shutdown 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 msgr2=0x55f3ad469310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 0x55f3ad469310 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efe44003d80 tx=0x7efe44003e50 comp rx=0 tx=0).stop 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 shutdown_connections 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad470820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45f850 0x55f3ad469850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad474e80 0x55f3ad477300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 --2- 172.21.15.7:0/4174975190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45ee60 0x55f3ad469310 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 >> 172.21.15.7:0/4174975190 conn(0x55f3ad455fc0 msgr2=0x55f3ad4551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe317fa640 0 client.16212 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 shutdown_connections 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 1 -- 172.21.15.7:0/4174975190 wait complete. 2024-03-19T00:41:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.078+0000 7efe52a2d740 20 client.16212 trim_cache size 0 max 16384 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 Processor -- start 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 -- start start 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad450f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad4707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad468930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe481aa9b0 con 0x55f3ad4649c0 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe481a8280 con 0x55f3ad45f850 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe52a2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe481a60d0 con 0x55f3ad45ee60 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe4e3e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad468930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe4e3e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad468930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54118/0 (socket says 172.21.15.7:54118) 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe4e3e1640 1 -- 172.21.15.7:0/1819862365 learned_addr learned my addr 172.21.15.7:0/1819862365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe4d3df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad4707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.095+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad450f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe4e3e1640 1 -- 172.21.15.7:0/1819862365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 msgr2=0x55f3ad450f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad450f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe4e3e1640 1 -- 172.21.15.7:0/1819862365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 msgr2=0x55f3ad4707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad4707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe4e3e1640 1 -- 172.21.15.7:0/1819862365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ad0930f0 con 0x55f3ad4649c0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad468930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efe40036000 tx=0x7efe40041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f30cb6abc5673f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe337fe640 1 -- 172.21.15.7:0/1819862365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40039040 con 0x55f3ad4649c0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe337fe640 1 -- 172.21.15.7:0/1819862365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe40046030 con 0x55f3ad4649c0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.096+0000 7efe337fe640 1 -- 172.21.15.7:0/1819862365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4004e420 con 0x55f3ad4649c0 2024-03-19T00:41:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- 172.21.15.7:0/1819862365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 msgr2=0x55f3ad468930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad468930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efe40036000 tx=0x7efe40041000 comp rx=0 tx=0).stop 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- 172.21.15.7:0/1819862365 shutdown_connections 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad4649c0 0x55f3ad468930 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad45f850 0x55f3ad4707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- 172.21.15.7:0/1819862365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad450f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- 172.21.15.7:0/1819862365 >> 172.21.15.7:0/1819862365 conn(0x55f3ad455fc0 msgr2=0x55f3ad459ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- 172.21.15.7:0/1819862365 shutdown_connections 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- 172.21.15.7:0/1819862365 wait complete. 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- start start 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 Processor -- start 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad46dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 0x55f3ad46e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad46f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe481a6740 con 0x55f3ad45f850 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe481a7bb0 con 0x55f3ad4649c0 2024-03-19T00:41:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe481a6e10 con 0x55f3ad45ee60 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe52a2d740 10 client.0 _ll_register_callbacks cb 0x7efe380d2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe4d3df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 0x55f3ad46e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe4dbe0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad46dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.097+0000 7efe4d3df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 0x55f3ad46e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54130/0 (socket says 172.21.15.7:54130) 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4d3df640 1 -- 172.21.15.7:0/2459361664 learned_addr learned my addr 172.21.15.7:0/2459361664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4e3e1640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad46f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4dbe0640 1 -- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 msgr2=0x55f3ad46f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad46f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4dbe0640 1 -- 172.21.15.7:0/2459361664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 msgr2=0x55f3ad46e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 0x55f3ad46e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4dbe0640 1 -- 172.21.15.7:0/2459361664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ad46f770 con 0x55f3ad45ee60 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4d3df640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 0x55f3ad46e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe4dbe0640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad46dc80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe44011850 tx=0x7efe440198b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ecba3dadbb8389c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe317fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe317fa640 1 -- 172.21.15.7:0/2459361664 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44028040 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe40046030 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe317fa640 1 -- 172.21.15.7:0/2459361664 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efe44004fa0 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe317fa640 1 -- 172.21.15.7:0/2459361664 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe44003800 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe52a2d740 10 client.25978 fetch_fsmap learned FSMap version 26 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.098+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3ad470a80 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe317fa640 1 -- 172.21.15.7:0/2459361664 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe40046030 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe317fa640 1 -- 172.21.15.7:0/2459361664 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7efe440043a0 con 0x55f3ad45ee60 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 10 client.25978 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 20 client.25978 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad4db8c0 0x55f3ad4ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 4 client.25978 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:41:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3ad3c6d90 con 0x55f3ad4db8c0 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe4d3df640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad4db8c0 0x55f3ad4ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe317fa640 10 client.25978 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe4d3df640 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad4db8c0 0x55f3ad4ddce0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f45f54516bc3acb server_cookie=97ce6dc783db396d in_seq=0 out_seq=0 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe317fa640 1 -- 172.21.15.7:0/2459361664 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3ad3c6d90 con 0x55f3ad4db8c0 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe317fa640 10 client.25978 handle_command_reply: tid=0 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.099+0000 7efe52a2d740 1 client.25978 shutdown 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 msgr2=0x55f3ad46dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad46dc80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe44011850 tx=0x7efe440198b0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 shutdown_connections 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3ad4649c0 0x55f3ad46f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3ad4db8c0 0x55f3ad4ddce0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3ad45f850 0x55f3ad46e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 --2- 172.21.15.7:0/2459361664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3ad45ee60 0x55f3ad46dc80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe317fa640 0 client.25978 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 >> 172.21.15.7:0/2459361664 conn(0x55f3ad455fc0 msgr2=0x55f3ad454950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 shutdown_connections 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 1 -- 172.21.15.7:0/2459361664 wait complete. 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe52a2d740 20 client.25978 trim_cache size 0 max 16384 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "7cdb768a-206d-48b3-ac8d-291d4d8413ea": { 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7cdb768a-206d-48b3-ac8d-291d4d8413ea", 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:41:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:41:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:41:13.109 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:41:13.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe340a2800 msgr2=0x7efe340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe340a2800 0x7efe340a4cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efe40006010 tx=0x7efe4002e040 comp rx=0 tx=0).stop 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 msgr2=0x7efe4819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 0x7efe4819c740 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efe3800ea60 tx=0x7efe3800ef20 comp rx=0 tx=0).stop 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 shutdown_connections 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efe340a2800 0x7efe340a4cc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe481022f0 0x7efe4819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48101930 0x7efe4819c740 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 --2- 172.21.15.7:0/2278396168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe48100740 0x7efe48069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 >> 172.21.15.7:0/2278396168 conn(0x7efe480fbcb0 msgr2=0x7efe48104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 shutdown_connections 2024-03-19T00:41:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:13.100+0000 7efe4fe44640 1 -- 172.21.15.7:0/2278396168 wait complete. 2024-03-19T00:41:13.111 INFO:tasks.cephfs.filesystem:scrub status for tag:7cdb768a-206d-48b3-ac8d-291d4d8413ea - {'path': '/', 'tag': '7cdb768a-206d-48b3-ac8d-291d4d8413ea', 'options': 'recursive,force'} 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: pgmap v449: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:13.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:13 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: pgmap v449: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:13 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: pgmap v449: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:13 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:15.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:15 smithi044 ceph-mon[66366]: pgmap v450: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:41:15.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:15 smithi007 ceph-mon[62854]: pgmap v450: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:41:16.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:15 smithi204 ceph-mon[66572]: pgmap v450: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:41:17.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:17 smithi044 ceph-mon[66366]: pgmap v451: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:17.905 INFO:tasks.workunit.client.0.smithi007.stdout:64/600: open fd = 3 2024-03-19T00:41:17.905 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:65/600: open fd = 3 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:66/600: open fd = 3 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:67/600: open fd = 3 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:68/600: open fd = 3 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:69/600: open fd = 3 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:70/600: open fd = 3 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:71/600: open fd = 3 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.907 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:72/600: open fd = 3 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:73/600: open fd = 3 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:74/600: open fd = 3 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:75/600: open fd = 3 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:76/600: open fd = 3 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:77/600: open fd = 3 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:78/600: open fd = 3 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:79/600: open fd = 3 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:80/600: open fd = 3 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:81/600: open fd = 3 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:82/600: open fd = 3 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:83/600: open fd = 3 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:84/600: open fd = 3 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:85/600: open fd = 3 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.911 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:86/600: open fd = 3 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:87/600: open fd = 3 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:88/600: open fd = 3 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.912 INFO:tasks.workunit.client.0.smithi007.stdout:89/600: open fd = 3 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:90/600: open fd = 3 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:91/600: open fd = 3 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:92/600: open fd = 3 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:93/600: open fd = 3 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:94/600: open fd = 3 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:95/600: open fd = 3 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:96/600: open fd = 3 2024-03-19T00:41:17.914 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:97/600: open fd = 3 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:98/600: open fd = 3 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:99/600: open fd = 3 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:100/600: open fd = 3 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:101/600: open fd = 3 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:102/600: open fd = 3 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:103/600: open fd = 3 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:104/600: open fd = 3 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:105/600: open fd = 3 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:106/600: open fd = 3 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:107/600: open fd = 3 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:108/600: open fd = 3 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:109/600: open fd = 3 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:110/600: open fd = 3 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:111/600: open fd = 3 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:112/600: open fd = 3 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:113/600: open fd = 3 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:114/600: open fd = 3 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:115/600: open fd = 3 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:116/600: open fd = 3 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:117/600: open fd = 3 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:118/600: open fd = 3 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:119/600: open fd = 3 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:120/600: open fd = 3 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:121/600: open fd = 3 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:122/600: open fd = 3 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:123/600: open fd = 3 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:124/600: open fd = 3 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:125/600: open fd = 3 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.922 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.923 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:41:17.923 INFO:tasks.workunit.client.0.smithi007.stdout:126/600: open fd = 3 2024-03-19T00:41:17.923 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:41:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:17 smithi007 ceph-mon[62854]: pgmap v451: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:18.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:17 smithi204 ceph-mon[66572]: pgmap v451: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:19.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:19 smithi044 ceph-mon[66366]: pgmap v452: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:19.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:19 smithi007 ceph-mon[62854]: pgmap v452: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:20.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:19 smithi204 ceph-mon[66572]: pgmap v452: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:21 smithi044 ceph-mon[66366]: pgmap v453: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:41:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:21 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:41:21.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:21 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:41:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:21 smithi007 ceph-mon[62854]: pgmap v453: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:41:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:21 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:41:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:21 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:41:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:21 smithi204 ceph-mon[66572]: pgmap v453: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:41:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:21 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:41:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:21 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:41:22.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:22 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:41:23.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:22 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:41:23.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:22 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:41:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:23 smithi007 ceph-mon[62854]: pgmap v454: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:41:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:23 smithi204 ceph-mon[66572]: pgmap v454: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:41:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:23 smithi044 ceph-mon[66366]: pgmap v454: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:41:25.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:25 smithi007 ceph-mon[62854]: pgmap v455: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.6 KiB/s wr, 15 op/s 2024-03-19T00:41:26.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:25 smithi204 ceph-mon[66572]: pgmap v455: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.6 KiB/s wr, 15 op/s 2024-03-19T00:41:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:25 smithi044 ceph-mon[66366]: pgmap v455: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.6 KiB/s wr, 15 op/s 2024-03-19T00:41:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:27 smithi007 ceph-mon[62854]: pgmap v456: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:27 smithi204 ceph-mon[66572]: pgmap v456: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:27 smithi044 ceph-mon[66366]: pgmap v456: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:29 smithi007 ceph-mon[62854]: pgmap v457: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:29 smithi204 ceph-mon[66572]: pgmap v457: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:29 smithi044 ceph-mon[66366]: pgmap v457: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:31 smithi007 ceph-mon[62854]: pgmap v458: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:41:32.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:31 smithi204 ceph-mon[66572]: pgmap v458: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:41:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:31 smithi044 ceph-mon[66366]: pgmap v458: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:41:33.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:33 smithi007 ceph-mon[62854]: pgmap v459: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:33 smithi204 ceph-mon[66572]: pgmap v459: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:34.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:33 smithi044 ceph-mon[66366]: pgmap v459: 129 pgs: 129 active+clean; 1.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:35 smithi007 ceph-mon[62854]: pgmap v460: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:41:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:35 smithi204 ceph-mon[66572]: pgmap v460: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:41:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:35 smithi044 ceph-mon[66366]: pgmap v460: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:41:37.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:37 smithi007 ceph-mon[62854]: pgmap v461: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:38.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:37 smithi204 ceph-mon[66572]: pgmap v461: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:37 smithi044 ceph-mon[66366]: pgmap v461: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:39.908 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:39 smithi044 ceph-mon[66366]: pgmap v462: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:39 smithi007 ceph-mon[62854]: pgmap v462: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:39 smithi204 ceph-mon[66572]: pgmap v462: 129 pgs: 129 active+clean; 1.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:41.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:41 smithi007 ceph-mon[62854]: pgmap v463: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:41:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:41 smithi204 ceph-mon[66572]: pgmap v463: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:41:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:41 smithi044 ceph-mon[66366]: pgmap v463: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:41:43.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:41:43.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:41:43.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 -- 172.21.15.7:0/4146844714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 msgr2=0x7fc51c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4146844714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c100b60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009900 tx=0x7fc50c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 -- 172.21.15.7:0/4146844714 shutdown_connections 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4146844714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c1022f0 0x7fc51c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4146844714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc51c101930 0x7fc51c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4146844714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c100b60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 -- 172.21.15.7:0/4146844714 >> 172.21.15.7:0/4146844714 conn(0x7fc51c0fbcb0 msgr2=0x7fc51c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 -- 172.21.15.7:0/4146844714 shutdown_connections 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.231+0000 7fc521c4b640 1 -- 172.21.15.7:0/4146844714 wait complete. 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 Processor -- start 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 -- start start 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c101930 0x7fc51c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc51c1022f0 0x7fc51c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c10c240 con 0x7fc51c100740 2024-03-19T00:41:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c10c0c0 con 0x7fc51c1022f0 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc521c4b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c10c3c0 con 0x7fc51c101930 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55002/0 (socket says 172.21.15.7:55002) 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 -- 172.21.15.7:0/4134986564 learned_addr learned my addr 172.21.15.7:0/4134986564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51bfff640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc51c1022f0 0x7fc51c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51affd640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c101930 0x7fc51c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 -- 172.21.15.7:0/4134986564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c101930 msgr2=0x7fc51c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c101930 0x7fc51c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 -- 172.21.15.7:0/4134986564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc51c1022f0 msgr2=0x7fc51c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc51c1022f0 0x7fc51c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.232+0000 7fc51b7fe640 1 -- 172.21.15.7:0/4134986564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c1a1900 con 0x7fc51c100740 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.233+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c199cf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000c00 tx=0x7fc50c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.233+0000 7fc518ff9640 1 -- 172.21.15.7:0/4134986564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c043020 con 0x7fc51c100740 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.233+0000 7fc518ff9640 1 -- 172.21.15.7:0/4134986564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc50c004440 con 0x7fc51c100740 2024-03-19T00:41:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.233+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c1a1b90 con 0x7fc51c100740 2024-03-19T00:41:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.233+0000 7fc518ff9640 1 -- 172.21.15.7:0/4134986564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003c90 con 0x7fc51c100740 2024-03-19T00:41:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.233+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c1a20e0 con 0x7fc51c100740 2024-03-19T00:41:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.235+0000 7fc518ff9640 1 -- 172.21.15.7:0/4134986564 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fc50c046020 con 0x7fc51c100740 2024-03-19T00:41:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.235+0000 7fc524832740 1 Processor -- start 2024-03-19T00:41:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.235+0000 7fc524832740 1 -- start start 2024-03-19T00:41:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc518ff9640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5040a2800 0x7fc5040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc524832740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e2856e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc524832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e285bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc524832740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e285c5b0 0x5605e285ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc524832740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5605e27bb500 con 0x5605e2856a50 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc524832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5605e2763be0 con 0x5605e2857440 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc524832740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5605e27b75a0 con 0x5605e285c5b0 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e2856e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51bfff640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5040a2800 0x7fc5040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e285bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e2856e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55008/0 (socket says 172.21.15.7:55008) 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51b7fe640 1 -- 172.21.15.7:0/4130561680 learned_addr learned my addr 172.21.15.7:0/4130561680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e285bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47146/0 (socket says 172.21.15.7:47146) 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc518ff9640 1 -- 172.21.15.7:0/4134986564 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc50c0f1060 con 0x7fc51c100740 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51bfff640 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e285c5b0 0x5605e285ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 -- 172.21.15.7:0/4130561680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e285c5b0 msgr2=0x5605e285ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e285c5b0 0x5605e285ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 -- 172.21.15.7:0/4130561680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 msgr2=0x5605e2856e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e2856e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 -- 172.21.15.7:0/4130561680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e2785000 con 0x5605e2857440 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e2856e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51bfff640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5040a2800 0x7fc5040a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc514005fd0 tx=0x7fc514009e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.236+0000 7fc51affd640 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e285bf40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc510009ba0 tx=0x7fc510011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b646d341ee969f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc501ffb640 1 -- 172.21.15.7:0/4130561680 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510004050 con 0x5605e2857440 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc501ffb640 1 -- 172.21.15.7:0/4130561680 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5100054d0 con 0x5605e2857440 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc501ffb640 1 -- 172.21.15.7:0/4130561680 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510004cf0 con 0x5605e2857440 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 -- 172.21.15.7:0/4130561680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 msgr2=0x5605e285bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e285bf40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc510009ba0 tx=0x7fc510011810 comp rx=0 tx=0).stop 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 -- 172.21.15.7:0/4130561680 shutdown_connections 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e285c5b0 0x5605e285ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e285bf40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 --2- 172.21.15.7:0/4130561680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e2856e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 -- 172.21.15.7:0/4130561680 >> 172.21.15.7:0/4130561680 conn(0x5605e2851fc0 msgr2=0x5605e28523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 -- 172.21.15.7:0/4130561680 shutdown_connections 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.237+0000 7fc524832740 1 -- 172.21.15.7:0/4130561680 wait complete. 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 -- start start 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 Processor -- start 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2856a50 0x5605e2860ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e28613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e2868420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5605e285f560 con 0x5605e285c5b0 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5605e2777160 con 0x5605e2857440 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5605e285f6e0 con 0x5605e2856a50 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc524832740 10 client.0 _ll_register_callbacks cb 0x5605e27b3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e2868420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc51affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e28613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e2868420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55018/0 (socket says 172.21.15.7:55018) 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc51bfff640 1 -- 172.21.15.7:0/3188579191 learned_addr learned my addr 172.21.15.7:0/3188579191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.238+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2856a50 0x5605e2860ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51bfff640 1 -- 172.21.15.7:0/3188579191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2856a50 msgr2=0x5605e2860ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51bfff640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2856a50 0x5605e2860ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51bfff640 1 -- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 msgr2=0x5605e28613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51bfff640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e28613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51bfff640 1 -- 172.21.15.7:0/3188579191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e25997d0 con 0x5605e285c5b0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51affd640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e28613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc51bfff640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e2868420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc514003b40 tx=0x7fc514004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c36fd762b9b017a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc4faffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc4faffd640 1 -- 172.21.15.7:0/3188579191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514024070 con 0x5605e285c5b0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc4faffd640 1 -- 172.21.15.7:0/3188579191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc514010af0 con 0x5605e285c5b0 2024-03-19T00:41:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5100054d0 con 0x5605e285c5b0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc4faffd640 1 -- 172.21.15.7:0/3188579191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc514010cf0 con 0x5605e285c5b0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc524832740 10 client.16230 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605e24cf3f0 con 0x5605e285c5b0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc4faffd640 1 -- 172.21.15.7:0/3188579191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5100054d0 con 0x5605e285c5b0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.239+0000 7fc4faffd640 1 -- 172.21.15.7:0/3188579191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc51401f070 con 0x5605e285c5b0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc524832740 10 client.16230 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc524832740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc524832740 20 client.16230 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc524832740 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e286cda0 0x5605e286f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc524832740 4 client.16230 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605e27c8a40 con 0x5605e286cda0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e286cda0 0x5605e286f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc4faffd640 10 client.16230 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.240+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e286cda0 0x5605e286f1f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634f28b5b6b85313 server_cookie=52e81e6cbeac718 in_seq=0 out_seq=0 2024-03-19T00:41:43.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc4faffd640 1 -- 172.21.15.7:0/3188579191 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5605e27c8a40 con 0x5605e286cda0 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc4faffd640 10 client.16230 handle_command_reply: tid=0 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 client.16230 shutdown 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 msgr2=0x5605e2868420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e2868420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc514003b40 tx=0x7fc514004ab0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 shutdown_connections 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e286cda0 0x5605e286f1f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e2868420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2857440 0x5605e28613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc4faffd640 0 client.16230 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 --2- 172.21.15.7:0/3188579191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2856a50 0x5605e2860ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 >> 172.21.15.7:0/3188579191 conn(0x5605e2851fc0 msgr2=0x5605e28511e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 shutdown_connections 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 1 -- 172.21.15.7:0/3188579191 wait complete. 2024-03-19T00:41:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.241+0000 7fc524832740 20 client.16230 trim_cache size 0 max 16384 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 Processor -- start 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 -- start start 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2856a50 0x5605e284d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e28588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1aed30 con 0x5605e285c5b0 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1ac600 con 0x5605e2856a50 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc524832740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1aa450 con 0x5605e2857440 2024-03-19T00:41:43.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e28588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e28588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55026/0 (socket says 172.21.15.7:55026) 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2856a50 0x5605e284d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 -- 172.21.15.7:0/3539643288 learned_addr learned my addr 172.21.15.7:0/3539643288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 -- 172.21.15.7:0/3539643288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 msgr2=0x5605e28683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 -- 172.21.15.7:0/3539643288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2856a50 msgr2=0x5605e284d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2856a50 0x5605e284d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.259+0000 7fc51bfff640 1 -- 172.21.15.7:0/3539643288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e248f0f0 con 0x5605e285c5b0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc51bfff640 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e28588a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc514003ce0 tx=0x7fc514004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785c0797e27ebda4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc5017fa640 1 -- 172.21.15.7:0/3539643288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51403b070 con 0x5605e285c5b0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc5017fa640 1 -- 172.21.15.7:0/3539643288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc514009a60 con 0x5605e285c5b0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc5017fa640 1 -- 172.21.15.7:0/3539643288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514024040 con 0x5605e285c5b0 2024-03-19T00:41:43.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 -- 172.21.15.7:0/3539643288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 msgr2=0x5605e28588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e28588a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc514003ce0 tx=0x7fc514004040 comp rx=0 tx=0).stop 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 -- 172.21.15.7:0/3539643288 shutdown_connections 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e285c5b0 0x5605e28588a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 --2- 172.21.15.7:0/3539643288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e2856a50 0x5605e284d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 -- 172.21.15.7:0/3539643288 >> 172.21.15.7:0/3539643288 conn(0x5605e2851fc0 msgr2=0x5605e2860ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 -- 172.21.15.7:0/3539643288 shutdown_connections 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.260+0000 7fc524832740 1 -- 172.21.15.7:0/3539643288 wait complete. 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 -- start start 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 Processor -- start 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e28653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e285c5b0 0x5605e28d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1aaac0 con 0x5605e2856a50 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1abf30 con 0x5605e285c5b0 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1ab190 con 0x5605e2857440 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc524832740 10 client.0 _ll_register_callbacks cb 0x7fc514003bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e28653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e28653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55038/0 (socket says 172.21.15.7:55038) 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 -- 172.21.15.7:0/521659456 learned_addr learned my addr 172.21.15.7:0/521659456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51affd640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51bfff640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e285c5b0 0x5605e28d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 -- 172.21.15.7:0/521659456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 msgr2=0x5605e28658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 -- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e285c5b0 msgr2=0x5605e28d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.261+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e285c5b0 0x5605e28d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc51b7fe640 1 -- 172.21.15.7:0/521659456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e28d1620 con 0x5605e2856a50 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc51bfff640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e285c5b0 0x5605e28d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc51b7fe640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e28653a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ee800 tx=0x7fc50c005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404f4c742b0cbe18 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc4faffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc4faffd640 1 -- 172.21.15.7:0/521659456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b98b0 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc4faffd640 1 -- 172.21.15.7:0/521659456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc50c0b9c80 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc514009a60 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc4faffd640 1 -- 172.21.15.7:0/521659456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc50c0035f0 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605e28d29b0 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc4faffd640 1 -- 172.21.15.7:0/521659456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc514009a60 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc4faffd640 1 -- 172.21.15.7:0/521659456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc50c0c59b0 con 0x5605e2856a50 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 20 client.16236 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e28d32e0 0x5605e28d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 4 client.16236 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:41:43.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605e27c2d90 con 0x5605e28d32e0 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.262+0000 7fc51affd640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e28d32e0 0x5605e28d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc4faffd640 10 client.16236 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc51affd640 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e28d32e0 0x5605e28d5700 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a805bb4b81f408 server_cookie=e3bea1b261dc8f32 in_seq=0 out_seq=0 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc4faffd640 1 -- 172.21.15.7:0/521659456 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5605e27c2d90 con 0x5605e28d32e0 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc4faffd640 10 client.16236 handle_command_reply: tid=0 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 client.16236 shutdown 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 msgr2=0x5605e28653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e28653a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ee800 tx=0x7fc50c005500 comp rx=0 tx=0).stop 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 shutdown_connections 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605e285c5b0 0x5605e28d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e2857440 0x5605e28658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5605e28d32e0 0x5605e28d5700 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 --2- 172.21.15.7:0/521659456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5605e2856a50 0x5605e28653a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc4faffd640 0 client.16236 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 >> 172.21.15.7:0/521659456 conn(0x5605e2851fc0 msgr2=0x5605e2860ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 shutdown_connections 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 1 -- 172.21.15.7:0/521659456 wait complete. 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.263+0000 7fc524832740 20 client.16236 trim_cache size 0 max 16384 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:41:43.271 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5040a2800 msgr2=0x7fc5040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5040a2800 0x7fc5040a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc514005fd0 tx=0x7fc514009e50 comp rx=0 tx=0).stop 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 msgr2=0x7fc51c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c199cf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc50c000c00 tx=0x7fc50c004040 comp rx=0 tx=0).stop 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 shutdown_connections 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc5040a2800 0x7fc5040a4cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc51c1022f0 0x7fc51c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc51c101930 0x7fc51c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 --2- 172.21.15.7:0/4134986564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c100740 0x7fc51c199cf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 >> 172.21.15.7:0/4134986564 conn(0x7fc51c0fbcb0 msgr2=0x7fc51c068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 shutdown_connections 2024-03-19T00:41:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.264+0000 7fc521c4b640 1 -- 172.21.15.7:0/4134986564 wait complete. 2024-03-19T00:41:43.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:41:43.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 -- 172.21.15.7:0/1908771808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8102370 msgr2=0x7f22c8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/1908771808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8102370 0x7f22c8108c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009be0 tx=0x7f22c002bef0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 -- 172.21.15.7:0/1908771808 shutdown_connections 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/1908771808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c8102370 0x7f22c8108c00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/1908771808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22c81019b0 0x7f22c8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/1908771808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 -- 172.21.15.7:0/1908771808 >> 172.21.15.7:0/1908771808 conn(0x7f22c80fbd30 msgr2=0x7f22c80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 -- 172.21.15.7:0/1908771808 shutdown_connections 2024-03-19T00:41:43.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.378+0000 7f22cf7e6640 1 -- 172.21.15.7:0/1908771808 wait complete. 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 Processor -- start 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 -- start start 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c81019b0 0x7f22c819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22c8102370 0x7f22c819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22c810c280 con 0x7f22c81007c0 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22c810c100 con 0x7f22c81019b0 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cf7e6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22c810c400 con 0x7f22c8102370 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c806aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55056/0 (socket says 172.21.15.7:55056) 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 -- 172.21.15.7:0/863635443 learned_addr learned my addr 172.21.15.7:0/863635443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22ccd81640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c81019b0 0x7f22c819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cdd83640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22c8102370 0x7f22c819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 -- 172.21.15.7:0/863635443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22c8102370 msgr2=0x7f22c819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22c8102370 0x7f22c819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 -- 172.21.15.7:0/863635443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c81019b0 msgr2=0x7f22c819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c81019b0 0x7f22c819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 -- 172.21.15.7:0/863635443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c819f6a0 con 0x7f22c81007c0 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.379+0000 7f22cd582640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c806aab0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e9a0 tx=0x7f22c400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.380+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400cd80 con 0x7f22c81007c0 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.380+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22c40108e0 con 0x7f22c81007c0 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.380+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c819f930 con 0x7f22c81007c0 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.380+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4018950 con 0x7f22c81007c0 2024-03-19T00:41:43.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.380+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c819fe80 con 0x7f22c81007c0 2024-03-19T00:41:43.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.381+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c8107ae0 con 0x7f22c81007c0 2024-03-19T00:41:43.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.382+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f22c4010a90 con 0x7f22c81007c0 2024-03-19T00:41:43.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.382+0000 7f22be7fc640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22b40a2800 0x7f22b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.383+0000 7f22ccd81640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22b40a2800 0x7f22b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.383+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f22c40cf8b0 con 0x7f22c81007c0 2024-03-19T00:41:43.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.383+0000 7f22ccd81640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22b40a2800 0x7f22b40a4cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22c806a820 tx=0x7f22b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.386+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22c4026070 con 0x7f22c81007c0 2024-03-19T00:41:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.611+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22c802d3b0 con 0x7f22c81007c0 2024-03-19T00:41:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.612+0000 7f22be7fc640 1 -- 172.21.15.7:0/863635443 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f22c409c590 con 0x7f22c81007c0 2024-03-19T00:41:43.613 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:41:43.613 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:41:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:41:43.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22b40a2800 msgr2=0x7f22b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22b40a2800 0x7f22b40a4cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22c806a820 tx=0x7f22b8005f90 comp rx=0 tx=0).stop 2024-03-19T00:41:43.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 msgr2=0x7f22c806aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c806aab0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e9a0 tx=0x7f22c400ee60 comp rx=0 tx=0).stop 2024-03-19T00:41:43.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 shutdown_connections 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f22b40a2800 0x7f22b40a4cc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22c8102370 0x7f22c819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22c81019b0 0x7f22c819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 --2- 172.21.15.7:0/863635443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c81007c0 0x7f22c806aab0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 >> 172.21.15.7:0/863635443 conn(0x7f22c80fbd30 msgr2=0x7f22c8104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 shutdown_connections 2024-03-19T00:41:43.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.614+0000 7f22cf7e6640 1 -- 172.21.15.7:0/863635443 wait complete. 2024-03-19T00:41:43.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 -- 172.21.15.7:0/789758810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c100720 msgr2=0x7f7e0c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/789758810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c100720 0x7f7e0c100b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009900 tx=0x7f7dfc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 -- 172.21.15.7:0/789758810 shutdown_connections 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/789758810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c1022d0 0x7f7e0c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/789758810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e0c101910 0x7f7e0c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/789758810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c100720 0x7f7e0c100b40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 -- 172.21.15.7:0/789758810 >> 172.21.15.7:0/789758810 conn(0x7f7e0c0fbc50 msgr2=0x7f7e0c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.732+0000 7f7e11c4b640 1 -- 172.21.15.7:0/789758810 shutdown_connections 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 -- 172.21.15.7:0/789758810 wait complete. 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 Processor -- start 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 -- start start 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c100720 0x7f7e0c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 0x7f7e0c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e0c1022d0 0x7f7e0c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c10b110 con 0x7f7e0c101910 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c10af90 con 0x7f7e0c100720 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e11c4b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c10b290 con 0x7f7e0c1022d0 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 0x7f7e0c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.733+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 0x7f7e0c195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55078/0 (socket says 172.21.15.7:55078) 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 -- 172.21.15.7:0/1663779045 learned_addr learned my addr 172.21.15.7:0/1663779045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c100720 0x7f7e0c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e0c1022d0 0x7f7e0c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 -- 172.21.15.7:0/1663779045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e0c1022d0 msgr2=0x7f7e0c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e0c1022d0 0x7f7e0c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 -- 172.21.15.7:0/1663779045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c100720 msgr2=0x7f7e0c195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c100720 0x7f7e0c195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 -- 172.21.15.7:0/1663779045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c19d410 con 0x7f7e0c101910 2024-03-19T00:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c100720 0x7f7e0c195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 0x7f7e0c195d80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009be0 tx=0x7f7df40071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e097fa640 1 -- 172.21.15.7:0/1663779045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4007e00 con 0x7f7e0c101910 2024-03-19T00:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c19d700 con 0x7f7e0c101910 2024-03-19T00:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e097fa640 1 -- 172.21.15.7:0/1663779045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df401e070 con 0x7f7e0c101910 2024-03-19T00:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e097fa640 1 -- 172.21.15.7:0/1663779045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4018710 con 0x7f7e0c101910 2024-03-19T00:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.734+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c19dc50 con 0x7f7e0c101910 2024-03-19T00:41:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 Processor -- start 2024-03-19T00:41:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 -- start start 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a05710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eb449694b0 con 0x55eb44a052f0 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eb44911d70 con 0x55eb44a0ae50 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e14838740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb44965550 con 0x55eb44a05ce0 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a05710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a05710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55090/0 (socket says 172.21.15.7:55090) 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 -- 172.21.15.7:0/4231194590 learned_addr learned my addr 172.21.15.7:0/4231194590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 -- 172.21.15.7:0/4231194590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 msgr2=0x55eb44a0a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 -- 172.21.15.7:0/4231194590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 msgr2=0x55eb44a0d2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T00:41:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a0d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e097fa640 1 -- 172.21.15.7:0/1663779045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7df4019b30 con 0x7f7e0c101910 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.737+0000 7f7e0bfff640 1 -- 172.21.15.7:0/4231194590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb44934210 con 0x55eb44a052f0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a05710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400bf10 tx=0x7f7e04009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dde385e9915ab39 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e01ffb640 1 -- 172.21.15.7:0/4231194590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401a070 con 0x55eb44a052f0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e01ffb640 1 -- 172.21.15.7:0/4231194590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e04012e70 con 0x55eb44a052f0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e01ffb640 1 -- 172.21.15.7:0/4231194590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04005020 con 0x55eb44a052f0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e097fa640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df80a2800 0x7f7df80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df80a2800 0x7f7df80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.738+0000 7f7e097fa640 1 -- 172.21.15.7:0/1663779045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7df40d26d0 con 0x7f7e0c101910 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 -- 172.21.15.7:0/4231194590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 msgr2=0x55eb44a05710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df80a2800 0x7f7df80a4cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005ab0 tx=0x7f7dfc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a05710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400bf10 tx=0x7f7e04009260 comp rx=0 tx=0).stop 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 -- 172.21.15.7:0/4231194590 shutdown_connections 2024-03-19T00:41:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 --2- 172.21.15.7:0/4231194590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a05710 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 -- 172.21.15.7:0/4231194590 >> 172.21.15.7:0/4231194590 conn(0x55eb44a00860 msgr2=0x55eb44a00c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 -- 172.21.15.7:0/4231194590 shutdown_connections 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 -- 172.21.15.7:0/4231194590 wait complete. 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 -- start start 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.739+0000 7f7e14838740 1 Processor -- start 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a16d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eb44a0de00 con 0x55eb44a052f0 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eb44925310 con 0x55eb44a0ae50 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb44a0df80 con 0x55eb44a05ce0 2024-03-19T00:41:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e14838740 10 client.0 _ll_register_callbacks cb 0x55eb449192d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a0f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55098/0 (socket says 172.21.15.7:55098) 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e0b7fe640 1 -- 172.21.15.7:0/2014866537 learned_addr learned my addr 172.21.15.7:0/2014866537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a16d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 -- 172.21.15.7:0/2014866537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 msgr2=0x55eb44a16d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a16d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 -- 172.21.15.7:0/2014866537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 msgr2=0x55eb44a0f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a0f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 -- 172.21.15.7:0/2014866537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4467d3f0 con 0x55eb44a05ce0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7df40cfec0 tx=0x7f7df409e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87dd274ff10da408 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.740+0000 7f7deaffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:41:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7deaffd640 1 -- 172.21.15.7:0/2014866537 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df40a74b0 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7deaffd640 1 -- 172.21.15.7:0/2014866537 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7df409e160 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e04012e70 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7deaffd640 1 -- 172.21.15.7:0/2014866537 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7df409e360 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 10 client.26014 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb447477d0 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7deaffd640 1 -- 172.21.15.7:0/2014866537 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e04012e70 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7deaffd640 1 -- 172.21.15.7:0/2014866537 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7df409dc60 con 0x55eb44a05ce0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 10 client.26014 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 10 client.26014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 20 client.26014 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a1b6a0 0x55eb44a1da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 4 client.26014 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb449c73e0 con 0x55eb44a1b6a0 2024-03-19T00:41:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.741+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a1b6a0 0x55eb44a1da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.742+0000 7f7deaffd640 10 client.26014 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.742+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a1b6a0 0x55eb44a1da90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bfd3e4aee162a1b server_cookie=7e403e5d5e7da946 in_seq=0 out_seq=0 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7deaffd640 1 -- 172.21.15.7:0/2014866537 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55eb449c73e0 con 0x55eb44a1b6a0 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7deaffd640 10 client.26014 handle_command_reply: tid=0 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 client.26014 shutdown 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 msgr2=0x55eb44a0fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7df40cfec0 tx=0x7f7df409e730 comp rx=0 tx=0).stop 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 shutdown_connections 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a1b6a0 0x55eb44a1da90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a16d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a0fd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 --2- 172.21.15.7:0/2014866537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7deaffd640 0 client.26014 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 >> 172.21.15.7:0/2014866537 conn(0x55eb44a00860 msgr2=0x55eb449ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 shutdown_connections 2024-03-19T00:41:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 1 -- 172.21.15.7:0/2014866537 wait complete. 2024-03-19T00:41:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.743+0000 7f7e14838740 20 client.26014 trim_cache size 0 max 16384 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 Processor -- start 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 -- start start 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a052f0 0x55eb449ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 0x55eb44a16d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 0x55eb449ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1aa9b0 con 0x55eb44a0ae50 2024-03-19T00:41:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a8280 con 0x55eb44a05ce0 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.756+0000 7f7e14838740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a60d0 con 0x55eb44a052f0 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 0x55eb449ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 0x55eb44a16d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 0x55eb449ff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55112/0 (socket says 172.21.15.7:55112) 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 0x55eb44a16d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47220/0 (socket says 172.21.15.7:47220) 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e0bfff640 1 -- 172.21.15.7:0/157652214 learned_addr learned my addr 172.21.15.7:0/157652214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a052f0 0x55eb449ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 -- 172.21.15.7:0/157652214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a052f0 msgr2=0x55eb449ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a052f0 0x55eb449ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 -- 172.21.15.7:0/157652214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 msgr2=0x55eb449ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 0x55eb449ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 -- 172.21.15.7:0/157652214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb44a0a4d0 con 0x55eb44a05ce0 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 0x55eb449ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 0x55eb44a16d50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7df409cfc0 tx=0x7f7df409b0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f87b6a0cfc996e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e017fa640 1 -- 172.21.15.7:0/157652214 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df40a4140 con 0x55eb44a05ce0 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e017fa640 1 -- 172.21.15.7:0/157652214 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df409d010 con 0x55eb44a05ce0 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.757+0000 7f7e017fa640 1 -- 172.21.15.7:0/157652214 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df407dde0 con 0x55eb44a05ce0 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 -- 172.21.15.7:0/157652214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 msgr2=0x55eb44a16d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 0x55eb44a16d50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7df409cfc0 tx=0x7f7df409b0e0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 -- 172.21.15.7:0/157652214 shutdown_connections 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a0ae50 0x55eb449ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a05ce0 0x55eb44a16d50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 --2- 172.21.15.7:0/157652214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a052f0 0x55eb449ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 -- 172.21.15.7:0/157652214 >> 172.21.15.7:0/157652214 conn(0x55eb44a00860 msgr2=0x55eb44a0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 -- 172.21.15.7:0/157652214 shutdown_connections 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 -- 172.21.15.7:0/157652214 wait complete. 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.758+0000 7f7e14838740 1 -- start start 2024-03-19T00:41:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 Processor -- start 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a13de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a14320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a6740 con 0x55eb44a052f0 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a7bb0 con 0x55eb44a0ae50 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a6e10 con 0x55eb44a05ce0 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e14838740 10 client.0 _ll_register_callbacks cb 0x7f7df409ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a13de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a13de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55128/0 (socket says 172.21.15.7:55128) 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 -- 172.21.15.7:0/2429579553 learned_addr learned my addr 172.21.15.7:0/2429579553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a14320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0bfff640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 -- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 msgr2=0x55eb44a14320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a14320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 -- 172.21.15.7:0/2429579553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 msgr2=0x55eb44a152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 -- 172.21.15.7:0/2429579553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb44a158d0 con 0x55eb44a052f0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7e0b7fe640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a13de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc036040 tx=0x7f7dfc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3281bf2ffe578afb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.759+0000 7f7deaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7deaffd640 1 -- 172.21.15.7:0/2429579553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc056c90 con 0x55eb44a052f0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df409d010 con 0x55eb44a052f0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7deaffd640 1 -- 172.21.15.7:0/2429579553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7dfc056380 con 0x55eb44a052f0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7deaffd640 1 -- 172.21.15.7:0/2429579553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7dfc056e30 con 0x55eb44a052f0 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 10 client.16260 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:43.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb44a16c40 con 0x55eb44a052f0 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7deaffd640 1 -- 172.21.15.7:0/2429579553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7df409d010 con 0x55eb44a052f0 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7deaffd640 1 -- 172.21.15.7:0/2429579553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7dfc0035f0 con 0x55eb44a052f0 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 10 client.16260 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 10 client.16260 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 20 client.16260 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a81d20 0x55eb44a84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 4 client.16260 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.760+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb449769f0 con 0x55eb44a81d20 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a81d20 0x55eb44a84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7deaffd640 10 client.16260 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e02ffd640 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a81d20 0x55eb44a84140 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba987489f3b8212 server_cookie=a462b87908f1f3c9 in_seq=0 out_seq=0 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7deaffd640 1 -- 172.21.15.7:0/2429579553 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb449769f0 con 0x55eb44a81d20 2024-03-19T00:41:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7deaffd640 10 client.16260 handle_command_reply: tid=0 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 client.16260 shutdown 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 msgr2=0x55eb44a13de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a13de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc036040 tx=0x7f7dfc009260 comp rx=0 tx=0).stop 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 shutdown_connections 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55eb44a81d20 0x55eb44a84140 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb44a0ae50 0x55eb44a152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb44a05ce0 0x55eb44a14320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7deaffd640 0 client.16260 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 --2- 172.21.15.7:0/2429579553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb44a052f0 0x55eb44a13de0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 >> 172.21.15.7:0/2429579553 conn(0x55eb44a00860 msgr2=0x55eb44a03ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 shutdown_connections 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.761+0000 7f7e14838740 1 -- 172.21.15.7:0/2429579553 wait complete. 2024-03-19T00:41:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e14838740 20 client.16260 trim_cache size 0 max 16384 2024-03-19T00:41:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df80a2800 msgr2=0x7f7df80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df80a2800 0x7f7df80a4cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005ab0 tx=0x7f7dfc005fd0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 msgr2=0x7f7e0c195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 0x7f7e0c195d80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009be0 tx=0x7f7df40071c0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 shutdown_connections 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7df80a2800 0x7f7df80a4cc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e0c1022d0 0x7f7e0c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0c101910 0x7f7e0c195d80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 --2- 172.21.15.7:0/1663779045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c100720 0x7f7e0c195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 >> 172.21.15.7:0/1663779045 conn(0x7f7e0c0fbc50 msgr2=0x7f7e0c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 shutdown_connections 2024-03-19T00:41:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.762+0000 7f7e11c4b640 1 -- 172.21.15.7:0/1663779045 wait complete. 2024-03-19T00:41:43.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:41:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 -- 172.21.15.7:0/2349239335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f4100720 msgr2=0x7f01f4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 --2- 172.21.15.7:0/2349239335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f4100720 0x7f01f4100b40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009900 tx=0x7f01ec02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 -- 172.21.15.7:0/2349239335 shutdown_connections 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 --2- 172.21.15.7:0/2349239335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f41022d0 0x7f01f4108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 --2- 172.21.15.7:0/2349239335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4101910 0x7f01f4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 --2- 172.21.15.7:0/2349239335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f4100720 0x7f01f4100b40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.879+0000 7f01fc663640 1 -- 172.21.15.7:0/2349239335 >> 172.21.15.7:0/2349239335 conn(0x7f01f40fbc50 msgr2=0x7f01f40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 -- 172.21.15.7:0/2349239335 shutdown_connections 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 -- 172.21.15.7:0/2349239335 wait complete. 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 Processor -- start 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 -- start start 2024-03-19T00:41:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4100720 0x7f01f4069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f4101910 0x7f01f419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41022d0 0x7f01f419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01f410c250 con 0x7f01f41022d0 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01f410c0d0 con 0x7f01f4100720 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fc663640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01f410c3d0 con 0x7f01f4101910 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41022d0 0x7f01f419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f4101910 0x7f01f419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41022d0 0x7f01f419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55134/0 (socket says 172.21.15.7:55134) 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fac00640 1 -- 172.21.15.7:0/2891980468 learned_addr learned my addr 172.21.15.7:0/2891980468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.880+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4100720 0x7f01f4069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f9bfe640 1 -- 172.21.15.7:0/2891980468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4100720 msgr2=0x7f01f4069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4100720 0x7f01f4069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f9bfe640 1 -- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41022d0 msgr2=0x7f01f419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41022d0 0x7f01f419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f9bfe640 1 -- 172.21.15.7:0/2891980468 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f419f680 con 0x7f01f4101910 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4100720 0x7f01f4069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f4101910 0x7f01f419c780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01e800e9f0 tx=0x7f01e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f37fe640 1 -- 172.21.15.7:0/2891980468 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800ccc0 con 0x7f01f4101910 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f419f970 con 0x7f01f4101910 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f37fe640 1 -- 172.21.15.7:0/2891980468 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e80108e0 con 0x7f01f4101910 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01f37fe640 1 -- 172.21.15.7:0/2891980468 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8018880 con 0x7f01f4101910 2024-03-19T00:41:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.881+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f419ff10 con 0x7f01f4101910 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.883+0000 7f01f37fe640 1 -- 172.21.15.7:0/2891980468 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f01e8010a90 con 0x7f01f4101910 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 Processor -- start 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 -- start start 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e7b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 0x5579d8e807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5579d8ddf4b0 con 0x5579d8e80e50 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5579d8d87d70 con 0x5579d8e7b2f0 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01ff247740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5579d8ddb550 con 0x5579d8e7bce0 2024-03-19T00:41:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.884+0000 7f01f37fe640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01dc0a2800 0x7f01dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e7b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 0x5579d8e807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e7b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47246/0 (socket says 172.21.15.7:47246) 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fac00640 1 -- 172.21.15.7:0/2225331037 learned_addr learned my addr 172.21.15.7:0/2225331037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 0x5579d8e807e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:36376/0 (socket says 172.21.15.7:36376) 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fac00640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01dc0a2800 0x7f01dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01f37fe640 1 -- 172.21.15.7:0/2891980468 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f01e80cf160 con 0x7f01f4101910 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 -- 172.21.15.7:0/2225331037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 msgr2=0x5579d8e7b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e7b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 -- 172.21.15.7:0/2225331037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 msgr2=0x5579d8e832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 -- 172.21.15.7:0/2225331037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d8daa210 con 0x5579d8e7bce0 2024-03-19T00:41:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fac00640 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e7b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 0x5579d8e807e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01ec036040 tx=0x7f01ec009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf921d2fc568534 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01fac00640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01dc0a2800 0x7f01dc0a4cc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5579d8e7c540 tx=0x7f01e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01f0bfe640 1 -- 172.21.15.7:0/2225331037 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec045020 con 0x5579d8e7bce0 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01f0bfe640 1 -- 172.21.15.7:0/2225331037 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01ec02cc90 con 0x5579d8e7bce0 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.885+0000 7f01f0bfe640 1 -- 172.21.15.7:0/2225331037 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0048c0 con 0x5579d8e7bce0 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 -- 172.21.15.7:0/2225331037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 msgr2=0x5579d8e807e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 0x5579d8e807e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01ec036040 tx=0x7f01ec009580 comp rx=0 tx=0).stop 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 -- 172.21.15.7:0/2225331037 shutdown_connections 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7bce0 0x5579d8e807e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 --2- 172.21.15.7:0/2225331037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e7b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 -- 172.21.15.7:0/2225331037 >> 172.21.15.7:0/2225331037 conn(0x5579d8e76860 msgr2=0x5579d8e76c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 -- 172.21.15.7:0/2225331037 shutdown_connections 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 -- 172.21.15.7:0/2225331037 wait complete. 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.886+0000 7f01ff247740 1 -- start start 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 Processor -- start 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7b2f0 0x5579d8e85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7bce0 0x5579d8e85cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5579d8e83da0 con 0x5579d8e80e50 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5579d8d9b310 con 0x5579d8e7bce0 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5579d8e83f20 con 0x5579d8e7b2f0 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01ff247740 10 client.0 _ll_register_callbacks cb 0x5579d8d8f2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fa3ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7b2f0 0x5579d8e85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7bce0 0x5579d8e85cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e8cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55156/0 (socket says 172.21.15.7:55156) 2024-03-19T00:41:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 -- 172.21.15.7:0/4157907253 learned_addr learned my addr 172.21.15.7:0/4157907253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7b2f0 0x5579d8e85780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:36390/0 (socket says 172.21.15.7:36390) 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 -- 172.21.15.7:0/4157907253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7b2f0 msgr2=0x5579d8e85780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7b2f0 0x5579d8e85780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 -- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7bce0 msgr2=0x5579d8e85cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7bce0 0x5579d8e85cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 -- 172.21.15.7:0/4157907253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d8af33f0 con 0x5579d8e80e50 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7bce0 0x5579d8e85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01fac00640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e8cd00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009aa0 tx=0x7f01e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b38f01de16ba0f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.887+0000 7f01d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01d9ffb640 1 -- 172.21.15.7:0/4157907253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4004b20 con 0x5579d8e80e50 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01d9ffb640 1 -- 172.21.15.7:0/4157907253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f01e401c070 con 0x5579d8e80e50 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec02cc90 con 0x5579d8e80e50 2024-03-19T00:41:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01d9ffb640 1 -- 172.21.15.7:0/4157907253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f01e4037050 con 0x5579d8e80e50 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579d8bbd7d0 con 0x5579d8e80e50 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01d9ffb640 1 -- 172.21.15.7:0/4157907253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01ec02cc90 con 0x5579d8e80e50 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01d9ffb640 1 -- 172.21.15.7:0/4157907253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f01e4005760 con 0x5579d8e80e50 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 10 client.16272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 20 client.16272 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8e93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 4 client.16272 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.888+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579d8e3d3e0 con 0x5579d8e916f0 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.889+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8e93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.889+0000 7f01d9ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.889+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8e93b50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=439b715f95f3f00d server_cookie=bea15c492ce11a7a in_seq=0 out_seq=0 2024-03-19T00:41:43.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01d9ffb640 1 -- 172.21.15.7:0/4157907253 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5579d8e3d3e0 con 0x5579d8e916f0 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01d9ffb640 10 client.16272 handle_command_reply: tid=0 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 client.16272 shutdown 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 msgr2=0x5579d8e8cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e8cd00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009aa0 tx=0x7f01e4004a40 comp rx=0 tx=0).stop 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 shutdown_connections 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8e93b50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e80e50 0x5579d8e8cd00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01d9ffb640 0 client.16272 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7bce0 0x5579d8e85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 --2- 172.21.15.7:0/4157907253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e7b2f0 0x5579d8e85780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 >> 172.21.15.7:0/4157907253 conn(0x5579d8e76860 msgr2=0x5579d8e75ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 shutdown_connections 2024-03-19T00:41:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 1 -- 172.21.15.7:0/4157907253 wait complete. 2024-03-19T00:41:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.890+0000 7f01ff247740 20 client.16272 trim_cache size 0 max 16384 2024-03-19T00:41:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 Processor -- start 2024-03-19T00:41:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 -- start start 2024-03-19T00:41:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e75310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41aa890 con 0x5579d8e7bce0 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41a8160 con 0x5579d8e7b2f0 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.903+0000 7f01ff247740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41a5fb0 con 0x5579d8e80e50 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55172/0 (socket says 172.21.15.7:55172) 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1106572474 learned_addr learned my addr 172.21.15.7:0/1106572474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e75310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01fac00640 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1106572474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 msgr2=0x5579d8e75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1106572474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 msgr2=0x5579d8e75310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e75310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1106572474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d8e4c880 con 0x5579d8e7bce0 2024-03-19T00:41:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8ccc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f01e80cf400 tx=0x7f01e806e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2012b138d653882 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01dbfff640 1 -- 172.21.15.7:0/1106572474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e809f890 con 0x5579d8e7bce0 2024-03-19T00:41:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01dbfff640 1 -- 172.21.15.7:0/1106572474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e80a3ee0 con 0x5579d8e7bce0 2024-03-19T00:41:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.904+0000 7f01dbfff640 1 -- 172.21.15.7:0/1106572474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80d1340 con 0x5579d8e7bce0 2024-03-19T00:41:43.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 -- 172.21.15.7:0/1106572474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 msgr2=0x5579d8e8ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8ccc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f01e80cf400 tx=0x7f01e806e520 comp rx=0 tx=0).stop 2024-03-19T00:41:43.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 -- 172.21.15.7:0/1106572474 shutdown_connections 2024-03-19T00:41:43.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8ccc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 --2- 172.21.15.7:0/1106572474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e75310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 -- 172.21.15.7:0/1106572474 >> 172.21.15.7:0/1106572474 conn(0x5579d8e76860 msgr2=0x5579d8e85780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 -- 172.21.15.7:0/1106572474 shutdown_connections 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 -- 172.21.15.7:0/1106572474 wait complete. 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 -- start start 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.905+0000 7f01ff247740 1 Processor -- start 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e8a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e8ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41a6620 con 0x5579d8e7bce0 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41a7a90 con 0x5579d8e7b2f0 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41a6cf0 con 0x5579d8e80e50 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 10 client.0 _ll_register_callbacks cb 0x7f01e4013710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:43.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55176/0 (socket says 172.21.15.7:55176) 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1894935846 learned_addr learned my addr 172.21.15.7:0/1894935846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e8a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01fac00640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e8ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1894935846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 msgr2=0x5579d8e8ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e8ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 msgr2=0x5579d8e8a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e8a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 -- 172.21.15.7:0/1894935846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d8e8c060 con 0x5579d8e7bce0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01fac00640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01f9bfe640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8aa60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f01e801a010 tx=0x7f01e8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83302ca392c6e3df server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01d9ffb640 1 -- 172.21.15.7:0/1894935846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80b4060 con 0x5579d8e7bce0 2024-03-19T00:41:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01d9ffb640 1 -- 172.21.15.7:0/1894935846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f01e80d1310 con 0x5579d8e7bce0 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e80a3ee0 con 0x5579d8e7bce0 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01d9ffb640 1 -- 172.21.15.7:0/1894935846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f01e8099bb0 con 0x5579d8e7bce0 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 10 client.16278 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.906+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579d8ef7bb0 con 0x5579d8e7bce0 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01d9ffb640 1 -- 172.21.15.7:0/1894935846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e80a3ee0 con 0x5579d8e7bce0 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01d9ffb640 1 -- 172.21.15.7:0/1894935846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f01e806e040 con 0x5579d8e7bce0 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01ff247740 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01ff247740 10 client.16278 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01ff247740 20 client.16278 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01ff247740 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8efa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01ff247740 4 client.16278 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579d8dec9f0 con 0x5579d8e916f0 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8efa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01fa3ff640 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8efa6c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89f8f53dc777b558 server_cookie=9c3c61058f1c898a in_seq=0 out_seq=0 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.907+0000 7f01d9ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01d9ffb640 1 -- 172.21.15.7:0/1894935846 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579d8dec9f0 con 0x5579d8e916f0 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01d9ffb640 10 client.16278 handle_command_reply: tid=0 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 client.16278 shutdown 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 msgr2=0x5579d8e8aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8aa60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f01e801a010 tx=0x7f01e8005c50 comp rx=0 tx=0).stop 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 shutdown_connections 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5579d8e916f0 0x5579d8efa6c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579d8e80e50 0x5579d8e8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01d9ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5579d8e7bce0 0x5579d8e8aa60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 --2- 172.21.15.7:0/1894935846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5579d8e7b2f0 0x5579d8e8a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 >> 172.21.15.7:0/1894935846 conn(0x5579d8e76860 msgr2=0x5579d8e79ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 shutdown_connections 2024-03-19T00:41:43.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 1 -- 172.21.15.7:0/1894935846 wait complete. 2024-03-19T00:41:43.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01ff247740 20 client.16278 trim_cache size 0 max 16384 2024-03-19T00:41:43.916 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01dc0a2800 msgr2=0x7f01dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01dc0a2800 0x7f01dc0a4cc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5579d8e7c540 tx=0x7f01e4009500 comp rx=0 tx=0).stop 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f4101910 msgr2=0x7f01f419c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f4101910 0x7f01f419c780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01e800e9f0 tx=0x7f01e800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 shutdown_connections 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f01dc0a2800 0x7f01dc0a4cc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41022d0 0x7f01f419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01f4101910 0x7f01f419c780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.908+0000 7f01fc663640 1 --2- 172.21.15.7:0/2891980468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01f4100720 0x7f01f4069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.909+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 >> 172.21.15.7:0/2891980468 conn(0x7f01f40fbc50 msgr2=0x7f01f4104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.909+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 shutdown_connections 2024-03-19T00:41:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:43.909+0000 7f01fc663640 1 -- 172.21.15.7:0/2891980468 wait complete. 2024-03-19T00:41:43.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:43 smithi007 ceph-mon[62854]: pgmap v464: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:43.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:43 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/863635443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:41:43.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:41:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:43 smithi204 ceph-mon[66572]: pgmap v464: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:43 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/863635443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:41:44.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.025+0000 7f50dea61640 1 -- 172.21.15.7:0/997459341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d81022d0 msgr2=0x7f50d8108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.025+0000 7f50dea61640 1 --2- 172.21.15.7:0/997459341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d81022d0 0x7f50d8108b60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009ca0 tx=0x7f50d002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 -- 172.21.15.7:0/997459341 shutdown_connections 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 --2- 172.21.15.7:0/997459341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d81022d0 0x7f50d8108b60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 --2- 172.21.15.7:0/997459341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d8101910 0x7f50d8101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 --2- 172.21.15.7:0/997459341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50d8100720 0x7f50d8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 -- 172.21.15.7:0/997459341 >> 172.21.15.7:0/997459341 conn(0x7f50d80fbc50 msgr2=0x7f50d80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 -- 172.21.15.7:0/997459341 shutdown_connections 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 -- 172.21.15.7:0/997459341 wait complete. 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 Processor -- start 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 -- start start 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.026+0000 7f50dea61640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50d8100720 0x7f50d81046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 0x7f50d8102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d81022d0 0x7f50d8103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d810c250 con 0x7f50d8101910 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50d810c0d0 con 0x7f50d81022d0 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50d810c3d0 con 0x7f50d8100720 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 0x7f50d8102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 0x7f50d8102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55192/0 (socket says 172.21.15.7:55192) 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 -- 172.21.15.7:0/331305204 learned_addr learned my addr 172.21.15.7:0/331305204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d7fff640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50d8100720 0x7f50d81046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dcffe640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d81022d0 0x7f50d8103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 -- 172.21.15.7:0/331305204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50d8100720 msgr2=0x7f50d81046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50d8100720 0x7f50d81046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 -- 172.21.15.7:0/331305204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d81022d0 msgr2=0x7f50d8103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d81022d0 0x7f50d8103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 -- 172.21.15.7:0/331305204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d8103ac0 con 0x7f50d8101910 2024-03-19T00:41:44.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d77fe640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 0x7f50d8102cf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9f0 tx=0x7f50c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:44.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d57fa640 1 -- 172.21.15.7:0/331305204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c800cd80 con 0x7f50d8101910 2024-03-19T00:41:44.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d57fa640 1 -- 172.21.15.7:0/331305204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50c80108e0 con 0x7f50d8101910 2024-03-19T00:41:44.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d81a1b10 con 0x7f50d8101910 2024-03-19T00:41:44.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50d57fa640 1 -- 172.21.15.7:0/331305204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8018920 con 0x7f50d8101910 2024-03-19T00:41:44.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.027+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d81a2060 con 0x7f50d8101910 2024-03-19T00:41:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50d57fa640 1 -- 172.21.15.7:0/331305204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f50c8010a90 con 0x7f50d8101910 2024-03-19T00:41:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 Processor -- start 2024-03-19T00:41:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 -- start start 2024-03-19T00:41:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b29710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b2e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a598a8d4b0 con 0x55a598b29ce0 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a598a35d70 con 0x55a598b292f0 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50e1651740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a598a89550 con 0x55a598b2ee50 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b2e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50dcffe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b29710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b2e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55204/0 (socket says 172.21.15.7:55204) 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50d7fff640 1 -- 172.21.15.7:0/2037275487 learned_addr learned my addr 172.21.15.7:0/2037275487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50d57fa640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50c00a2800 0x7f50c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.030+0000 7f50d77fe640 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50dcffe640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50c00a2800 0x7f50c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d7fff640 1 -- 172.21.15.7:0/2037275487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 msgr2=0x55a598b312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d7fff640 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d7fff640 1 -- 172.21.15.7:0/2037275487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 msgr2=0x55a598b29710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d7fff640 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b29710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d7fff640 1 -- 172.21.15.7:0/2037275487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a598a58210 con 0x55a598b29ce0 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50dcffe640 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b29710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d57fa640 1 -- 172.21.15.7:0/331305204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50c80cf0b0 con 0x7f50d8101910 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d77fe640 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50d7fff640 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b2e7e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009ca0 tx=0x7f50cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6088673c3443369b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50dcffe640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50c00a2800 0x7f50c00a4cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50d0006010 tx=0x7f50d002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50be7fc640 1 -- 172.21.15.7:0/2037275487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc004220 con 0x55a598b29ce0 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50be7fc640 1 -- 172.21.15.7:0/2037275487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50cc0043c0 con 0x55a598b29ce0 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.031+0000 7f50be7fc640 1 -- 172.21.15.7:0/2037275487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc019890 con 0x55a598b29ce0 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 -- 172.21.15.7:0/2037275487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 msgr2=0x55a598b2e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b2e7e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009ca0 tx=0x7f50cc011810 comp rx=0 tx=0).stop 2024-03-19T00:41:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 -- 172.21.15.7:0/2037275487 shutdown_connections 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b2e7e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 --2- 172.21.15.7:0/2037275487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b29710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 -- 172.21.15.7:0/2037275487 >> 172.21.15.7:0/2037275487 conn(0x55a598b24860 msgr2=0x55a598b24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 -- 172.21.15.7:0/2037275487 shutdown_connections 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 -- 172.21.15.7:0/2037275487 wait complete. 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 -- start start 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.032+0000 7f50e1651740 1 Processor -- start 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a598b31e00 con 0x55a598b29ce0 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a598a49310 con 0x55a598b292f0 2024-03-19T00:41:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a598b31f80 con 0x55a598b2ee50 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50e1651740 10 client.0 _ll_register_callbacks cb 0x55a598a3d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50dcffe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47340/0 (socket says 172.21.15.7:47340) 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d7fff640 1 -- 172.21.15.7:0/2622176954 learned_addr learned my addr 172.21.15.7:0/2622176954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 -- 172.21.15.7:0/2622176954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 msgr2=0x55a598b3ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b3ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 -- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 msgr2=0x55a598b337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 -- 172.21.15.7:0/2622176954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5987a13f0 con 0x55a598b29ce0 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50dcffe640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d7fff640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50d77fe640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f50c80cc8a0 tx=0x7f50c809b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a036b4244e16ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.033+0000 7f50b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50b77fe640 1 -- 172.21.15.7:0/2622176954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c809f840 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50cc0043c0 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50b77fe640 1 -- 172.21.15.7:0/2622176954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50c80860c0 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50b77fe640 1 -- 172.21.15.7:0/2622176954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50c80a3e90 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 10 client.16290 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a59886b7d0 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50b77fe640 1 -- 172.21.15.7:0/2622176954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50cc0043c0 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50b77fe640 1 -- 172.21.15.7:0/2622176954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f50c809ad80 con 0x55a598b29ce0 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 10 client.16290 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 20 client.16290 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598b3f630 0x55a598b41ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 4 client.16290 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a598aeb3e0 con 0x55a598b3f630 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.034+0000 7f50d7fff640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598b3f630 0x55a598b41ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.035+0000 7f50b77fe640 10 client.16290 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.035+0000 7f50d7fff640 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598b3f630 0x55a598b41ad0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6915e3ca93e62c12 server_cookie=b780a5e66017b18c in_seq=0 out_seq=0 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.035+0000 7f50b77fe640 1 -- 172.21.15.7:0/2622176954 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a598aeb3e0 con 0x55a598b3f630 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.035+0000 7f50b77fe640 10 client.16290 handle_command_reply: tid=0 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.035+0000 7f50e1651740 1 client.16290 shutdown 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 msgr2=0x55a598b33ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f50c80cc8a0 tx=0x7f50c809b3e0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 shutdown_connections 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598b3f630 0x55a598b41ad0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 --2- 172.21.15.7:0/2622176954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50b77fe640 0 client.16290 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 >> 172.21.15.7:0/2622176954 conn(0x55a598b24860 msgr2=0x55a598b23ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 shutdown_connections 2024-03-19T00:41:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 1 -- 172.21.15.7:0/2622176954 wait complete. 2024-03-19T00:41:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.036+0000 7f50e1651740 20 client.16290 trim_cache size 0 max 16384 2024-03-19T00:41:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 Processor -- start 2024-03-19T00:41:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 -- start start 2024-03-19T00:41:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b2ee50 0x55a598b3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81aed30 con 0x55a598b292f0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81ac600 con 0x55a598b2ee50 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50e1651740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81aa450 con 0x55a598b29ce0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55230/0 (socket says 172.21.15.7:55230) 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50d7fff640 1 -- 172.21.15.7:0/1270952345 learned_addr learned my addr 172.21.15.7:0/1270952345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50d77fe640 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.049+0000 7f50dcffe640 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b2ee50 0x55a598b3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50d7fff640 1 -- 172.21.15.7:0/1270952345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b29ce0 msgr2=0x55a598b33ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50d7fff640 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50d7fff640 1 -- 172.21.15.7:0/1270952345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b2ee50 msgr2=0x55a598b3ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50d7fff640 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b2ee50 0x55a598b3ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50d7fff640 1 -- 172.21.15.7:0/1270952345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a598b3f3f0 con 0x55a598b292f0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50d7fff640 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 0x55a598b337a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f50cc013010 tx=0x7f50cc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e6f4a04df1310a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50bdffb640 1 -- 172.21.15.7:0/1270952345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc02ca30 con 0x55a598b292f0 2024-03-19T00:41:44.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50bdffb640 1 -- 172.21.15.7:0/1270952345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50cc02c780 con 0x55a598b292f0 2024-03-19T00:41:44.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.050+0000 7f50bdffb640 1 -- 172.21.15.7:0/1270952345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc005220 con 0x55a598b292f0 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 -- 172.21.15.7:0/1270952345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 msgr2=0x55a598b337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 0x55a598b337a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f50cc013010 tx=0x7f50cc009680 comp rx=0 tx=0).stop 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 -- 172.21.15.7:0/1270952345 shutdown_connections 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b2ee50 0x55a598b3ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b29ce0 0x55a598b33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 --2- 172.21.15.7:0/1270952345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b292f0 0x55a598b337a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 -- 172.21.15.7:0/1270952345 >> 172.21.15.7:0/1270952345 conn(0x55a598b24860 msgr2=0x55a598b24c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 -- 172.21.15.7:0/1270952345 shutdown_connections 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 -- 172.21.15.7:0/1270952345 wait complete. 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.051+0000 7f50e1651740 1 -- start start 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 Processor -- start 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81aaac0 con 0x55a598b29ce0 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81abf30 con 0x55a598b292f0 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81ab190 con 0x55a598b2ee50 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50e1651740 10 client.0 _ll_register_callbacks cb 0x7f50c8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b38260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55246/0 (socket says 172.21.15.7:55246) 2024-03-19T00:41:44.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 -- 172.21.15.7:0/419620383 learned_addr learned my addr 172.21.15.7:0/419620383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50dcffe640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 -- 172.21.15.7:0/419620383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 msgr2=0x55a598b39270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b39270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 -- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 msgr2=0x55a598b37d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b37d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d77fe640 1 -- 172.21.15.7:0/419620383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a598b39860 con 0x55a598b29ce0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.052+0000 7f50d7fff640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50d77fe640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b38260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9c0 tx=0x7f50c800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c17635816bdc9b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50b77fe640 1 -- 172.21.15.7:0/419620383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c80b3030 con 0x55a598b29ce0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50b77fe640 1 -- 172.21.15.7:0/419620383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50c808c030 con 0x55a598b29ce0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50cc02c780 con 0x55a598b29ce0 2024-03-19T00:41:44.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50b77fe640 1 -- 172.21.15.7:0/419620383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50c809b570 con 0x55a598b29ce0 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 10 client.16296 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a598b3abd0 con 0x55a598b29ce0 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50b77fe640 1 -- 172.21.15.7:0/419620383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50cc02c780 con 0x55a598b29ce0 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50b77fe640 1 -- 172.21.15.7:0/419620383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f50c80b4400 con 0x55a598b29ce0 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 10 client.16296 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 20 client.16296 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598ba5c70 0x55a598ba8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 4 client.16296 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.053+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a598a9a9f0 con 0x55a598ba5c70 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50d7fff640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598ba5c70 0x55a598ba8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50d7fff640 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598ba5c70 0x55a598ba8060 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e6461c1e72ef09 server_cookie=50cf7fee0a731e49 in_seq=0 out_seq=0 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50b77fe640 10 client.16296 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50b77fe640 1 -- 172.21.15.7:0/419620383 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a598a9a9f0 con 0x55a598ba5c70 2024-03-19T00:41:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50b77fe640 10 client.16296 handle_command_reply: tid=0 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 client.16296 shutdown 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 msgr2=0x55a598b38260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b38260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9c0 tx=0x7f50c800c7d0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 shutdown_connections 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a598b2ee50 0x55a598b39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a598ba5c70 0x55a598ba8060 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a598b29ce0 0x55a598b38260 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 --2- 172.21.15.7:0/419620383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a598b292f0 0x55a598b37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50b77fe640 0 client.16296 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 >> 172.21.15.7:0/419620383 conn(0x55a598b24860 msgr2=0x55a598b33760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 shutdown_connections 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.054+0000 7f50e1651740 1 -- 172.21.15.7:0/419620383 wait complete. 2024-03-19T00:41:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50e1651740 20 client.16296 trim_cache size 0 max 16384 2024-03-19T00:41:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:41:44.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50c00a2800 msgr2=0x7f50c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50c00a2800 0x7f50c00a4cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50d0006010 tx=0x7f50d002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 msgr2=0x7f50d8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 0x7f50d8102cf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9f0 tx=0x7f50c800eeb0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 shutdown_connections 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50c00a2800 0x7f50c00a4cc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d81022d0 0x7f50d8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d8101910 0x7f50d8102cf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 --2- 172.21.15.7:0/331305204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50d8100720 0x7f50d81046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 >> 172.21.15.7:0/331305204 conn(0x7f50d80fbc50 msgr2=0x7f50d80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 shutdown_connections 2024-03-19T00:41:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:44.055+0000 7f50dea61640 1 -- 172.21.15.7:0/331305204 wait complete. 2024-03-19T00:41:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:43 smithi044 ceph-mon[66366]: pgmap v464: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:43 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/863635443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:41:44.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:41:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:41:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:41:44.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:41:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:41:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:41:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:41:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:41:45.070 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:41:45.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:41:45.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:41:45.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:41:45.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:41:45.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:41:45.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 -- 172.21.15.7:0/2655034128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf18101930 msgr2=0x7faf18101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 --2- 172.21.15.7:0/2655034128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf18101930 0x7faf18101db0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faf08009900 tx=0x7faf0802bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 -- 172.21.15.7:0/2655034128 shutdown_connections 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 --2- 172.21.15.7:0/2655034128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf181022f0 0x7faf18108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 --2- 172.21.15.7:0/2655034128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf18101930 0x7faf18101db0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 --2- 172.21.15.7:0/2655034128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf18100740 0x7faf18100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 -- 172.21.15.7:0/2655034128 >> 172.21.15.7:0/2655034128 conn(0x7faf180fbcb0 msgr2=0x7faf180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 -- 172.21.15.7:0/2655034128 shutdown_connections 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.189+0000 7faf1fa81640 1 -- 172.21.15.7:0/2655034128 wait complete. 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 Processor -- start 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 -- start start 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf18100740 0x7faf18197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf18101930 0x7faf18198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 0x7faf1819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf1810d390 con 0x7faf181022f0 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf1810d210 con 0x7faf18100740 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1fa81640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf1810d510 con 0x7faf18101930 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 0x7faf1819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 0x7faf1819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48212/0 (socket says 172.21.15.7:48212) 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 -- 172.21.15.7:0/986625134 learned_addr learned my addr 172.21.15.7:0/986625134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1d81d640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf18100740 0x7faf18197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1d01c640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf18101930 0x7faf18198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 -- 172.21.15.7:0/986625134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf18101930 msgr2=0x7faf18198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf18101930 0x7faf18198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 -- 172.21.15.7:0/986625134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf18100740 msgr2=0x7faf18197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf18100740 0x7faf18197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.190+0000 7faf1e01e640 1 -- 172.21.15.7:0/986625134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1819f6d0 con 0x7faf181022f0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.191+0000 7faf1e01e640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 0x7faf1819f060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faf1000eb10 tx=0x7faf1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.191+0000 7faf0effd640 1 -- 172.21.15.7:0/986625134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10004b20 con 0x7faf181022f0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.191+0000 7faf0effd640 1 -- 172.21.15.7:0/986625134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf10004cc0 con 0x7faf181022f0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.191+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1819f9c0 con 0x7faf181022f0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.191+0000 7faf0effd640 1 -- 172.21.15.7:0/986625134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10010690 con 0x7faf181022f0 2024-03-19T00:41:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.191+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1819fe90 con 0x7faf181022f0 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.193+0000 7faf0effd640 1 -- 172.21.15.7:0/986625134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7faf10005020 con 0x7faf181022f0 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 Processor -- start 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 -- start start 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 0x55759806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55759806df00 0x557598070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557597fcd160 con 0x55759806df00 2024-03-19T00:41:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557597f75e10 con 0x5575980683a0 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf2266f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557597fc9200 con 0x557598068d90 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf0effd640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faf040a2800 0x7faf040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf1d81d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 0x55759806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf1d01c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf1d81d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 0x55759806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:59598/0 (socket says 172.21.15.7:59598) 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf1d81d640 1 -- 172.21.15.7:0/4148530451 learned_addr learned my addr 172.21.15.7:0/4148530451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf1e01e640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faf040a2800 0x7faf040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf1d01c640 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55759806df00 0x557598070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.194+0000 7faf0effd640 1 -- 172.21.15.7:0/986625134 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faf100cf390 con 0x7faf181022f0 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1d81d640 1 -- 172.21.15.7:0/4148530451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 msgr2=0x5575980687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1d81d640 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1d81d640 1 -- 172.21.15.7:0/4148530451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55759806df00 msgr2=0x557598070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1d81d640 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55759806df00 0x557598070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1d81d640 1 -- 172.21.15.7:0/4148530451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557597da97e0 con 0x557598068d90 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1e01e640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faf040a2800 0x7faf040a4cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf10098340 tx=0x7faf1000efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf1d81d640 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 0x55759806d890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf14009ca0 tx=0x7faf14011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfdb4305c8b0d76f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf03fff640 1 -- 172.21.15.7:0/4148530451 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14009e50 con 0x557598068d90 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf03fff640 1 -- 172.21.15.7:0/4148530451 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf140035f0 con 0x557598068d90 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.195+0000 7faf03fff640 1 -- 172.21.15.7:0/4148530451 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14019650 con 0x557598068d90 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 -- 172.21.15.7:0/4148530451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 msgr2=0x55759806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 0x55759806d890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf14009ca0 tx=0x7faf14011810 comp rx=0 tx=0).stop 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 -- 172.21.15.7:0/4148530451 shutdown_connections 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55759806df00 0x557598070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557598068d90 0x55759806d890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 --2- 172.21.15.7:0/4148530451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 -- 172.21.15.7:0/4148530451 >> 172.21.15.7:0/4148530451 conn(0x557598063910 msgr2=0x557598063d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 -- 172.21.15.7:0/4148530451 shutdown_connections 2024-03-19T00:41:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 -- 172.21.15.7:0/4148530451 wait complete. 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 -- start start 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.196+0000 7faf2266f740 1 Processor -- start 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x5575980727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x557598079d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557598070eb0 con 0x5575980683a0 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557597f891a0 con 0x557598068d90 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557598071030 con 0x55759806df00 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf2266f740 10 client.0 _ll_register_callbacks cb 0x557597ce0c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x5575980727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x5575980727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48234/0 (socket says 172.21.15.7:48234) 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 -- 172.21.15.7:0/217387059 learned_addr learned my addr 172.21.15.7:0/217387059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d81d640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x557598079d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 -- 172.21.15.7:0/217387059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 msgr2=0x557598079d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x557598079d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 -- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 msgr2=0x557598072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 -- 172.21.15.7:0/217387059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557597e09f70 con 0x5575980683a0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d81d640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf1d01c640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x5575980727c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faf0802c170 tx=0x7faf08005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d449e3732f181 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf017fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.197+0000 7faf017fa640 1 -- 172.21.15.7:0/217387059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08005850 con 0x5575980683a0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf017fa640 1 -- 172.21.15.7:0/217387059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faf080046c0 con 0x5575980683a0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf140035f0 con 0x5575980683a0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf017fa640 1 -- 172.21.15.7:0/217387059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf08057050 con 0x5575980683a0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 10 client.16308 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557597ca1170 con 0x5575980683a0 2024-03-19T00:41:45.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf017fa640 1 -- 172.21.15.7:0/217387059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf140035f0 con 0x5575980683a0 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf017fa640 1 -- 172.21.15.7:0/217387059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faf08003880 con 0x5575980683a0 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 10 client.16308 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 20 client.16308 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55759807e590 0x557598080a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 4 client.16308 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557597fda6a0 con 0x55759807e590 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.198+0000 7faf1d81d640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55759807e590 0x557598080a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.199+0000 7faf1d81d640 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55759807e590 0x557598080a30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8da2770f140f40e server_cookie=5b78482a7db40035 in_seq=0 out_seq=0 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.199+0000 7faf017fa640 10 client.16308 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.199+0000 7faf017fa640 1 -- 172.21.15.7:0/217387059 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557597fda6a0 con 0x55759807e590 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.199+0000 7faf017fa640 10 client.16308 handle_command_reply: tid=0 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.199+0000 7faf2266f740 1 client.16308 shutdown 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 msgr2=0x5575980727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x5575980727c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faf0802c170 tx=0x7faf08005d50 comp rx=0 tx=0).stop 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 shutdown_connections 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x557598079d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55759807e590 0x557598080a30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 --2- 172.21.15.7:0/217387059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x5575980727c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf017fa640 0 client.16308 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 >> 172.21.15.7:0/217387059 conn(0x557598063910 msgr2=0x557598062b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 shutdown_connections 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 1 -- 172.21.15.7:0/217387059 wait complete. 2024-03-19T00:41:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.200+0000 7faf2266f740 20 client.16308 trim_cache size 0 max 16384 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 Processor -- start 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 -- start start 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x557598072c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598067280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf181acab0 con 0x5575980683a0 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf181aa380 con 0x557598068d90 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf2266f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf181a81d0 con 0x55759806df00 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x557598072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x557598072c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48238/0 (socket says 172.21.15.7:48238) 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 -- 172.21.15.7:0/2877239400 learned_addr learned my addr 172.21.15.7:0/2877239400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d81d640 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 -- 172.21.15.7:0/2877239400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 msgr2=0x5575980852f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980852f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 -- 172.21.15.7:0/2877239400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 msgr2=0x557598067280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598067280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 -- 172.21.15.7:0/2877239400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55759806dbe0 con 0x5575980683a0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.218+0000 7faf1d01c640 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x557598072c60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7faf08009900 tx=0x7faf0802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18d7e3c1a82ead7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf037fe640 1 -- 172.21.15.7:0/2877239400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08003e70 con 0x5575980683a0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf037fe640 1 -- 172.21.15.7:0/2877239400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf080046c0 con 0x5575980683a0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf037fe640 1 -- 172.21.15.7:0/2877239400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080436b0 con 0x5575980683a0 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 -- 172.21.15.7:0/2877239400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 msgr2=0x557598072c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x557598072c60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7faf08009900 tx=0x7faf0802bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 -- 172.21.15.7:0/2877239400 shutdown_connections 2024-03-19T00:41:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980852f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557598068d90 0x557598067280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 --2- 172.21.15.7:0/2877239400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575980683a0 0x557598072c60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 -- 172.21.15.7:0/2877239400 >> 172.21.15.7:0/2877239400 conn(0x557598063910 msgr2=0x557598066b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 -- 172.21.15.7:0/2877239400 shutdown_connections 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 -- 172.21.15.7:0/2877239400 wait complete. 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.219+0000 7faf2266f740 1 -- start start 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 Processor -- start 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 0x5575980e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf181a8840 con 0x557598068d90 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf181a9cb0 con 0x5575980683a0 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf181a8f10 con 0x55759806df00 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf2266f740 10 client.0 _ll_register_callbacks cb 0x7faf08009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 0x5575980e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 0x5575980e0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48250/0 (socket says 172.21.15.7:48250) 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 -- 172.21.15.7:0/3393038844 learned_addr learned my addr 172.21.15.7:0/3393038844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d01c640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d01c640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 -- 172.21.15.7:0/3393038844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 msgr2=0x5575980e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 -- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 msgr2=0x5575980e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980e0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 -- 172.21.15.7:0/3393038844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575980e80a0 con 0x557598068d90 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf1d81d640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 0x5575980e0a70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faf14003c90 tx=0x7faf14009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33192da0a7a1099b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.220+0000 7faf017fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf017fa640 1 -- 172.21.15.7:0/3393038844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14032060 con 0x557598068d90 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf017fa640 1 -- 172.21.15.7:0/3393038844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faf14016040 con 0x557598068d90 2024-03-19T00:41:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf080046c0 con 0x557598068d90 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf017fa640 1 -- 172.21.15.7:0/3393038844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf14039050 con 0x557598068d90 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 10 client.16314 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575980e93e0 con 0x557598068d90 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf017fa640 1 -- 172.21.15.7:0/3393038844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf080046c0 con 0x557598068d90 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf017fa640 1 -- 172.21.15.7:0/3393038844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faf14026800 con 0x557598068d90 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 10 client.16314 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 20 client.16314 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5575980e9c20 0x5575980ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 4 client.16314 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.221+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557597fd49f0 con 0x5575980e9c20 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.222+0000 7faf1d01c640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5575980e9c20 0x5575980ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.222+0000 7faf1d01c640 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5575980e9c20 0x5575980ec040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447a54dbe4ca26c9 server_cookie=26600b553d90703b in_seq=0 out_seq=0 2024-03-19T00:41:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.222+0000 7faf017fa640 10 client.16314 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf017fa640 1 -- 172.21.15.7:0/3393038844 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557597fd49f0 con 0x5575980e9c20 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf017fa640 10 client.16314 handle_command_reply: tid=0 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 client.16314 shutdown 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 msgr2=0x5575980e0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 0x5575980e0a70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faf14003c90 tx=0x7faf14009ba0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 shutdown_connections 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55759806df00 0x5575980e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5575980e9c20 0x5575980ec040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557598068d90 0x5575980e0a70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 --2- 172.21.15.7:0/3393038844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575980683a0 0x5575980e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf017fa640 0 client.16314 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 >> 172.21.15.7:0/3393038844 conn(0x557598063910 msgr2=0x557598072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 shutdown_connections 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 1 -- 172.21.15.7:0/3393038844 wait complete. 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.223+0000 7faf2266f740 20 client.16314 trim_cache size 0 max 16384 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "047f948e-692b-44c3-9f60-52758b3bb3ef", 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:41:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:41:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faf040a2800 msgr2=0x7faf040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faf040a2800 0x7faf040a4cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf10098340 tx=0x7faf1000efd0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 msgr2=0x7faf1819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 0x7faf1819f060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faf1000eb10 tx=0x7faf1000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 shutdown_connections 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7faf040a2800 0x7faf040a4cc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf181022f0 0x7faf1819f060 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf18101930 0x7faf18198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 --2- 172.21.15.7:0/986625134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf18100740 0x7faf18197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 >> 172.21.15.7:0/986625134 conn(0x7faf180fbcb0 msgr2=0x7faf18104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 shutdown_connections 2024-03-19T00:41:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.224+0000 7faf1fa81640 1 -- 172.21.15.7:0/986625134 wait complete. 2024-03-19T00:41:45.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:41:45.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:41:45.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 -- 172.21.15.7:0/954009540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2244100740 msgr2=0x7f2244100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 --2- 172.21.15.7:0/954009540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2244100740 0x7f2244100b60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f223c009900 tx=0x7f223c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:41:45.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 -- 172.21.15.7:0/954009540 shutdown_connections 2024-03-19T00:41:45.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 --2- 172.21.15.7:0/954009540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22441022f0 0x7f2244108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 --2- 172.21.15.7:0/954009540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101930 0x7f2244101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 --2- 172.21.15.7:0/954009540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2244100740 0x7f2244100b60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 -- 172.21.15.7:0/954009540 >> 172.21.15.7:0/954009540 conn(0x7f22440fbcb0 msgr2=0x7f22440fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 -- 172.21.15.7:0/954009540 shutdown_connections 2024-03-19T00:41:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.339+0000 7f224bb5b640 1 -- 172.21.15.7:0/954009540 wait complete. 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 Processor -- start 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 -- start start 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100740 0x7f2244197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2244101930 0x7f2244198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 0x7f224419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f224410c260 con 0x7f22441022f0 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f224410c0e0 con 0x7f2244101930 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224bb5b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f224410c3e0 con 0x7f2244100740 2024-03-19T00:41:45.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 0x7f224419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 0x7f224419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48278/0 (socket says 172.21.15.7:48278) 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 -- 172.21.15.7:0/580595687 learned_addr learned my addr 172.21.15.7:0/580595687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f22498f7640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100740 0x7f2244197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f22490f6640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2244101930 0x7f2244198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 -- 172.21.15.7:0/580595687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100740 msgr2=0x7f2244197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100740 0x7f2244197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 -- 172.21.15.7:0/580595687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2244101930 msgr2=0x7f2244198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.340+0000 7f224a0f8640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2244101930 0x7f2244198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f224a0f8640 1 -- 172.21.15.7:0/580595687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224419f700 con 0x7f22441022f0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f224a0f8640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 0x7f224419f090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f223400eb10 tx=0x7f223400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f2242ffd640 1 -- 172.21.15.7:0/580595687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004b20 con 0x7f22441022f0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224419f9f0 con 0x7f22441022f0 2024-03-19T00:41:45.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f2242ffd640 1 -- 172.21.15.7:0/580595687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2234004cc0 con 0x7f22441022f0 2024-03-19T00:41:45.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f2242ffd640 1 -- 172.21.15.7:0/580595687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234010690 con 0x7f22441022f0 2024-03-19T00:41:45.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.341+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224419fef0 con 0x7f22441022f0 2024-03-19T00:41:45.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.343+0000 7f2242ffd640 1 -- 172.21.15.7:0/580595687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f2234005020 con 0x7f22441022f0 2024-03-19T00:41:45.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.343+0000 7f224e827740 1 Processor -- start 2024-03-19T00:41:45.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 -- start start 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305151e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x557305156f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573051575b0 0x557305159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5573050b6500 con 0x557305151a50 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55730505ebe0 con 0x5573051575b0 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224e827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5573050b25a0 con 0x557305152440 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f2242ffd640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f222c0a2800 0x7f222c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22490f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22498f7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x557305156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22490f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48286/0 (socket says 172.21.15.7:48286) 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22490f6640 1 -- 172.21.15.7:0/3726647054 learned_addr learned my addr 172.21.15.7:0/3726647054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f224a0f8640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f222c0a2800 0x7f222c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22490f6640 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573051575b0 0x557305159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f2242ffd640 1 -- 172.21.15.7:0/580595687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f22340cf330 con 0x7f22441022f0 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22498f7640 1 -- 172.21.15.7:0/3726647054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573051575b0 msgr2=0x557305159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22498f7640 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573051575b0 0x557305159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22498f7640 1 -- 172.21.15.7:0/3726647054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 msgr2=0x557305151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22498f7640 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22498f7640 1 -- 172.21.15.7:0/3726647054 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557305080000 con 0x557305152440 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.344+0000 7f22490f6640 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573051575b0 0x557305159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224a0f8640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f222c0a2800 0x7f222c0a4cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2234097dd0 tx=0x7f223406d5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f22498f7640 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x557305156f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f223c036040 tx=0x7f223c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d472b8e61dda76 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f222bfff640 1 -- 172.21.15.7:0/3726647054 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c045020 con 0x557305152440 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f222bfff640 1 -- 172.21.15.7:0/3726647054 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f223c004040 con 0x557305152440 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f222bfff640 1 -- 172.21.15.7:0/3726647054 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c0046c0 con 0x557305152440 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 -- 172.21.15.7:0/3726647054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 msgr2=0x557305156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x557305156f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f223c036040 tx=0x7f223c009560 comp rx=0 tx=0).stop 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 -- 172.21.15.7:0/3726647054 shutdown_connections 2024-03-19T00:41:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573051575b0 0x557305159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x557305156f40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 --2- 172.21.15.7:0/3726647054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.345+0000 7f224e827740 1 -- 172.21.15.7:0/3726647054 >> 172.21.15.7:0/3726647054 conn(0x55730514cfc0 msgr2=0x55730514d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 -- 172.21.15.7:0/3726647054 shutdown_connections 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 -- 172.21.15.7:0/3726647054 wait complete. 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 -- start start 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 Processor -- start 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x55730515beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 0x55730515c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573051575b0 0x557305163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55730515a500 con 0x557305152440 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557305072160 con 0x557305151a50 2024-03-19T00:41:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55730515a680 con 0x5573051575b0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f224e827740 10 client.0 _ll_register_callbacks cb 0x5573050aea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f22498f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 0x55730515c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f22498f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 0x55730515c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48288/0 (socket says 172.21.15.7:48288) 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f22498f7640 1 -- 172.21.15.7:0/2956166220 learned_addr learned my addr 172.21.15.7:0/2956166220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.346+0000 7f22490f6640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x55730515beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22490f6640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573051575b0 0x557305163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22498f7640 1 -- 172.21.15.7:0/2956166220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573051575b0 msgr2=0x557305163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22498f7640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573051575b0 0x557305163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22498f7640 1 -- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 msgr2=0x55730515beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22498f7640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x55730515beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22498f7640 1 -- 172.21.15.7:0/2956166220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557304e947d0 con 0x557305152440 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22498f7640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 0x55730515c3f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f223c02c170 tx=0x7f223c004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e547f385986d7269 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22297fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22297fa640 1 -- 172.21.15.7:0/2956166220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c0035f0 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22297fa640 1 -- 172.21.15.7:0/2956166220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f223c003790 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f223c004040 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22297fa640 1 -- 172.21.15.7:0/2956166220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f223c05b050 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f224e827740 10 client.16326 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557304dca3f0 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.347+0000 7f22297fa640 1 -- 172.21.15.7:0/2956166220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f223c004040 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f22297fa640 1 -- 172.21.15.7:0/2956166220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f223c004ca0 con 0x557305152440 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f224e827740 10 client.16326 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f224e827740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f224e827740 20 client.16326 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f224e827740 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557305167920 0x557305169d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f224e827740 4 client.16326 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573050c3a40 con 0x557305167920 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f22490f6640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557305167920 0x557305169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f22490f6640 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557305167920 0x557305169d10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5456a2c23655f0d5 server_cookie=6807d88a1616b47 in_seq=0 out_seq=0 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.348+0000 7f22297fa640 10 client.16326 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f22297fa640 1 -- 172.21.15.7:0/2956166220 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573050c3a40 con 0x557305167920 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f22297fa640 10 client.16326 handle_command_reply: tid=0 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 client.16326 shutdown 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 msgr2=0x55730515c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 0x55730515c3f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f223c02c170 tx=0x7f223c004490 comp rx=0 tx=0).stop 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 shutdown_connections 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573051575b0 0x557305163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557305167920 0x557305169d10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305152440 0x55730515c3f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 --2- 172.21.15.7:0/2956166220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x55730515beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f22297fa640 0 client.16326 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.349+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 >> 172.21.15.7:0/2956166220 conn(0x55730514cfc0 msgr2=0x55730514c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.350+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 shutdown_connections 2024-03-19T00:41:45.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.350+0000 7f224e827740 1 -- 172.21.15.7:0/2956166220 wait complete. 2024-03-19T00:41:45.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.350+0000 7f224e827740 20 client.16326 trim_cache size 0 max 16384 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 Processor -- start 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 -- start start 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 0x55730514d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x5573051633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x55730516e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22441acb00 con 0x55730514cfc0 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22441aa3d0 con 0x557305151a50 2024-03-19T00:41:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f224e827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22441a8220 con 0x557305152440 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f22490f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 0x55730514d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f22490f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 0x55730514d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48298/0 (socket says 172.21.15.7:48298) 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f22490f6640 1 -- 172.21.15.7:0/76210602 learned_addr learned my addr 172.21.15.7:0/76210602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.367+0000 7f22498f7640 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x5573051633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x55730516e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 -- 172.21.15.7:0/76210602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 msgr2=0x55730516e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x55730516e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 -- 172.21.15.7:0/76210602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 msgr2=0x5573051633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x5573051633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 -- 172.21.15.7:0/76210602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557304d8a0f0 con 0x55730514cfc0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f22490f6640 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 0x55730514d3a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2238000c00 tx=0x7f22380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cf6555aed4286 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f222b7fe640 1 -- 172.21.15.7:0/76210602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223801c040 con 0x55730514cfc0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f222b7fe640 1 -- 172.21.15.7:0/76210602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2238004380 con 0x55730514cfc0 2024-03-19T00:41:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f222b7fe640 1 -- 172.21.15.7:0/76210602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238005650 con 0x55730514cfc0 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f224e827740 1 -- 172.21.15.7:0/76210602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 msgr2=0x55730514d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.368+0000 7f224e827740 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 0x55730514d3a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2238000c00 tx=0x7f22380046c0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- 172.21.15.7:0/76210602 shutdown_connections 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557305152440 0x55730516e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305151a50 0x5573051633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 --2- 172.21.15.7:0/76210602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55730514cfc0 0x55730514d3a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- 172.21.15.7:0/76210602 >> 172.21.15.7:0/76210602 conn(0x557305153220 msgr2=0x557305150ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- 172.21.15.7:0/76210602 shutdown_connections 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- 172.21.15.7:0/76210602 wait complete. 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- start start 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 Processor -- start 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730514cfc0 0x557305162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305152440 0x557305161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22441a8890 con 0x557305151a50 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22441a9d00 con 0x557305152440 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22441a8f60 con 0x55730514cfc0 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f224e827740 10 client.0 _ll_register_callbacks cb 0x7f2238009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f22498f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f22498f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305163210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48304/0 (socket says 172.21.15.7:48304) 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f22498f7640 1 -- 172.21.15.7:0/3604421485 learned_addr learned my addr 172.21.15.7:0/3604421485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.369+0000 7f22490f6640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730514cfc0 0x557305162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22490f6640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305152440 0x557305161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22498f7640 1 -- 172.21.15.7:0/3604421485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730514cfc0 msgr2=0x557305162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22498f7640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730514cfc0 0x557305162cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22498f7640 1 -- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305152440 msgr2=0x557305161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:41:45.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22498f7640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305152440 0x557305161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22498f7640 1 -- 172.21.15.7:0/3604421485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557305161a70 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22490f6640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730514cfc0 0x557305162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22498f7640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305163210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55730515d390 tx=0x7f223c02cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23c353cd4a9653e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22297fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22297fa640 1 -- 172.21.15.7:0/3604421485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c045040 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22297fa640 1 -- 172.21.15.7:0/3604421485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f223c0051c0 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2238004380 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22297fa640 1 -- 172.21.15.7:0/3604421485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f223c005710 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 10 client.16332 fetch_fsmap learned FSMap version 27 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573051ce5c0 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22297fa640 1 -- 172.21.15.7:0/3604421485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2238004380 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f22297fa640 1 -- 172.21.15.7:0/3604421485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f223c004b70 con 0x557305151a50 2024-03-19T00:41:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 10 client.16332 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 20 client.16332 populate_metadata read hostname 'smithi007' 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573051575b0 0x5573051d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 4 client.16332 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.370+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573050bdd90 con 0x5573051575b0 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f22490f6640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573051575b0 0x5573051d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f22490f6640 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573051575b0 0x5573051d0fd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=775e9de36c26d0f3 server_cookie=95ac77c085248437 in_seq=0 out_seq=0 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f22297fa640 10 client.16332 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f22297fa640 1 -- 172.21.15.7:0/3604421485 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5573050bdd90 con 0x5573051575b0 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f22297fa640 10 client.16332 handle_command_reply: tid=0 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 client.16332 shutdown 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 msgr2=0x557305163210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305163210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55730515d390 tx=0x7f223c02cdc0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 shutdown_connections 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557305152440 0x557305161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573051575b0 0x5573051d0fd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557305151a50 0x557305163210 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 --2- 172.21.15.7:0/3604421485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55730514cfc0 0x557305162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f22297fa640 0 client.16332 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 >> 172.21.15.7:0/3604421485 conn(0x557305153220 msgr2=0x55730514b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 shutdown_connections 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.371+0000 7f224e827740 1 -- 172.21.15.7:0/3604421485 wait complete. 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224e827740 20 client.16332 trim_cache size 0 max 16384 2024-03-19T00:41:45.380 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: "047f948e-692b-44c3-9f60-52758b3bb3ef": { 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "047f948e-692b-44c3-9f60-52758b3bb3ef", 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:41:45.381 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:41:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f222c0a2800 msgr2=0x7f222c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f222c0a2800 0x7f222c0a4cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2234097dd0 tx=0x7f223406d5f0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 msgr2=0x7f224419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:41:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 0x7f224419f090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f223400eb10 tx=0x7f223400c2d0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 shutdown_connections 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f222c0a2800 0x7f222c0a4cc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22441022f0 0x7f224419f090 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2244101930 0x7f2244198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 --2- 172.21.15.7:0/580595687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100740 0x7f2244197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 >> 172.21.15.7:0/580595687 conn(0x7f22440fbcb0 msgr2=0x7f2244069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 shutdown_connections 2024-03-19T00:41:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:41:45.372+0000 7f224bb5b640 1 -- 172.21.15.7:0/580595687 wait complete. 2024-03-19T00:41:45.384 INFO:tasks.cephfs.filesystem:scrub status for tag:047f948e-692b-44c3-9f60-52758b3bb3ef - {'path': '/', 'tag': '047f948e-692b-44c3-9f60-52758b3bb3ef', 'options': 'recursive,force'} 2024-03-19T00:41:45.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:45 smithi007 ceph-mon[62854]: pgmap v465: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:45 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:41:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:45 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:46.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:45 smithi204 ceph-mon[66572]: pgmap v465: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:45 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:41:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:45 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:45 smithi044 ceph-mon[66366]: pgmap v465: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:45 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:41:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:45 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:46.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:46 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:41:46.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:46 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:46.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:46 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:41:46.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:46 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:46 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:41:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:46 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:46 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:41:47.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:46 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:46 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:41:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:46 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:46 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:41:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:46 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:41:47.969 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:47 smithi007 ceph-mon[62854]: pgmap v466: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:48.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:47 smithi204 ceph-mon[66572]: pgmap v466: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:48.111 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:47 smithi044 ceph-mon[66366]: pgmap v466: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:41:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:49 smithi007 ceph-mon[62854]: pgmap v467: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:50.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:49 smithi204 ceph-mon[66572]: pgmap v467: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:49 smithi044 ceph-mon[66366]: pgmap v467: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:41:51.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:51 smithi007 ceph-mon[62854]: pgmap v468: 129 pgs: 129 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:51 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:41:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:51 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:41:52.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:51 smithi204 ceph-mon[66572]: pgmap v468: 129 pgs: 129 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:51 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:41:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:51 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:41:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:51 smithi044 ceph-mon[66366]: pgmap v468: 129 pgs: 129 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:41:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:51 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:41:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:51 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:41:53.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:53 smithi007 ceph-mon[62854]: pgmap v469: 129 pgs: 129 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:41:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:53 smithi204 ceph-mon[66572]: pgmap v469: 129 pgs: 129 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:41:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:53 smithi044 ceph-mon[66366]: pgmap v469: 129 pgs: 129 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:41:55.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:55 smithi007 ceph-mon[62854]: pgmap v470: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:41:56.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:55 smithi204 ceph-mon[66572]: pgmap v470: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:41:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:55 smithi044 ceph-mon[66366]: pgmap v470: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:41:57.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:57 smithi007 ceph-mon[62854]: pgmap v471: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:41:58.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:57 smithi204 ceph-mon[66572]: pgmap v471: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:41:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:57 smithi044 ceph-mon[66366]: pgmap v471: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:41:59.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:41:59 smithi007 ceph-mon[62854]: pgmap v472: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:00.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:41:59 smithi204 ceph-mon[66572]: pgmap v472: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:41:59 smithi044 ceph-mon[66366]: pgmap v472: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:01.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:01 smithi007 ceph-mon[62854]: pgmap v473: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:02.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:01 smithi204 ceph-mon[66572]: pgmap v473: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:02.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:01 smithi044 ceph-mon[66366]: pgmap v473: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:03.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:03 smithi007 ceph-mon[62854]: pgmap v474: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:04.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:03 smithi204 ceph-mon[66572]: pgmap v474: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:03 smithi044 ceph-mon[66366]: pgmap v474: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:05 smithi007 ceph-mon[62854]: pgmap v475: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:05 smithi204 ceph-mon[66572]: pgmap v475: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:05 smithi044 ceph-mon[66366]: pgmap v475: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:07.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:07 smithi007 ceph-mon[62854]: pgmap v476: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:07 smithi204 ceph-mon[66572]: pgmap v476: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:07 smithi044 ceph-mon[66366]: pgmap v476: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:09 smithi007 ceph-mon[62854]: pgmap v477: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:09 smithi204 ceph-mon[66572]: pgmap v477: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:09 smithi044 ceph-mon[66366]: pgmap v477: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:11 smithi007 ceph-mon[62854]: pgmap v478: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:42:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:11 smithi204 ceph-mon[66572]: pgmap v478: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:42:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:11 smithi044 ceph-mon[66366]: pgmap v478: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:42:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:13 smithi007 ceph-mon[62854]: pgmap v479: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:13 smithi204 ceph-mon[66572]: pgmap v479: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:13 smithi044 ceph-mon[66366]: pgmap v479: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:15.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:42:15.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 -- 172.21.15.7:0/3351321774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57501022f0 msgr2=0x7f5750108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 --2- 172.21.15.7:0/3351321774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57501022f0 0x7f5750108b80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ca0 tx=0x7f574002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 -- 172.21.15.7:0/3351321774 shutdown_connections 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 --2- 172.21.15.7:0/3351321774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57501022f0 0x7f5750108b80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 --2- 172.21.15.7:0/3351321774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750101930 0x7f5750101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 --2- 172.21.15.7:0/3351321774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5750100740 0x7f5750100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 -- 172.21.15.7:0/3351321774 >> 172.21.15.7:0/3351321774 conn(0x7f57500fbcb0 msgr2=0x7f57500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 -- 172.21.15.7:0/3351321774 shutdown_connections 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 -- 172.21.15.7:0/3351321774 wait complete. 2024-03-19T00:42:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 Processor -- start 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.502+0000 7f575531d640 1 -- start start 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750100740 0x7f5750197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 0x7f5750197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57501022f0 0x7f575019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f575010d390 con 0x7f5750101930 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f575010d210 con 0x7f5750100740 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f575010d510 con 0x7f57501022f0 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 0x7f5750197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 0x7f5750197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34982/0 (socket says 172.21.15.7:34982) 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 -- 172.21.15.7:0/3769341612 learned_addr learned my addr 172.21.15.7:0/3769341612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574effd640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750100740 0x7f5750197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574f7fe640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57501022f0 0x7f575019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 -- 172.21.15.7:0/3769341612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57501022f0 msgr2=0x7f575019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57501022f0 0x7f575019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 -- 172.21.15.7:0/3769341612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750100740 msgr2=0x7f5750197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750100740 0x7f5750197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 -- 172.21.15.7:0/3769341612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575019f630 con 0x7f5750101930 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 0x7f5750197f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f573800ea60 tx=0x7f573800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f5747fff640 1 -- 172.21.15.7:0/3769341612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573800cde0 con 0x7f5750101930 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f5747fff640 1 -- 172.21.15.7:0/3769341612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57380035f0 con 0x7f5750101930 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575019f920 con 0x7f5750101930 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f5747fff640 1 -- 172.21.15.7:0/3769341612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738010690 con 0x7f5750101930 2024-03-19T00:42:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.503+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575019fe70 con 0x7f5750101930 2024-03-19T00:42:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.506+0000 7f5747fff640 1 -- 172.21.15.7:0/3769341612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f57380037a0 con 0x7f5750101930 2024-03-19T00:42:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.506+0000 7f5757fdf740 1 Processor -- start 2024-03-19T00:42:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.506+0000 7f5747fff640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f573c0a2800 0x7f573c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.506+0000 7f5757fdf740 1 -- start start 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.506+0000 7f574f7fe640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f573c0a2800 0x7f573c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.506+0000 7f5747fff640 1 -- 172.21.15.7:0/3769341612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f57380cf050 con 0x7f5750101930 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af0a50 0x5650c0af0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0af5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af65b0 0x5650c0af8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f5757fdf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650c0a55500 con 0x5650c0af65b0 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f5757fdf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5650c09fdbe0 con 0x5650c0af1440 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f5757fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5650c0a515a0 con 0x5650c0af0a50 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0af5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0af5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36662/0 (socket says 172.21.15.7:36662) 2024-03-19T00:42:15.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 -- 172.21.15.7:0/2046522144 learned_addr learned my addr 172.21.15.7:0/2046522144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:15.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574effd640 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af0a50 0x5650c0af0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574f7fe640 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af65b0 0x5650c0af8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 -- 172.21.15.7:0/2046522144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af0a50 msgr2=0x5650c0af0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af0a50 0x5650c0af0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 -- 172.21.15.7:0/2046522144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af65b0 msgr2=0x5650c0af8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af65b0 0x5650c0af8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 -- 172.21.15.7:0/2046522144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c0a1f000 con 0x5650c0af1440 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574f7fe640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f573c0a2800 0x7f573c0a4cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5740006010 tx=0x7f5740032ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f574e7fc640 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0af5f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f57380a3900 tx=0x7f573809b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1239a6e3c5113e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f57457fa640 1 -- 172.21.15.7:0/2046522144 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380b4070 con 0x5650c0af1440 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f57457fa640 1 -- 172.21.15.7:0/2046522144 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57380860c0 con 0x5650c0af1440 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.507+0000 7f57457fa640 1 -- 172.21.15.7:0/2046522144 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738097d00 con 0x5650c0af1440 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 -- 172.21.15.7:0/2046522144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 msgr2=0x5650c0af5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0af5f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f57380a3900 tx=0x7f573809b1d0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 -- 172.21.15.7:0/2046522144 shutdown_connections 2024-03-19T00:42:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af65b0 0x5650c0af8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0af5f40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2046522144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af0a50 0x5650c0af0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 -- 172.21.15.7:0/2046522144 >> 172.21.15.7:0/2046522144 conn(0x5650c0aebfc0 msgr2=0x5650c0aec3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 -- 172.21.15.7:0/2046522144 shutdown_connections 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.508+0000 7f5757fdf740 1 -- 172.21.15.7:0/2046522144 wait complete. 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 -- start start 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 Processor -- start 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0afaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0afb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b02420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650c0af9560 con 0x5650c0af0a50 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5650c0a11160 con 0x5650c0af1440 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5650c0af96e0 con 0x5650c0af65b0 2024-03-19T00:42:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f5757fdf740 10 client.0 _ll_register_callbacks cb 0x5650c0a4da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0afaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0afaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35000/0 (socket says 172.21.15.7:35000) 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 -- 172.21.15.7:0/3189398791 learned_addr learned my addr 172.21.15.7:0/3189398791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0afb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574f7fe640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b02420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 -- 172.21.15.7:0/3189398791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 msgr2=0x5650c0b02420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b02420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 -- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 msgr2=0x5650c0afb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0afb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 -- 172.21.15.7:0/3189398791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c08337d0 con 0x5650c0af0a50 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574f7fe640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b02420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0afb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f574effd640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0afaea0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f57480126e0 tx=0x7f5748012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13916c9f129f18af server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.509+0000 7f57367fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f57367fc640 1 -- 172.21.15.7:0/3189398791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57480121d0 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f57367fc640 1 -- 172.21.15.7:0/3189398791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5748004050 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650c0a62a40 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f57367fc640 1 -- 172.21.15.7:0/3189398791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5748025050 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 10 client.16344 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650c07693f0 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f57367fc640 1 -- 172.21.15.7:0/3189398791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5650c0a62a40 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f57367fc640 1 -- 172.21.15.7:0/3189398791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5748003710 con 0x5650c0af0a50 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 10 client.16344 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 20 client.16344 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b06d90 0x5650c0b09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 4 client.16344 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:15.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c0a5cd90 con 0x5650c0b06d90 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.510+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b06d90 0x5650c0b09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.511+0000 7f57367fc640 10 client.16344 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.511+0000 7f574e7fc640 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b06d90 0x5650c0b09250 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80818ddc710ce4f3 server_cookie=cddd55a244d00a7b in_seq=0 out_seq=0 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f57367fc640 1 -- 172.21.15.7:0/3189398791 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650c0a5cd90 con 0x5650c0b06d90 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f57367fc640 10 client.16344 handle_command_reply: tid=0 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 client.16344 shutdown 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 msgr2=0x5650c0afaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0afaea0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f57480126e0 tx=0x7f5748012ba0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 shutdown_connections 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b02420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0afb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b06d90 0x5650c0b09250 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 --2- 172.21.15.7:0/3189398791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0afaea0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f57367fc640 0 client.16344 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:15.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 >> 172.21.15.7:0/3189398791 conn(0x5650c0aebfc0 msgr2=0x5650c0aeb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 shutdown_connections 2024-03-19T00:42:15.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 1 -- 172.21.15.7:0/3189398791 wait complete. 2024-03-19T00:42:15.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.512+0000 7f5757fdf740 20 client.16344 trim_cache size 0 max 16384 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 Processor -- start 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 -- start start 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0aefab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0afb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57501acc00 con 0x5650c0af0a50 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57501aa4d0 con 0x5650c0af1440 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f5757fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a8320 con 0x5650c0af65b0 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f574effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0aefab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f574effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0aefab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35010/0 (socket says 172.21.15.7:35010) 2024-03-19T00:42:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f574effd640 1 -- 172.21.15.7:0/1059945603 learned_addr learned my addr 172.21.15.7:0/1059945603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f574e7fc640 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.530+0000 7f574f7fe640 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0afb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574effd640 1 -- 172.21.15.7:0/1059945603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 msgr2=0x5650c0afb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574effd640 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0afb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574effd640 1 -- 172.21.15.7:0/1059945603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 msgr2=0x5650c0b023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574effd640 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574effd640 1 -- 172.21.15.7:0/1059945603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c07290f0 con 0x5650c0af0a50 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574e7fc640 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f574effd640 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0aefab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5748001eb0 tx=0x7f57480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62228b9f0b7065de server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f5744ff9640 1 -- 172.21.15.7:0/1059945603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57480035f0 con 0x5650c0af0a50 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f5744ff9640 1 -- 172.21.15.7:0/1059945603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5748003790 con 0x5650c0af0a50 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.531+0000 7f5744ff9640 1 -- 172.21.15.7:0/1059945603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574800fd60 con 0x5650c0af0a50 2024-03-19T00:42:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- 172.21.15.7:0/1059945603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 msgr2=0x5650c0aefab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0aefab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5748001eb0 tx=0x7f57480096a0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- 172.21.15.7:0/1059945603 shutdown_connections 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0afb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- 172.21.15.7:0/1059945603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0aefab0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- 172.21.15.7:0/1059945603 >> 172.21.15.7:0/1059945603 conn(0x5650c0aebfc0 msgr2=0x5650c0aea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- 172.21.15.7:0/1059945603 shutdown_connections 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- 172.21.15.7:0/1059945603 wait complete. 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- start start 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 Processor -- start 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0affb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a8990 con 0x5650c0af0a50 2024-03-19T00:42:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a9e00 con 0x5650c0af1440 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57501a9060 con 0x5650c0af65b0 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f5757fdf740 10 client.0 _ll_register_callbacks cb 0x7f5748009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f574effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0affb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f574effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0affb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35026/0 (socket says 172.21.15.7:35026) 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f574effd640 1 -- 172.21.15.7:0/2332161163 learned_addr learned my addr 172.21.15.7:0/2332161163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.532+0000 7f574e7fc640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574f7fe640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574effd640 1 -- 172.21.15.7:0/2332161163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 msgr2=0x5650c0b010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574effd640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574effd640 1 -- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 msgr2=0x5650c0b000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574effd640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574effd640 1 -- 172.21.15.7:0/2332161163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c0b016a0 con 0x5650c0af0a50 2024-03-19T00:42:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f574effd640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0affb60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5748003c50 tx=0x7f574800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2a6d0d9013d91b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f57367fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f57367fc640 1 -- 172.21.15.7:0/2332161163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748003940 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f57367fc640 1 -- 172.21.15.7:0/2332161163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f574800fcb0 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5748003790 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f57367fc640 1 -- 172.21.15.7:0/2332161163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5748005090 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f5757fdf740 10 client.16350 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650c0b6d150 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.533+0000 7f57367fc640 1 -- 172.21.15.7:0/2332161163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5748003790 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f57367fc640 1 -- 172.21.15.7:0/2332161163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5748010990 con 0x5650c0af0a50 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 10 client.16350 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 20 client.16350 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b6da30 0x5650c0b6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 4 client.16350 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650c09da540 con 0x5650c0b6da30 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f574e7fc640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b6da30 0x5650c0b6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f57367fc640 10 client.16350 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f574e7fc640 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b6da30 0x5650c0b6fe50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3263873b7f32705 server_cookie=6c1837392343d899 in_seq=0 out_seq=0 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f57367fc640 1 -- 172.21.15.7:0/2332161163 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5650c09da540 con 0x5650c0b6da30 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f57367fc640 10 client.16350 handle_command_reply: tid=0 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 client.16350 shutdown 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 msgr2=0x5650c0affb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0affb60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5748003c50 tx=0x7f574800f790 comp rx=0 tx=0).stop 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 shutdown_connections 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650c0af65b0 0x5650c0b010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5650c0af1440 0x5650c0b000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5650c0b6da30 0x5650c0b6fe50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 --2- 172.21.15.7:0/2332161163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5650c0af0a50 0x5650c0affb60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f57367fc640 0 client.16350 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.534+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 >> 172.21.15.7:0/2332161163 conn(0x5650c0aebfc0 msgr2=0x5650c0aef930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 shutdown_connections 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f5757fdf740 1 -- 172.21.15.7:0/2332161163 wait complete. 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f5757fdf740 20 client.16350 trim_cache size 0 max 16384 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:42:15.543 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:42:15.544 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:42:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f573c0a2800 msgr2=0x7f573c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f573c0a2800 0x7f573c0a4cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5740006010 tx=0x7f5740032ea0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 msgr2=0x7f5750197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 0x7f5750197f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f573800ea60 tx=0x7f573800ef20 comp rx=0 tx=0).stop 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 shutdown_connections 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f573c0a2800 0x7f573c0a4cc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57501022f0 0x7f575019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5750101930 0x7f5750197f50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 --2- 172.21.15.7:0/3769341612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5750100740 0x7f5750197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 >> 172.21.15.7:0/3769341612 conn(0x7f57500fbcb0 msgr2=0x7f5750104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 shutdown_connections 2024-03-19T00:42:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.535+0000 7f575531d640 1 -- 172.21.15.7:0/3769341612 wait complete. 2024-03-19T00:42:15.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:42:15.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 -- 172.21.15.7:0/2207417966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 msgr2=0x7f5c28102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/2207417966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28102f20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009900 tx=0x7f5c1002bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 -- 172.21.15.7:0/2207417966 shutdown_connections 2024-03-19T00:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/2207417966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c28103460 0x7f5c28109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/2207417966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28102f20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/2207417966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c281018b0 0x7f5c28101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 -- 172.21.15.7:0/2207417966 >> 172.21.15.7:0/2207417966 conn(0x7f5c280fce40 msgr2=0x7f5c280ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 -- 172.21.15.7:0/2207417966 shutdown_connections 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 -- 172.21.15.7:0/2207417966 wait complete. 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 Processor -- start 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.649+0000 7f5c2cd09640 1 -- start start 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 0x7f5c28197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c28103460 0x7f5c2819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2810d350 con 0x7f5c28102aa0 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2810d1d0 con 0x7f5c281018b0 2024-03-19T00:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2810d4d0 con 0x7f5c28103460 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c1ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 0x7f5c28197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c26d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c28103460 0x7f5c2819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 0x7f5c28197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36694/0 (socket says 172.21.15.7:36694) 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 -- 172.21.15.7:0/1948464020 learned_addr learned my addr 172.21.15.7:0/1948464020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c1ffff640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35048/0 (socket says 172.21.15.7:35048) 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 -- 172.21.15.7:0/1948464020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c28103460 msgr2=0x7f5c2819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c28103460 0x7f5c2819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 -- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 msgr2=0x7f5c28197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 -- 172.21.15.7:0/1948464020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2819f6a0 con 0x7f5c281018b0 2024-03-19T00:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2659c640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 0x7f5c28197a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9c0 tx=0x7f5c1800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800ccc0 con 0x7f5c281018b0 2024-03-19T00:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c180035f0 con 0x7f5c281018b0 2024-03-19T00:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800ce60 con 0x7f5c281018b0 2024-03-19T00:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2819f990 con 0x7f5c281018b0 2024-03-19T00:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.650+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2819feb0 con 0x7f5c281018b0 2024-03-19T00:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.652+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c28108bf0 con 0x7f5c281018b0 2024-03-19T00:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.652+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5c180037a0 con 0x7f5c281018b0 2024-03-19T00:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.653+0000 7f5c1f7fe640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c140a2800 0x7f5c140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.653+0000 7f5c1ffff640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c140a2800 0x7f5c140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.653+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5c180cf170 con 0x7f5c281018b0 2024-03-19T00:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.653+0000 7f5c1ffff640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c140a2800 0x7f5c140a4cc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009230 tx=0x7f5c10005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:15.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.658+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5c1809beb0 con 0x7f5c281018b0 2024-03-19T00:42:15.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.880+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c28062420 con 0x7f5c281018b0 2024-03-19T00:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.881+0000 7f5c1f7fe640 1 -- 172.21.15.7:0/1948464020 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f5c1809b7b0 con 0x7f5c281018b0 2024-03-19T00:42:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:42:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:42:15.883 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:42:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c140a2800 msgr2=0x7f5c140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c140a2800 0x7f5c140a4cc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009230 tx=0x7f5c10005fd0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 msgr2=0x7f5c28197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 0x7f5c28197a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9c0 tx=0x7f5c1800edc0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 shutdown_connections 2024-03-19T00:42:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c140a2800 0x7f5c140a4cc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c28103460 0x7f5c2819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c28102aa0 0x7f5c28197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 --2- 172.21.15.7:0/1948464020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c281018b0 0x7f5c28197a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 >> 172.21.15.7:0/1948464020 conn(0x7f5c280fce40 msgr2=0x7f5c28105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 shutdown_connections 2024-03-19T00:42:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.883+0000 7f5c2cd09640 1 -- 172.21.15.7:0/1948464020 wait complete. 2024-03-19T00:42:15.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:42:15.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:15 smithi007 ceph-mon[62854]: pgmap v480: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-03-19T00:42:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 -- 172.21.15.7:0/1140479587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 msgr2=0x7fee38108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 --2- 172.21.15.7:0/1140479587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee38108b60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fee30009ca0 tx=0x7fee3002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 -- 172.21.15.7:0/1140479587 shutdown_connections 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 --2- 172.21.15.7:0/1140479587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee38108b60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 --2- 172.21.15.7:0/1140479587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee38101910 0x7fee38101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 --2- 172.21.15.7:0/1140479587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38100720 0x7fee38100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 -- 172.21.15.7:0/1140479587 >> 172.21.15.7:0/1140479587 conn(0x7fee380fbc50 msgr2=0x7fee380fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 -- 172.21.15.7:0/1140479587 shutdown_connections 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:15.999+0000 7fee3e834640 1 -- 172.21.15.7:0/1140479587 wait complete. 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 Processor -- start 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 -- start start 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38100720 0x7fee381979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee38101910 0x7fee38197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee3819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee3810c230 con 0x7fee381022d0 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee3810c0b0 con 0x7fee38101910 2024-03-19T00:42:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3e834640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee3810c3b0 con 0x7fee38100720 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.000+0000 7fee3cdd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee3819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee3819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35058/0 (socket says 172.21.15.7:35058) 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2451477846 learned_addr learned my addr 172.21.15.7:0/2451477846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee377fe640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee38101910 0x7fee38197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee37fff640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38100720 0x7fee381979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2451477846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38100720 msgr2=0x7fee381979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38100720 0x7fee381979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2451477846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee38101910 msgr2=0x7fee38197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee38101910 0x7fee38197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2451477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3819f5e0 con 0x7fee381022d0 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee3819ef70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fee3002c700 tx=0x7fee300328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee357fa640 1 -- 172.21.15.7:0/2451477846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30041020 con 0x7fee381022d0 2024-03-19T00:42:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee357fa640 1 -- 172.21.15.7:0/2451477846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee30003e20 con 0x7fee381022d0 2024-03-19T00:42:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3819f870 con 0x7fee381022d0 2024-03-19T00:42:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee357fa640 1 -- 172.21.15.7:0/2451477846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30039040 con 0x7fee381022d0 2024-03-19T00:42:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.001+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3819fd60 con 0x7fee381022d0 2024-03-19T00:42:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.003+0000 7fee357fa640 1 -- 172.21.15.7:0/2451477846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fee30044020 con 0x7fee381022d0 2024-03-19T00:42:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 Processor -- start 2024-03-19T00:42:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 -- start start 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee357fa640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee200a2800 0x7fee200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aaa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7aaf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4f7a0e4b0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f4f79b6d70 con 0x55f4f7aaace0 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee4141c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4f7a0a550 con 0x55f4f7aafe50 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee377fe640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee200a2800 0x7fee200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee357fa640 1 -- 172.21.15.7:0/2451477846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fee300f0fc0 con 0x7fee381022d0 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aaa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aaa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35062/0 (socket says 172.21.15.7:35062) 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee37fff640 1 -- 172.21.15.7:0/2137290291 learned_addr learned my addr 172.21.15.7:0/2137290291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee377fe640 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7aaf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2137290291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 msgr2=0x55f4f7aaf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7aaf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2137290291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 msgr2=0x55f4f7aaa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aaa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 -- 172.21.15.7:0/2137290291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f79d9210 con 0x55f4f7aafe50 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee37fff640 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aaa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee377fe640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee200a2800 0x7fee200a4cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee38198ed0 tx=0x7fee2c009b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.004+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab22e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fee300eead0 tx=0x7fee300bd500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d2b8494136d6489 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee1e7fc640 1 -- 172.21.15.7:0/2137290291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30034700 con 0x55f4f7aafe50 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee1e7fc640 1 -- 172.21.15.7:0/2137290291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee300bdd00 con 0x55f4f7aafe50 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee1e7fc640 1 -- 172.21.15.7:0/2137290291 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee300bb900 con 0x55f4f7aafe50 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 -- 172.21.15.7:0/2137290291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 msgr2=0x55f4f7ab22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab22e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fee300eead0 tx=0x7fee300bd500 comp rx=0 tx=0).stop 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 -- 172.21.15.7:0/2137290291 shutdown_connections 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab22e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7aaf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 --2- 172.21.15.7:0/2137290291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aaa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 -- 172.21.15.7:0/2137290291 >> 172.21.15.7:0/2137290291 conn(0x55f4f7aa5860 msgr2=0x55f4f7aa5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 -- 172.21.15.7:0/2137290291 shutdown_connections 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.005+0000 7fee4141c740 1 -- 172.21.15.7:0/2137290291 wait complete. 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 -- start start 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 Processor -- start 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7abba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4f7ab2bc0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f4f79ca310 con 0x55f4f7aaace0 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4f7ab2d40 con 0x55f4f7aafe50 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee4141c740 10 client.0 _ll_register_callbacks cb 0x55f4f79be2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35066/0 (socket says 172.21.15.7:35066) 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee37fff640 1 -- 172.21.15.7:0/3481696978 learned_addr learned my addr 172.21.15.7:0/3481696978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee377fe640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.006+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7abba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee37fff640 1 -- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 msgr2=0x55f4f7abba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee37fff640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7abba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee37fff640 1 -- 172.21.15.7:0/3481696978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 msgr2=0x55f4f7ab4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee37fff640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee37fff640 1 -- 172.21.15.7:0/3481696978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f77223f0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7abba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee377fe640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee37fff640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab4510 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fee28012720 tx=0x7fee28012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390ca42ef5ac4067 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee177fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee177fe640 1 -- 172.21.15.7:0/3481696978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28004030 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee177fe640 1 -- 172.21.15.7:0/3481696978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fee280041d0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee300bdd00 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee177fe640 1 -- 172.21.15.7:0/3481696978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee28025050 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee4141c740 10 client.16368 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4f77ec7d0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee177fe640 1 -- 172.21.15.7:0/3481696978 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee300bdd00 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee177fe640 1 -- 172.21.15.7:0/3481696978 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fee28003990 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee4141c740 10 client.16368 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee4141c740 10 client.16368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.007+0000 7fee4141c740 20 client.16368 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.008+0000 7fee4141c740 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7ac0420 0x55f4f7ac2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.008+0000 7fee4141c740 4 client.16368 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.008+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f7a6c3e0 con 0x55f4f7ac0420 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.008+0000 7fee377fe640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7ac0420 0x55f4f7ac2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.008+0000 7fee377fe640 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7ac0420 0x55f4f7ac2890 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a36c16698d0d8b server_cookie=e78455eca43c9b7a in_seq=0 out_seq=0 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.008+0000 7fee177fe640 10 client.16368 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee177fe640 1 -- 172.21.15.7:0/3481696978 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f4f7a6c3e0 con 0x55f4f7ac0420 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee177fe640 10 client.16368 handle_command_reply: tid=0 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 client.16368 shutdown 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 msgr2=0x55f4f7ab4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab4510 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fee28012720 tx=0x7fee28012b20 comp rx=0 tx=0).stop 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 shutdown_connections 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7ac0420 0x55f4f7ac2890 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aafe50 0x55f4f7abba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee177fe640 0 client.16368 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 --2- 172.21.15.7:0/3481696978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab4510 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 >> 172.21.15.7:0/3481696978 conn(0x55f4f7aa5860 msgr2=0x55f4f7aa4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 shutdown_connections 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 1 -- 172.21.15.7:0/3481696978 wait complete. 2024-03-19T00:42:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.009+0000 7fee4141c740 20 client.16368 trim_cache size 0 max 16384 2024-03-19T00:42:16.020 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:15 smithi204 ceph-mon[66572]: pgmap v480: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 Processor -- start 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 -- start start 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aa0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7abba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee381acb20 con 0x55f4f7aafe50 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee381aa3f0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.022+0000 7fee4141c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee381a8240 con 0x55f4f7aaace0 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee3cdd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aa0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee377fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7abba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aa0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36720/0 (socket says 172.21.15.7:36720) 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 -- 172.21.15.7:0/3822305388 learned_addr learned my addr 172.21.15.7:0/3822305388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 -- 172.21.15.7:0/3822305388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 msgr2=0x55f4f7abba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7abba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 -- 172.21.15.7:0/3822305388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 msgr2=0x55f4f7ab3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 -- 172.21.15.7:0/3822305388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f7a7b880 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee3cdd1640 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee37fff640 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aa0a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fee280126f0 tx=0x7fee28012360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4ef7818d7c96a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee1dffb640 1 -- 172.21.15.7:0/3822305388 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2802e070 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee1dffb640 1 -- 172.21.15.7:0/3822305388 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee28004450 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.023+0000 7fee1dffb640 1 -- 172.21.15.7:0/3822305388 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28003aa0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- 172.21.15.7:0/3822305388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 msgr2=0x55f4f7aa0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aa0a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fee280126f0 tx=0x7fee28012360 comp rx=0 tx=0).stop 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- 172.21.15.7:0/3822305388 shutdown_connections 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7ab3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7abba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- 172.21.15.7:0/3822305388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7aa0a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- 172.21.15.7:0/3822305388 >> 172.21.15.7:0/3822305388 conn(0x55f4f7aa5860 msgr2=0x55f4f7aa4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- 172.21.15.7:0/3822305388 shutdown_connections 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- 172.21.15.7:0/3822305388 wait complete. 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- start start 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 Processor -- start 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7aba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee381a88b0 con 0x55f4f7aafe50 2024-03-19T00:42:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee381a9d20 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee381a8f80 con 0x55f4f7aaace0 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.024+0000 7fee4141c740 10 client.0 _ll_register_callbacks cb 0x7fee28009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee3cdd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7aba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36730/0 (socket says 172.21.15.7:36730) 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 -- 172.21.15.7:0/3685813909 learned_addr learned my addr 172.21.15.7:0/3685813909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee377fe640 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 -- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 msgr2=0x55f4f7ab9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 -- 172.21.15.7:0/3685813909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 msgr2=0x55f4f7aba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7aba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 -- 172.21.15.7:0/3685813909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f7b24870 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee37fff640 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab8dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee28012b60 tx=0x7fee28009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8fa7ba645b9c6b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee177fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee177fe640 1 -- 172.21.15.7:0/3685813909 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2802e030 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4f7a1b9f0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee177fe640 1 -- 172.21.15.7:0/3685813909 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fee28018070 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee177fe640 1 -- 172.21.15.7:0/3685813909 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee28038050 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee4141c740 10 client.26222 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.025+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4f7abb830 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee177fe640 1 -- 172.21.15.7:0/3685813909 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f4f7a1b9f0 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee177fe640 1 -- 172.21.15.7:0/3685813909 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fee2800f790 con 0x55f4f7aaa2f0 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 10 client.26222 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 10 client.26222 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 20 client.26222 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7b2f570 0x55f4f7b31990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 4 client.26222 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4f7a15d40 con 0x55f4f7b2f570 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee377fe640 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7b2f570 0x55f4f7b31990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee177fe640 10 client.26222 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee377fe640 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7b2f570 0x55f4f7b31990 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20fe9f71d531d073 server_cookie=7d8ea6dc4aa15da in_seq=0 out_seq=0 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee177fe640 1 -- 172.21.15.7:0/3685813909 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4f7a15d40 con 0x55f4f7b2f570 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee177fe640 10 client.26222 handle_command_reply: tid=0 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.026+0000 7fee4141c740 1 client.26222 shutdown 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 msgr2=0x55f4f7ab8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab8dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee28012b60 tx=0x7fee28009e50 comp rx=0 tx=0).stop 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 shutdown_connections 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4f7b2f570 0x55f4f7b31990 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4f7aafe50 0x55f4f7aba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f7aaace0 0x55f4f7ab9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 --2- 172.21.15.7:0/3685813909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4f7aaa2f0 0x55f4f7ab8dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee177fe640 0 client.26222 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 >> 172.21.15.7:0/3685813909 conn(0x55f4f7aa5860 msgr2=0x55f4f7aa9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 shutdown_connections 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 1 -- 172.21.15.7:0/3685813909 wait complete. 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee4141c740 20 client.26222 trim_cache size 0 max 16384 2024-03-19T00:42:16.035 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:42:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee200a2800 msgr2=0x7fee200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee200a2800 0x7fee200a4cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee38198ed0 tx=0x7fee2c009b70 comp rx=0 tx=0).stop 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 msgr2=0x7fee3819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee3819ef70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fee3002c700 tx=0x7fee300328a0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 shutdown_connections 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fee200a2800 0x7fee200a4cc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee381022d0 0x7fee3819ef70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee38101910 0x7fee38197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 --2- 172.21.15.7:0/2451477846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee38100720 0x7fee381979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 >> 172.21.15.7:0/2451477846 conn(0x7fee380fbc50 msgr2=0x7fee380fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 shutdown_connections 2024-03-19T00:42:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.027+0000 7fee3e834640 1 -- 172.21.15.7:0/2451477846 wait complete. 2024-03-19T00:42:16.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:42:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:15 smithi044 ceph-mon[66366]: pgmap v480: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 -- 172.21.15.7:0/1875844147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 msgr2=0x7f1bc8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 --2- 172.21.15.7:0/1875844147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8101db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009900 tx=0x7f1bbc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 -- 172.21.15.7:0/1875844147 shutdown_connections 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 --2- 172.21.15.7:0/1875844147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bc81022f0 0x7f1bc8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 --2- 172.21.15.7:0/1875844147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8101db0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 --2- 172.21.15.7:0/1875844147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc8100740 0x7f1bc8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 -- 172.21.15.7:0/1875844147 >> 172.21.15.7:0/1875844147 conn(0x7f1bc80fbc90 msgr2=0x7f1bc80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 -- 172.21.15.7:0/1875844147 shutdown_connections 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 -- 172.21.15.7:0/1875844147 wait complete. 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 Processor -- start 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.143+0000 7f1bce434640 1 -- start start 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bce434640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bc8100740 0x7f1bc8196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bce434640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bce434640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81022f0 0x7f1bc819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bce434640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc810c250 con 0x7f1bc8101930 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bce434640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc810c0d0 con 0x7f1bc81022f0 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bce434640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc810c3d0 con 0x7f1bc8100740 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35110/0 (socket says 172.21.15.7:35110) 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1586092800 learned_addr learned my addr 172.21.15.7:0/1586092800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81022f0 0x7f1bc819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bc8100740 0x7f1bc8196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1586092800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bc8100740 msgr2=0x7f1bc8196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bc8100740 0x7f1bc8196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1586092800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81022f0 msgr2=0x7f1bc819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81022f0 0x7f1bc819df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1586092800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc819e590 con 0x7f1bc8101930 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81022f0 0x7f1bc819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8196eb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006010 tx=0x7f1bbc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.144+0000 7f1bc57fa640 1 -- 172.21.15.7:0/1586092800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc043020 con 0x7f1bc8101930 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.145+0000 7f1bc57fa640 1 -- 172.21.15.7:0/1586092800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bbc0043d0 con 0x7f1bc8101930 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.145+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc819e820 con 0x7f1bc8101930 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.145+0000 7f1bc57fa640 1 -- 172.21.15.7:0/1586092800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc003a80 con 0x7f1bc8101930 2024-03-19T00:42:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.145+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc819ed70 con 0x7f1bc8101930 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bc57fa640 1 -- 172.21.15.7:0/1586092800 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1bbc046020 con 0x7f1bc8101930 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 Processor -- start 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 -- start start 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 0x5595ce361710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce361ce0 0x5595ce3667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce3692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5595ce2c54b0 con 0x5595ce3612f0 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595ce26dd70 con 0x5595ce366e50 2024-03-19T00:42:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bd1019740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595ce2c1550 con 0x5595ce361ce0 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bcc9d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 0x5595ce361710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bcc9d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 0x5595ce361710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35116/0 (socket says 172.21.15.7:35116) 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/3886530487 learned_addr learned my addr 172.21.15.7:0/3886530487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce3692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce361ce0 0x5595ce3667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.147+0000 7f1bc57fa640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bb00a2800 0x7f1bb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/3886530487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce361ce0 msgr2=0x5595ce3667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce361ce0 0x5595ce3667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/3886530487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 msgr2=0x5595ce3692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce3692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/3886530487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ce290210 con 0x5595ce3612f0 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce3692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bc57fa640 1 -- 172.21.15.7:0/1586092800 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1bbc0f0f90 con 0x7f1bc8101930 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bb00a2800 0x7f1bb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 0x5595ce361710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011990 tx=0x7f1bc0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d04d0c99a3d80e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bae7fc640 1 -- 172.21.15.7:0/3886530487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0017050 con 0x5595ce3612f0 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bae7fc640 1 -- 172.21.15.7:0/3886530487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bc001cc30 con 0x5595ce3612f0 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bae7fc640 1 -- 172.21.15.7:0/3886530487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0010960 con 0x5595ce3612f0 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.148+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bb00a2800 0x7f1bb00a4cc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0023000 tx=0x7f1bc0024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 -- 172.21.15.7:0/3886530487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 msgr2=0x5595ce361710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 0x5595ce361710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011990 tx=0x7f1bc0011e50 comp rx=0 tx=0).stop 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 -- 172.21.15.7:0/3886530487 shutdown_connections 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce3692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce361ce0 0x5595ce3667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3886530487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce3612f0 0x5595ce361710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 -- 172.21.15.7:0/3886530487 >> 172.21.15.7:0/3886530487 conn(0x5595ce35c860 msgr2=0x5595ce35cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 -- 172.21.15.7:0/3886530487 shutdown_connections 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 -- 172.21.15.7:0/3886530487 wait complete. 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 -- start start 2024-03-19T00:42:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.149+0000 7f1bd1019740 1 Processor -- start 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce36b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce36bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce372db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5595ce369e00 con 0x5595ce361ce0 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595ce281310 con 0x5595ce366e50 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595ce369f80 con 0x5595ce3612f0 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce36bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 10 client.0 _ll_register_callbacks cb 0x5595ce2752d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce36bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35118/0 (socket says 172.21.15.7:35118) 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce36b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 -- 172.21.15.7:0/3304538939 learned_addr learned my addr 172.21.15.7:0/3304538939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce372db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 -- 172.21.15.7:0/3304538939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 msgr2=0x5595ce36b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce36b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 -- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 msgr2=0x5595ce372db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce372db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 -- 172.21.15.7:0/3304538939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cdfd93f0 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce36bd70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0b9fa0 tx=0x7f1bbc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ade0fdfc665492 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1ba77fe640 1 -- 172.21.15.7:0/3304538939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0bcf30 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1ba77fe640 1 -- 172.21.15.7:0/3304538939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1bbc0053b0 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc001cc30 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1ba77fe640 1 -- 172.21.15.7:0/3304538939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bbc0bdcd0 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 10 client.16386 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.150+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595ce0a37d0 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1ba77fe640 1 -- 172.21.15.7:0/3304538939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bc001cc30 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1ba77fe640 1 -- 172.21.15.7:0/3304538939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1bbc0bd830 con 0x5595ce361ce0 2024-03-19T00:42:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bd1019740 10 client.16386 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bd1019740 10 client.16386 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bd1019740 20 client.16386 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3776b0 0x5595ce379af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bd1019740 4 client.16386 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595ce3233e0 con 0x5595ce3776b0 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3776b0 0x5595ce379af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1ba77fe640 10 client.16386 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.151+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3776b0 0x5595ce379af0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a64309325d17f62f server_cookie=64389d1ef0c5f702 in_seq=0 out_seq=0 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1ba77fe640 1 -- 172.21.15.7:0/3304538939 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5595ce3233e0 con 0x5595ce3776b0 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1ba77fe640 10 client.16386 handle_command_reply: tid=0 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 client.16386 shutdown 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 msgr2=0x5595ce36bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce36bd70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0b9fa0 tx=0x7f1bbc004600 comp rx=0 tx=0).stop 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 shutdown_connections 2024-03-19T00:42:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3776b0 0x5595ce379af0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce366e50 0x5595ce372db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce36bd70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1ba77fe640 0 client.16386 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 --2- 172.21.15.7:0/3304538939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce36b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 >> 172.21.15.7:0/3304538939 conn(0x5595ce35c860 msgr2=0x5595ce35bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 shutdown_connections 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 1 -- 172.21.15.7:0/3304538939 wait complete. 2024-03-19T00:42:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.152+0000 7f1bd1019740 20 client.16386 trim_cache size 0 max 16384 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 Processor -- start 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 -- start start 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 0x5595ce36ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce372d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce366e50 0x5595ce36b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81ab9a0 con 0x5595ce361ce0 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81a9270 con 0x5595ce3612f0 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bd1019740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81a70c0 con 0x5595ce366e50 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce372d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 0x5595ce36ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce372d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35124/0 (socket says 172.21.15.7:35124) 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1426542633 learned_addr learned my addr 172.21.15.7:0/1426542633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 0x5595ce36ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36794/0 (socket says 172.21.15.7:36794) 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce366e50 0x5595ce36b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1426542633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce366e50 msgr2=0x5595ce36b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce366e50 0x5595ce36b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1426542633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 msgr2=0x5595ce36ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 0x5595ce36ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 -- 172.21.15.7:0/1426542633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ce37a170 con 0x5595ce361ce0 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 0x5595ce36ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.166+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce372d70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0bb880 tx=0x7f1bbc0bbae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5bb5911e6f7692 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1badffb640 1 -- 172.21.15.7:0/1426542633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0bcf30 con 0x5595ce361ce0 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1badffb640 1 -- 172.21.15.7:0/1426542633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bbc0c5b20 con 0x5595ce361ce0 2024-03-19T00:42:16.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1badffb640 1 -- 172.21.15.7:0/1426542633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0c58e0 con 0x5595ce361ce0 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 -- 172.21.15.7:0/1426542633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 msgr2=0x5595ce372d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce372d70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0bb880 tx=0x7f1bbc0bbae0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 -- 172.21.15.7:0/1426542633 shutdown_connections 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce366e50 0x5595ce36b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce361ce0 0x5595ce372d70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1426542633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce3612f0 0x5595ce36ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 -- 172.21.15.7:0/1426542633 >> 172.21.15.7:0/1426542633 conn(0x5595ce35c860 msgr2=0x5595ce35cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 -- 172.21.15.7:0/1426542633 shutdown_connections 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 -- 172.21.15.7:0/1426542633 wait complete. 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.167+0000 7f1bd1019740 1 -- start start 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 Processor -- start 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce370a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce361ce0 0x5595ce370f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 0x5595ce371f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81a7730 con 0x5595ce366e50 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81a8ba0 con 0x5595ce361ce0 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81a7e00 con 0x5595ce3612f0 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bd1019740 10 client.0 _ll_register_callbacks cb 0x7f1bbc005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 0x5595ce371f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 0x5595ce371f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35128/0 (socket says 172.21.15.7:35128) 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/1661405825 learned_addr learned my addr 172.21.15.7:0/1661405825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bc77fe640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce361ce0 0x5595ce370f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce370a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/1661405825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 msgr2=0x5595ce370a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce370a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce361ce0 msgr2=0x5595ce370f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce361ce0 0x5595ce370f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 -- 172.21.15.7:0/1661405825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ce332880 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1bcc9d1640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 0x5595ce371f50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011330 tx=0x7f1bc00112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ad4d33409b6736 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.168+0000 7f1ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1ba77fe640 1 -- 172.21.15.7:0/1661405825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0004a00 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1ba77fe640 1 -- 172.21.15.7:0/1661405825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1bc0004ba0 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bbc0c5b20 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1ba77fe640 1 -- 172.21.15.7:0/1661405825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1bc0017020 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 10 client.16392 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595ce3dddd0 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1ba77fe640 1 -- 172.21.15.7:0/1661405825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bbc0c5b20 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1ba77fe640 1 -- 172.21.15.7:0/1661405825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1bc0005050 con 0x5595ce366e50 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 10 client.16392 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 10 client.16392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 20 client.16392 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3de650 0x5595ce3e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 4 client.16392 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595ce2d29f0 con 0x5595ce3de650 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.169+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3de650 0x5595ce3e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1ba77fe640 10 client.16392 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bc7fff640 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3de650 0x5595ce3e0a70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a56f3df53edefeb4 server_cookie=ec3c797ebbaa9e85 in_seq=0 out_seq=0 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1ba77fe640 1 -- 172.21.15.7:0/1661405825 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595ce2d29f0 con 0x5595ce3de650 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1ba77fe640 10 client.16392 handle_command_reply: tid=0 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 client.16392 shutdown 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 msgr2=0x5595ce371f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 0x5595ce371f50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011330 tx=0x7f1bc00112c0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 shutdown_connections 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5595ce3de650 0x5595ce3e0a70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595ce366e50 0x5595ce371f50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595ce361ce0 0x5595ce370f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1ba77fe640 0 client.16392 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 --2- 172.21.15.7:0/1661405825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ce3612f0 0x5595ce370a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 >> 172.21.15.7:0/1661405825 conn(0x5595ce35c860 msgr2=0x5595ce36ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 shutdown_connections 2024-03-19T00:42:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 1 -- 172.21.15.7:0/1661405825 wait complete. 2024-03-19T00:42:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.170+0000 7f1bd1019740 20 client.16392 trim_cache size 0 max 16384 2024-03-19T00:42:16.178 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bb00a2800 msgr2=0x7f1bb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bb00a2800 0x7f1bb00a4cc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0023000 tx=0x7f1bc0024000 comp rx=0 tx=0).stop 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 msgr2=0x7f1bc8196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8196eb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006010 tx=0x7f1bbc005e40 comp rx=0 tx=0).stop 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 shutdown_connections 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1bb00a2800 0x7f1bb00a4cc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bc81022f0 0x7f1bc819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bc8101930 0x7f1bc8196eb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 --2- 172.21.15.7:0/1586092800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bc8100740 0x7f1bc8196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 >> 172.21.15.7:0/1586092800 conn(0x7f1bc80fbc90 msgr2=0x7f1bc8104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 shutdown_connections 2024-03-19T00:42:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.171+0000 7f1bce434640 1 -- 172.21.15.7:0/1586092800 wait complete. 2024-03-19T00:42:16.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:42:16.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 -- 172.21.15.7:0/1530141081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c104c50 msgr2=0x7f519c107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 --2- 172.21.15.7:0/1530141081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c104c50 0x7f519c107040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5190009900 tx=0x7f519002bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:16.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 -- 172.21.15.7:0/1530141081 shutdown_connections 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 --2- 172.21.15.7:0/1530141081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f519c107580 0x7f519c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 --2- 172.21.15.7:0/1530141081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c104c50 0x7f519c107040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 --2- 172.21.15.7:0/1530141081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c102320 0x7f519c104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 -- 172.21.15.7:0/1530141081 >> 172.21.15.7:0/1530141081 conn(0x7f519c0fbc50 msgr2=0x7f519c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 -- 172.21.15.7:0/1530141081 shutdown_connections 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.288+0000 7f51a14f1640 1 -- 172.21.15.7:0/1530141081 wait complete. 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 Processor -- start 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 -- start start 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 0x7f519c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c104c50 0x7f519c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f519c107580 0x7f519c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d350 con 0x7f519c102320 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d1d0 con 0x7f519c104c50 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f51a14f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d4d0 con 0x7f519c107580 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f519affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 0x7f519c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f519affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 0x7f519c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35158/0 (socket says 172.21.15.7:35158) 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.289+0000 7f519affd640 1 -- 172.21.15.7:0/3545055 learned_addr learned my addr 172.21.15.7:0/3545055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519a7fc640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c104c50 0x7f519c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519b7fe640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f519c107580 0x7f519c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519affd640 1 -- 172.21.15.7:0/3545055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f519c107580 msgr2=0x7f519c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519affd640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f519c107580 0x7f519c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519affd640 1 -- 172.21.15.7:0/3545055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c104c50 msgr2=0x7f519c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519affd640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c104c50 0x7f519c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519affd640 1 -- 172.21.15.7:0/3545055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c19f670 con 0x7f519c102320 2024-03-19T00:42:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f519affd640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 0x7f519c197a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9f0 tx=0x7f519400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f5183fff640 1 -- 172.21.15.7:0/3545055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519400cd80 con 0x7f519c102320 2024-03-19T00:42:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f5183fff640 1 -- 172.21.15.7:0/3545055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51940108e0 con 0x7f519c102320 2024-03-19T00:42:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c19f960 con 0x7f519c102320 2024-03-19T00:42:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f5183fff640 1 -- 172.21.15.7:0/3545055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194018920 con 0x7f519c102320 2024-03-19T00:42:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.290+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c19fed0 con 0x7f519c102320 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.292+0000 7f5183fff640 1 -- 172.21.15.7:0/3545055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5194010a90 con 0x7f519c102320 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.292+0000 7f51a41bb740 1 Processor -- start 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 -- start start 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d09710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d0e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5586c3c6d4b0 con 0x5586c3d09ce0 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5586c3c15d70 con 0x5586c3d0ee50 2024-03-19T00:42:16.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f51a41bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586c3c69550 con 0x5586c3d092f0 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d0e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d09710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d0e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35168/0 (socket says 172.21.15.7:35168) 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519affd640 1 -- 172.21.15.7:0/983452121 learned_addr learned my addr 172.21.15.7:0/983452121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519a7fc640 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f5183fff640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51840a2880 0x7f51840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519b7fe640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51840a2880 0x7f51840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519a7fc640 1 -- 172.21.15.7:0/983452121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 msgr2=0x5586c3d09710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519a7fc640 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d09710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519a7fc640 1 -- 172.21.15.7:0/983452121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 msgr2=0x5586c3d0e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f5183fff640 1 -- 172.21.15.7:0/3545055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f51940cf050 con 0x7f519c102320 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519a7fc640 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d0e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519a7fc640 1 -- 172.21.15.7:0/983452121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c3c38210 con 0x5586c3d0ee50 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519affd640 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d0e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.293+0000 7f519b7fe640 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d09710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f519a7fc640 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d112e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5190036040 tx=0x7f5190009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1274493fd7194115 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f519b7fe640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51840a2880 0x7f51840a4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5586c3d0a540 tx=0x7f518c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f51817fa640 1 -- 172.21.15.7:0/983452121 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190045020 con 0x5586c3d0ee50 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f51817fa640 1 -- 172.21.15.7:0/983452121 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5190004040 con 0x5586c3d0ee50 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f51817fa640 1 -- 172.21.15.7:0/983452121 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51900046c0 con 0x5586c3d0ee50 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f51a41bb740 1 -- 172.21.15.7:0/983452121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 msgr2=0x5586c3d112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.294+0000 7f51a41bb740 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d112e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5190036040 tx=0x7f5190009560 comp rx=0 tx=0).stop 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- 172.21.15.7:0/983452121 shutdown_connections 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d112e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d0e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 --2- 172.21.15.7:0/983452121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d09710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- 172.21.15.7:0/983452121 >> 172.21.15.7:0/983452121 conn(0x5586c3d04860 msgr2=0x5586c3d04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- 172.21.15.7:0/983452121 shutdown_connections 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- 172.21.15.7:0/983452121 wait complete. 2024-03-19T00:42:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- start start 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 Processor -- start 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d092f0 0x5586c3d13750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d13c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 0x5586c3d1acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5586c3d11da0 con 0x5586c3d0ee50 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5586c3c29310 con 0x5586c3d092f0 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586c3d11f20 con 0x5586c3d09ce0 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f51a41bb740 10 client.0 _ll_register_callbacks cb 0x5586c3c1d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 0x5586c3d1acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 0x5586c3d1acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35172/0 (socket says 172.21.15.7:35172) 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519b7fe640 1 -- 172.21.15.7:0/1994420929 learned_addr learned my addr 172.21.15.7:0/1994420929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519affd640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d092f0 0x5586c3d13750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519a7fc640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d13c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519b7fe640 1 -- 172.21.15.7:0/1994420929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 msgr2=0x5586c3d13c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519b7fe640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d13c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.295+0000 7f519b7fe640 1 -- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d092f0 msgr2=0x5586c3d13750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f519b7fe640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d092f0 0x5586c3d13750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f519b7fe640 1 -- 172.21.15.7:0/1994420929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c39813f0 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f519b7fe640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 0x5586c3d1acd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f518c00fcb0 tx=0x7f518c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34c428eba1cdd62 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f517eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f517eff5640 1 -- 172.21.15.7:0/1994420929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c025070 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f517eff5640 1 -- 172.21.15.7:0/1994420929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f518c0047d0 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5190004040 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f517eff5640 1 -- 172.21.15.7:0/1994420929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f518c031050 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 10 client.16404 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586c3a4b7d0 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f517eff5640 1 -- 172.21.15.7:0/1994420929 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5190004040 con 0x5586c3d0ee50 2024-03-19T00:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f517eff5640 1 -- 172.21.15.7:0/1994420929 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f518c01f040 con 0x5586c3d0ee50 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 10 client.16404 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 20 client.16404 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d1f670 0x5586c3d21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 4 client.16404 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.296+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586c3ccb3e0 con 0x5586c3d1f670 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.297+0000 7f519affd640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d1f670 0x5586c3d21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.297+0000 7f517eff5640 10 client.16404 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.297+0000 7f519affd640 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d1f670 0x5586c3d21b10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8beddf612c4dc65 server_cookie=c2d8916d250828dc in_seq=0 out_seq=0 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f517eff5640 1 -- 172.21.15.7:0/1994420929 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5586c3ccb3e0 con 0x5586c3d1f670 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f517eff5640 10 client.16404 handle_command_reply: tid=0 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 client.16404 shutdown 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 msgr2=0x5586c3d1acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 0x5586c3d1acd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f518c00fcb0 tx=0x7f518c004a40 comp rx=0 tx=0).stop 2024-03-19T00:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 shutdown_connections 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d1f670 0x5586c3d21b10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d0ee50 0x5586c3d1acd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d13c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f517eff5640 0 client.16404 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1994420929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d092f0 0x5586c3d13750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 >> 172.21.15.7:0/1994420929 conn(0x5586c3d04860 msgr2=0x5586c3d03ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 shutdown_connections 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 1 -- 172.21.15.7:0/1994420929 wait complete. 2024-03-19T00:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.298+0000 7f51a41bb740 20 client.16404 trim_cache size 0 max 16384 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.311+0000 7f51a41bb740 1 Processor -- start 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.311+0000 7f51a41bb740 1 -- start start 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.311+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d03310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.311+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d1ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d03850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f51a41bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1acb50 con 0x5586c3d09ce0 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f51a41bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1aa420 con 0x5586c3d0ee50 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f51a41bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1a8270 con 0x5586c3d092f0 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d1ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d1ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35174/0 (socket says 172.21.15.7:35174) 2024-03-19T00:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 -- 172.21.15.7:0/2921276206 learned_addr learned my addr 172.21.15.7:0/2921276206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519affd640 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d03310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519b7fe640 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d03850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 -- 172.21.15.7:0/2921276206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 msgr2=0x5586c3d03310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d03310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 -- 172.21.15.7:0/2921276206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 msgr2=0x5586c3d03850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d03850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 -- 172.21.15.7:0/2921276206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c3d0ba70 con 0x5586c3d09ce0 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519affd640 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d03310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f519a7fc640 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d1ac90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5190005c40 tx=0x7f519002ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba8904fe0945dcd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f5180ff9640 1 -- 172.21.15.7:0/2921276206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190045070 con 0x5586c3d09ce0 2024-03-19T00:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f5180ff9640 1 -- 172.21.15.7:0/2921276206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f519002cc60 con 0x5586c3d09ce0 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.312+0000 7f5180ff9640 1 -- 172.21.15.7:0/2921276206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190004920 con 0x5586c3d09ce0 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 -- 172.21.15.7:0/2921276206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 msgr2=0x5586c3d1ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d1ac90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5190005c40 tx=0x7f519002ca70 comp rx=0 tx=0).stop 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 -- 172.21.15.7:0/2921276206 shutdown_connections 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d03850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d09ce0 0x5586c3d1ac90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 --2- 172.21.15.7:0/2921276206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d092f0 0x5586c3d03310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 -- 172.21.15.7:0/2921276206 >> 172.21.15.7:0/2921276206 conn(0x5586c3d04860 msgr2=0x5586c3d04c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 -- 172.21.15.7:0/2921276206 shutdown_connections 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 -- 172.21.15.7:0/2921276206 wait complete. 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 -- start start 2024-03-19T00:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.313+0000 7f51a41bb740 1 Processor -- start 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 0x5586c3d1a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d1aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d18bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1a88e0 con 0x5586c3d092f0 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1a9d50 con 0x5586c3d0ee50 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1a8fb0 con 0x5586c3d09ce0 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 10 client.0 _ll_register_callbacks cb 0x7f518c0136d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 0x5586c3d1a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 0x5586c3d1a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35184/0 (socket says 172.21.15.7:35184) 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 -- 172.21.15.7:0/1137697409 learned_addr learned my addr 172.21.15.7:0/1137697409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519a7fc640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d1aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519b7fe640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d18bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 -- 172.21.15.7:0/1137697409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 msgr2=0x5586c3d1aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d1aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 -- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 msgr2=0x5586c3d18bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d18bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 -- 172.21.15.7:0/1137697409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c3d192f0 con 0x5586c3d092f0 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f519affd640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 0x5586c3d1a5a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f519400c6a0 tx=0x7f51940d1530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a81aa24e38d70b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f517eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f517eff5640 1 -- 172.21.15.7:0/1137697409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519409f7e0 con 0x5586c3d092f0 2024-03-19T00:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f517eff5640 1 -- 172.21.15.7:0/1137697409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f51940cc900 con 0x5586c3d092f0 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519002cc60 con 0x5586c3d092f0 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f517eff5640 1 -- 172.21.15.7:0/1137697409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51940a3e30 con 0x5586c3d092f0 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 10 client.16410 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.314+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586c3d86500 con 0x5586c3d092f0 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f517eff5640 1 -- 172.21.15.7:0/1137697409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f519002cc60 con 0x5586c3d092f0 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f517eff5640 1 -- 172.21.15.7:0/1137697409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f51940d10b0 con 0x5586c3d092f0 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f51a41bb740 10 client.16410 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f51a41bb740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f51a41bb740 20 client.16410 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d86e00 0x5586c3d89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f51a41bb740 4 client.16410 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586c3c7a9f0 con 0x5586c3d86e00 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f519a7fc640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d86e00 0x5586c3d89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f519a7fc640 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d86e00 0x5586c3d89220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e618a4e91f86a482 server_cookie=7703e772669240ec in_seq=0 out_seq=0 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.315+0000 7f517eff5640 10 client.16410 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f517eff5640 1 -- 172.21.15.7:0/1137697409 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586c3c7a9f0 con 0x5586c3d86e00 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f517eff5640 10 client.16410 handle_command_reply: tid=0 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 client.16410 shutdown 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 msgr2=0x5586c3d1a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 0x5586c3d1a5a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f519400c6a0 tx=0x7f51940d1530 comp rx=0 tx=0).stop 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 shutdown_connections 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586c3d0ee50 0x5586c3d18bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586c3d09ce0 0x5586c3d1aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5586c3d86e00 0x5586c3d89220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 --2- 172.21.15.7:0/1137697409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586c3d092f0 0x5586c3d1a5a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f517eff5640 0 client.16410 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 >> 172.21.15.7:0/1137697409 conn(0x5586c3d04860 msgr2=0x5586c3d03310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 shutdown_connections 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 1 -- 172.21.15.7:0/1137697409 wait complete. 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a41bb740 20 client.16410 trim_cache size 0 max 16384 2024-03-19T00:42:16.326 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51840a2880 msgr2=0x7f51840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51840a2880 0x7f51840a4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5586c3d0a540 tx=0x7f518c004ab0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 msgr2=0x7f519c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 0x7f519c197a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9f0 tx=0x7f519400eeb0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 shutdown_connections 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f51840a2880 0x7f51840a4d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f519c107580 0x7f519c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c104c50 0x7f519c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 --2- 172.21.15.7:0/3545055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c102320 0x7f519c197a50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.316+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 >> 172.21.15.7:0/3545055 conn(0x7f519c0fbc50 msgr2=0x7f519c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.317+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 shutdown_connections 2024-03-19T00:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:16.317+0000 7f51a14f1640 1 -- 172.21.15.7:0/3545055 wait complete. 2024-03-19T00:42:16.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:16 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1948464020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:42:17.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:16 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1948464020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:42:17.105 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:16 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1948464020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:42:17.330 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:42:17.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:42:17.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:42:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 -- 172.21.15.7:0/2214127238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 msgr2=0x7f7eac101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/2214127238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac101c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009900 tx=0x7f7e9c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 -- 172.21.15.7:0/2214127238 shutdown_connections 2024-03-19T00:42:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/2214127238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eac1033e0 0x7f7eac109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/2214127238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac102a20 0x7f7eac102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/2214127238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac101c50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 -- 172.21.15.7:0/2214127238 >> 172.21.15.7:0/2214127238 conn(0x7f7eac0fcda0 msgr2=0x7f7eac0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 -- 172.21.15.7:0/2214127238 shutdown_connections 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.448+0000 7f7eb1d09640 1 -- 172.21.15.7:0/2214127238 wait complete. 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 Processor -- start 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 -- start start 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eac102a20 0x7f7eac197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac1033e0 0x7f7eac19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac10d330 con 0x7f7eac101830 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac10d1b0 con 0x7f7eac1033e0 2024-03-19T00:42:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eb1d09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac10d4b0 con 0x7f7eac102a20 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35214/0 (socket says 172.21.15.7:35214) 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 -- 172.21.15.7:0/1681421823 learned_addr learned my addr 172.21.15.7:0/1681421823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eabfff640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac1033e0 0x7f7eac19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eac102a20 0x7f7eac197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 -- 172.21.15.7:0/1681421823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eac102a20 msgr2=0x7f7eac197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eac102a20 0x7f7eac197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 -- 172.21.15.7:0/1681421823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac1033e0 msgr2=0x7f7eac19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac1033e0 0x7f7eac19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.449+0000 7f7eab7fe640 1 -- 172.21.15.7:0/1681421823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac19f680 con 0x7f7eac101830 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.450+0000 7f7eab7fe640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac197a60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c000c00 tx=0x7f7e9c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.450+0000 7f7ea8ff9640 1 -- 172.21.15.7:0/1681421823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c043020 con 0x7f7eac101830 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.450+0000 7f7ea8ff9640 1 -- 172.21.15.7:0/1681421823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e9c004350 con 0x7f7eac101830 2024-03-19T00:42:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.450+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac19f910 con 0x7f7eac101830 2024-03-19T00:42:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.450+0000 7f7ea8ff9640 1 -- 172.21.15.7:0/1681421823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c003b80 con 0x7f7eac101830 2024-03-19T00:42:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.450+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac19fe60 con 0x7f7eac101830 2024-03-19T00:42:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.452+0000 7f7ea8ff9640 1 -- 172.21.15.7:0/1681421823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7e9c046020 con 0x7f7eac101830 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 Processor -- start 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 -- start start 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7ea8ff9640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7e940a2800 0x7f7e940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb27890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb2a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56496ca87160 con 0x56496cb223a0 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56496ca2fe10 con 0x56496cb27f00 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56496ca83200 con 0x56496cb22d90 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eab7fe640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7e940a2800 0x7f7e940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb227c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35230/0 (socket says 172.21.15.7:35230) 2024-03-19T00:42:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 -- 172.21.15.7:0/3940167316 learned_addr learned my addr 172.21.15.7:0/3940167316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb27890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb2a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7ea8ff9640 1 -- 172.21.15.7:0/1681421823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7e9c0f0f40 con 0x7f7eac101830 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 -- 172.21.15.7:0/3940167316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 msgr2=0x56496cb27890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb27890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 -- 172.21.15.7:0/3940167316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 msgr2=0x56496cb2a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb2a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eabfff640 1 -- 172.21.15.7:0/3940167316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56496c8637e0 con 0x56496cb223a0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.453+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb27890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eab7fe640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7e940a2800 0x7f7e940a4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0bd4b0 tx=0x7f7e9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eabfff640 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb227c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4013b60 tx=0x7f7ea4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e4259efd3885e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7e91ffb640 1 -- 172.21.15.7:0/3940167316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4013130 con 0x56496cb223a0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7e91ffb640 1 -- 172.21.15.7:0/3940167316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea4013c50 con 0x56496cb223a0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7e91ffb640 1 -- 172.21.15.7:0/3940167316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4003c90 con 0x56496cb223a0 2024-03-19T00:42:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 -- 172.21.15.7:0/3940167316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 msgr2=0x56496cb227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb227c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4013b60 tx=0x7f7ea4009b30 comp rx=0 tx=0).stop 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 -- 172.21.15.7:0/3940167316 shutdown_connections 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb27890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/3940167316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb227c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 -- 172.21.15.7:0/3940167316 >> 172.21.15.7:0/3940167316 conn(0x56496cb1d910 msgr2=0x56496cb1dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 -- 172.21.15.7:0/3940167316 shutdown_connections 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.454+0000 7f7eb49d3740 1 -- 172.21.15.7:0/3940167316 wait complete. 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 -- start start 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 Processor -- start 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb22d90 0x56496cb2cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb27f00 0x56496cb33d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56496cb2ae50 con 0x56496cb223a0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56496ca431a0 con 0x56496cb22d90 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56496cb2afd0 con 0x56496cb27f00 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eb49d3740 10 client.0 _ll_register_callbacks cb 0x56496c79ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35242/0 (socket says 172.21.15.7:35242) 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eabfff640 1 -- 172.21.15.7:0/2140742180 learned_addr learned my addr 172.21.15.7:0/2140742180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.455+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb22d90 0x56496cb2cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb27f00 0x56496cb33d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 -- 172.21.15.7:0/2140742180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb27f00 msgr2=0x56496cb33d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb27f00 0x56496cb33d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 -- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb22d90 msgr2=0x56496cb2cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb22d90 0x56496cb2cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 -- 172.21.15.7:0/2140742180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496c8c3f70 con 0x56496cb223a0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eabfff640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2c7c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400fc90 tx=0x7f7ea4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810c9532bdfe4803 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7e8affd640 1 -- 172.21.15.7:0/2140742180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4010430 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7e8affd640 1 -- 172.21.15.7:0/2140742180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7ea40105d0 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea4013c50 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7e8affd640 1 -- 172.21.15.7:0/2140742180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ea4032050 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eb49d3740 10 client.16422 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56496c75b170 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.456+0000 7f7e8affd640 1 -- 172.21.15.7:0/2140742180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea4013c50 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7e8affd640 1 -- 172.21.15.7:0/2140742180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7ea401f700 con 0x56496cb223a0 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eb49d3740 10 client.16422 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eb49d3740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eb49d3740 20 client.16422 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb38400 0x56496cb3a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eb49d3740 4 client.16422 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56496ca946a0 con 0x56496cb38400 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb38400 0x56496cb3a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb38400 0x56496cb3a810 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12fb66d5a24b4f56 server_cookie=3062840fe60f118e in_seq=0 out_seq=0 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.457+0000 7f7e8affd640 10 client.16422 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7e8affd640 1 -- 172.21.15.7:0/2140742180 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56496ca946a0 con 0x56496cb38400 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7e8affd640 10 client.16422 handle_command_reply: tid=0 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 client.16422 shutdown 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 msgr2=0x56496cb2c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2c7c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400fc90 tx=0x7f7ea4005fb0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 shutdown_connections 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb27f00 0x56496cb33d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb22d90 0x56496cb2cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb38400 0x56496cb3a810 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/2140742180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2c7c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 >> 172.21.15.7:0/2140742180 conn(0x56496cb1d910 msgr2=0x56496cb1cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7e8affd640 0 client.16422 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 shutdown_connections 2024-03-19T00:42:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 1 -- 172.21.15.7:0/2140742180 wait complete. 2024-03-19T00:42:17.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.458+0000 7f7eb49d3740 20 client.16422 trim_cache size 0 max 16384 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 Processor -- start 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 -- start start 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb33cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb21280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1acb00 con 0x56496cb223a0 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1aa3d0 con 0x56496cb27f00 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1a8220 con 0x56496cb22d90 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35244/0 (socket says 172.21.15.7:35244) 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eabfff640 1 -- 172.21.15.7:0/213340904 learned_addr learned my addr 172.21.15.7:0/213340904 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb33cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.476+0000 7f7eabfff640 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb21280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eabfff640 1 -- 172.21.15.7:0/213340904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 msgr2=0x56496cb33cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eabfff640 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb33cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eabfff640 1 -- 172.21.15.7:0/213340904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 msgr2=0x56496cb21280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eabfff640 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb21280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eabfff640 1 -- 172.21.15.7:0/213340904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56496cb3ae90 con 0x56496cb223a0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb33cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eabfff640 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2cc60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4000f80 tx=0x7f7ea4005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6a37f7715b3636 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7e917fa640 1 -- 172.21.15.7:0/213340904 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4009e60 con 0x56496cb223a0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7e917fa640 1 -- 172.21.15.7:0/213340904 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea4029070 con 0x56496cb223a0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7e917fa640 1 -- 172.21.15.7:0/213340904 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4010ab0 con 0x56496cb223a0 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eb49d3740 1 -- 172.21.15.7:0/213340904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 msgr2=0x56496cb2cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2cc60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4000f80 tx=0x7f7ea4005cf0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.477+0000 7f7eb49d3740 1 -- 172.21.15.7:0/213340904 shutdown_connections 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb21280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb33cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/213340904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb2cc60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- 172.21.15.7:0/213340904 >> 172.21.15.7:0/213340904 conn(0x56496cb1d910 msgr2=0x56496cb20b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- 172.21.15.7:0/213340904 shutdown_connections 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- 172.21.15.7:0/213340904 wait complete. 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- start start 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 Processor -- start 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb31640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb31b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb32ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1a8890 con 0x56496cb223a0 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1a9d00 con 0x56496cb27f00 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1a8f60 con 0x56496cb22d90 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eb49d3740 10 client.0 _ll_register_callbacks cb 0x7f7ea4013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb31640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb31640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35250/0 (socket says 172.21.15.7:35250) 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.478+0000 7f7eabfff640 1 -- 172.21.15.7:0/1463360781 learned_addr learned my addr 172.21.15.7:0/1463360781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb32ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 -- 172.21.15.7:0/1463360781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 msgr2=0x56496cb31b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb31b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 -- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 msgr2=0x56496cb32ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb32ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 -- 172.21.15.7:0/1463360781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496cb330a0 con 0x56496cb223a0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eabfff640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb31640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7ea401fb70 tx=0x7f7ea4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c155f0dc5d1b8a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7e8affd640 1 -- 172.21.15.7:0/1463360781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea401f400 con 0x56496cb223a0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7e8affd640 1 -- 172.21.15.7:0/1463360781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7ea403b040 con 0x56496cb223a0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea4029070 con 0x56496cb223a0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7e8affd640 1 -- 172.21.15.7:0/1463360781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ea401f5a0 con 0x56496cb223a0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eb49d3740 10 client.16428 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56496cb9e810 con 0x56496cb223a0 2024-03-19T00:42:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.479+0000 7f7e8affd640 1 -- 172.21.15.7:0/1463360781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea4029070 con 0x56496cb223a0 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7e8affd640 1 -- 172.21.15.7:0/1463360781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7ea403e070 con 0x56496cb223a0 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eb49d3740 10 client.16428 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eb49d3740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eb49d3740 20 client.16428 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb9f190 0x56496cba15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eb49d3740 4 client.16428 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56496ca8e9f0 con 0x56496cb9f190 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb9f190 0x56496cba15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7eaaffd640 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb9f190 0x56496cba15b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff2695e1ffddd12f server_cookie=a0f3afa039525689 in_seq=0 out_seq=0 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.480+0000 7f7e8affd640 10 client.16428 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7e8affd640 1 -- 172.21.15.7:0/1463360781 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56496ca8e9f0 con 0x56496cb9f190 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7e8affd640 10 client.16428 handle_command_reply: tid=0 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 client.16428 shutdown 2024-03-19T00:42:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 msgr2=0x56496cb31640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb31640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7ea401fb70 tx=0x7f7ea4005fb0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 shutdown_connections 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56496cb27f00 0x56496cb32ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56496cb22d90 0x56496cb31b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56496cb9f190 0x56496cba15b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 --2- 172.21.15.7:0/1463360781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56496cb223a0 0x56496cb31640 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 >> 172.21.15.7:0/1463360781 conn(0x56496cb1d910 msgr2=0x56496cb21400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7e8affd640 0 client.16428 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 shutdown_connections 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.481+0000 7f7eb49d3740 1 -- 172.21.15.7:0/1463360781 wait complete. 2024-03-19T00:42:17.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb49d3740 20 client.16428 trim_cache size 0 max 16384 2024-03-19T00:42:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:42:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:42:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2f04ba38-bc09-43f8-8cda-51e1ca16a952", 2024-03-19T00:42:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:42:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:42:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7e940a2800 msgr2=0x7f7e940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7e940a2800 0x7f7e940a4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0bd4b0 tx=0x7f7e9c009260 comp rx=0 tx=0).stop 2024-03-19T00:42:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 msgr2=0x7f7eac197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac197a60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c000c00 tx=0x7f7e9c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 shutdown_connections 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7e940a2800 0x7f7e940a4cc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac1033e0 0x7f7eac19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7eac102a20 0x7f7eac197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 --2- 172.21.15.7:0/1681421823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac101830 0x7f7eac197a60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 >> 172.21.15.7:0/1681421823 conn(0x7f7eac0fcda0 msgr2=0x7f7eac105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 shutdown_connections 2024-03-19T00:42:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.482+0000 7f7eb1d09640 1 -- 172.21.15.7:0/1681421823 wait complete. 2024-03-19T00:42:17.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:42:17.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 -- 172.21.15.7:0/1839180176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e81033e0 msgr2=0x7f50e8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1839180176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e81033e0 0x7f50e8109c70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 -- 172.21.15.7:0/1839180176 shutdown_connections 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1839180176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e81033e0 0x7f50e8109c70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1839180176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e8102a20 0x7f50e8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1839180176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e8101830 0x7f50e8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 -- 172.21.15.7:0/1839180176 >> 172.21.15.7:0/1839180176 conn(0x7f50e80fcda0 msgr2=0x7f50e80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 -- 172.21.15.7:0/1839180176 shutdown_connections 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.598+0000 7f50efa0b640 1 -- 172.21.15.7:0/1839180176 wait complete. 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 Processor -- start 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 -- start start 2024-03-19T00:42:17.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e8101830 0x7f50e80755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 0x7f50e8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e81033e0 0x7f50e8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50e810d360 con 0x7f50e8102a20 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50e810d1e0 con 0x7f50e81033e0 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50e810d4e0 con 0x7f50e8101830 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 0x7f50e8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 0x7f50e8073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35274/0 (socket says 172.21.15.7:35274) 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 -- 172.21.15.7:0/1943783445 learned_addr learned my addr 172.21.15.7:0/1943783445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e8101830 0x7f50e80755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50edfa8640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e81033e0 0x7f50e8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 -- 172.21.15.7:0/1943783445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e8101830 msgr2=0x7f50e80755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e8101830 0x7f50e80755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 -- 172.21.15.7:0/1943783445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e81033e0 msgr2=0x7f50e8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e81033e0 0x7f50e8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 -- 172.21.15.7:0/1943783445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e80749e0 con 0x7f50e8102a20 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 0x7f50e8073c10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f50d800ea60 tx=0x7f50d800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50de7fc640 1 -- 172.21.15.7:0/1943783445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800cde0 con 0x7f50e8102a20 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e81a1b50 con 0x7f50e8102a20 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50de7fc640 1 -- 172.21.15.7:0/1943783445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d80035f0 con 0x7f50e8102a20 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.599+0000 7f50de7fc640 1 -- 172.21.15.7:0/1943783445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8010690 con 0x7f50e8102a20 2024-03-19T00:42:17.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.600+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e81a20f0 con 0x7f50e8102a20 2024-03-19T00:42:17.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50de7fc640 1 -- 172.21.15.7:0/1943783445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f50d80037a0 con 0x7f50e8102a20 2024-03-19T00:42:17.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 Processor -- start 2024-03-19T00:42:17.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 -- start start 2024-03-19T00:42:17.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555ee3423500 con 0x555ee34bea50 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555ee33cbbe0 con 0x555ee34c45b0 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50f25f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555ee341f5a0 con 0x555ee34bf440 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50de7fc640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50d40a2800 0x7f50d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.602+0000 7f50edfa8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50edfa8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34bee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35286/0 (socket says 172.21.15.7:35286) 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50edfa8640 1 -- 172.21.15.7:0/3882824688 learned_addr learned my addr 172.21.15.7:0/3882824688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50edfa8640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50d40a2800 0x7f50d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50de7fc640 1 -- 172.21.15.7:0/1943783445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50d80cf060 con 0x7f50e8102a20 2024-03-19T00:42:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 -- 172.21.15.7:0/3882824688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 msgr2=0x555ee34c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 -- 172.21.15.7:0/3882824688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 msgr2=0x555ee34bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34bee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 -- 172.21.15.7:0/3882824688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee33ed000 con 0x555ee34c45b0 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50edfa8640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50d40a2800 0x7f50d40a4cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x555ee34bfca0 tx=0x7f50e002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34c6a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50d8098070 tx=0x7f50d806e3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b7e803eee331621 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50d37fe640 1 -- 172.21.15.7:0/3882824688 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d809b470 con 0x555ee34c45b0 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50d37fe640 1 -- 172.21.15.7:0/3882824688 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d8003b30 con 0x555ee34c45b0 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.603+0000 7f50d37fe640 1 -- 172.21.15.7:0/3882824688 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d809b000 con 0x555ee34c45b0 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 -- 172.21.15.7:0/3882824688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 msgr2=0x555ee34c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34c6a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50d8098070 tx=0x7f50d806e3d0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 -- 172.21.15.7:0/3882824688 shutdown_connections 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34c6a40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 --2- 172.21.15.7:0/3882824688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 -- 172.21.15.7:0/3882824688 >> 172.21.15.7:0/3882824688 conn(0x555ee34b9fc0 msgr2=0x555ee34ba3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 -- 172.21.15.7:0/3882824688 shutdown_connections 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 -- 172.21.15.7:0/3882824688 wait complete. 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 -- start start 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.604+0000 7f50f25f3740 1 Processor -- start 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bea50 0x555ee34c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 0x555ee34d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555ee34c7560 con 0x555ee34c45b0 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555ee33df160 con 0x555ee34bea50 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555ee34c76e0 con 0x555ee34bf440 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50f25f3740 10 client.0 _ll_register_callbacks cb 0x555ee341ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50edfa8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 0x555ee34d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ed7a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bea50 0x555ee34c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50edfa8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 0x555ee34d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35300/0 (socket says 172.21.15.7:35300) 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50edfa8640 1 -- 172.21.15.7:0/672987481 learned_addr learned my addr 172.21.15.7:0/672987481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 -- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bea50 msgr2=0x555ee34c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bea50 0x555ee34c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 -- 172.21.15.7:0/672987481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 msgr2=0x555ee34d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 0x555ee34d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 -- 172.21.15.7:0/672987481 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee32017d0 con 0x555ee34bf440 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bea50 0x555ee34c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50edfa8640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 0x555ee34d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c9470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f50d80cc800 tx=0x7f50d809f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a9f43397171a125 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.605+0000 7f50d0ff9640 1 -- 172.21.15.7:0/672987481 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d809b1e0 con 0x555ee34bf440 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50d0ff9640 1 -- 172.21.15.7:0/672987481 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50d80b44f0 con 0x555ee34bf440 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ee3430a40 con 0x555ee34bf440 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50d0ff9640 1 -- 172.21.15.7:0/672987481 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50d809b470 con 0x555ee34bf440 2024-03-19T00:42:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 10 client.26200 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ee31373f0 con 0x555ee34bf440 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50d0ff9640 1 -- 172.21.15.7:0/672987481 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555ee3430a40 con 0x555ee34bf440 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50d0ff9640 1 -- 172.21.15.7:0/672987481 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f50d809b900 con 0x555ee34bf440 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 10 client.26200 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 20 client.26200 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee34d49a0 0x555ee34d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 4 client.26200 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ee342ad90 con 0x555ee34d49a0 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.606+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee34d49a0 0x555ee34d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.607+0000 7f50d0ff9640 10 client.26200 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.607+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee34d49a0 0x555ee34d6d90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c77ff4f8257d71c server_cookie=b06bcb782849fad7 in_seq=0 out_seq=0 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.607+0000 7f50d0ff9640 1 -- 172.21.15.7:0/672987481 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555ee342ad90 con 0x555ee34d49a0 2024-03-19T00:42:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.607+0000 7f50d0ff9640 10 client.26200 handle_command_reply: tid=0 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.607+0000 7f50f25f3740 1 client.26200 shutdown 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 msgr2=0x555ee34c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c9470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f50d80cc800 tx=0x7f50d809f7f0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 shutdown_connections 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee34d49a0 0x555ee34d6d90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34c45b0 0x555ee34d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50d0ff9640 0 client.26200 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34c9470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 --2- 172.21.15.7:0/672987481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bea50 0x555ee34c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 >> 172.21.15.7:0/672987481 conn(0x555ee34b9fc0 msgr2=0x555ee34b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 shutdown_connections 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 1 -- 172.21.15.7:0/672987481 wait complete. 2024-03-19T00:42:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.608+0000 7f50f25f3740 20 client.26200 trim_cache size 0 max 16384 2024-03-19T00:42:17.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 Processor -- start 2024-03-19T00:42:17.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 -- start start 2024-03-19T00:42:17.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81aecd0 con 0x555ee34bea50 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81ac5a0 con 0x555ee34c45b0 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.625+0000 7f50f25f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81aa3f0 con 0x555ee34bf440 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35312/0 (socket says 172.21.15.7:35312) 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 -- 172.21.15.7:0/1794768680 learned_addr learned my addr 172.21.15.7:0/1794768680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50edfa8640 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 -- 172.21.15.7:0/1794768680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 msgr2=0x555ee34d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 -- 172.21.15.7:0/1794768680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 msgr2=0x555ee34b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 -- 172.21.15.7:0/1794768680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee34d7100 con 0x555ee34bea50 2024-03-19T00:42:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50edfa8640 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34ba3a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f50e40119e0 tx=0x7f50e4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdfec6724b4b142 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50d2ffd640 1 -- 172.21.15.7:0/1794768680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400fe80 con 0x555ee34bea50 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50d2ffd640 1 -- 172.21.15.7:0/1794768680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e4003970 con 0x555ee34bea50 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.626+0000 7f50d2ffd640 1 -- 172.21.15.7:0/1794768680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4020740 con 0x555ee34bea50 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 -- 172.21.15.7:0/1794768680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 msgr2=0x555ee34ba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34ba3a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f50e40119e0 tx=0x7f50e4009500 comp rx=0 tx=0).stop 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 -- 172.21.15.7:0/1794768680 shutdown_connections 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34c45b0 0x555ee34b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34bf440 0x555ee34d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 --2- 172.21.15.7:0/1794768680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34ba3a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 -- 172.21.15.7:0/1794768680 >> 172.21.15.7:0/1794768680 conn(0x555ee34b9fc0 msgr2=0x555ee34b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 -- 172.21.15.7:0/1794768680 shutdown_connections 2024-03-19T00:42:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 -- 172.21.15.7:0/1794768680 wait complete. 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 -- start start 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.627+0000 7f50f25f3740 1 Processor -- start 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bf440 0x555ee34cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34c45b0 0x555ee34ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81aaa60 con 0x555ee34bea50 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81abed0 con 0x555ee34bf440 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81ab130 con 0x555ee34c45b0 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 10 client.0 _ll_register_callbacks cb 0x7f50e40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35328/0 (socket says 172.21.15.7:35328) 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 -- 172.21.15.7:0/393682198 learned_addr learned my addr 172.21.15.7:0/393682198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bf440 0x555ee34cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50edfa8640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34c45b0 0x555ee34ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 -- 172.21.15.7:0/393682198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34c45b0 msgr2=0x555ee34ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34c45b0 0x555ee34ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 -- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bf440 msgr2=0x555ee34cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bf440 0x555ee34cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 -- 172.21.15.7:0/393682198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee30f70f0 con 0x555ee34bea50 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50ed7a7640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34cd670 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f50e4011b10 tx=0x7f50e4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713a9eca3a3500c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50d0ff9640 1 -- 172.21.15.7:0/393682198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e402e020 con 0x555ee34bea50 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50d0ff9640 1 -- 172.21.15.7:0/393682198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f50e40204c0 con 0x555ee34bea50 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e4003970 con 0x555ee34bea50 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50d0ff9640 1 -- 172.21.15.7:0/393682198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50e4039050 con 0x555ee34bea50 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 10 client.16446 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:17.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.628+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ee34d02a0 con 0x555ee34bea50 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50d0ff9640 1 -- 172.21.15.7:0/393682198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e4003970 con 0x555ee34bea50 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50d0ff9640 1 -- 172.21.15.7:0/393682198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f50e4005490 con 0x555ee34bea50 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50f25f3740 10 client.16446 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50f25f3740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50f25f3740 20 client.16446 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50f25f3740 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee353aca0 0x555ee353d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50f25f3740 4 client.16446 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ee33a8540 con 0x555ee353aca0 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee353aca0 0x555ee353d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50d0ff9640 10 client.16446 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.629+0000 7f50ecfa6640 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee353aca0 0x555ee353d0c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a246957c06e85d7d server_cookie=f308cd0b417bf341 in_seq=0 out_seq=0 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50d0ff9640 1 -- 172.21.15.7:0/393682198 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555ee33a8540 con 0x555ee353aca0 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50d0ff9640 10 client.16446 handle_command_reply: tid=0 2024-03-19T00:42:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 client.16446 shutdown 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 msgr2=0x555ee34cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34cd670 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f50e4011b10 tx=0x7f50e4011b40 comp rx=0 tx=0).stop 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 shutdown_connections 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ee34c45b0 0x555ee34ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee34bf440 0x555ee34cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x555ee353aca0 0x555ee353d0c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 --2- 172.21.15.7:0/393682198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee34bea50 0x555ee34cd670 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50d0ff9640 0 client.16446 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 >> 172.21.15.7:0/393682198 conn(0x555ee34b9fc0 msgr2=0x555ee34ba3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 shutdown_connections 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 1 -- 172.21.15.7:0/393682198 wait complete. 2024-03-19T00:42:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50f25f3740 20 client.16446 trim_cache size 0 max 16384 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "2f04ba38-bc09-43f8-8cda-51e1ca16a952": { 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2f04ba38-bc09-43f8-8cda-51e1ca16a952", 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:42:17.638 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:42:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50d40a2800 msgr2=0x7f50d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50d40a2800 0x7f50d40a4cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x555ee34bfca0 tx=0x7f50e002c7e0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 msgr2=0x7f50e8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 0x7f50e8073c10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f50d800ea60 tx=0x7f50d800ef20 comp rx=0 tx=0).stop 2024-03-19T00:42:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 shutdown_connections 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f50d40a2800 0x7f50d40a4cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e81033e0 0x7f50e8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e8102a20 0x7f50e8073c10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.630+0000 7f50efa0b640 1 --2- 172.21.15.7:0/1943783445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50e8101830 0x7f50e80755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.631+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 >> 172.21.15.7:0/1943783445 conn(0x7f50e80fcda0 msgr2=0x7f50e8105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.631+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 shutdown_connections 2024-03-19T00:42:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:17.631+0000 7f50efa0b640 1 -- 172.21.15.7:0/1943783445 wait complete. 2024-03-19T00:42:17.642 INFO:tasks.cephfs.filesystem:scrub status for tag:2f04ba38-bc09-43f8-8cda-51e1ca16a952 - {'path': '/', 'tag': '2f04ba38-bc09-43f8-8cda-51e1ca16a952', 'options': 'recursive,force'} 2024-03-19T00:42:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:17 smithi007 ceph-mon[62854]: pgmap v481: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:18.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:17 smithi204 ceph-mon[66572]: pgmap v481: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:17 smithi044 ceph-mon[66366]: pgmap v481: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:18 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:42:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:18 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:18 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:42:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:18 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:42:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:18 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:18.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:18 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:18 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:42:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:18 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:18 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:42:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:18 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:42:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:18 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:18 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:18 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:42:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:18 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:18 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:42:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:18 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:42:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:18 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:18 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:19.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:19 smithi007 ceph-mon[62854]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:20.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:19 smithi204 ceph-mon[66572]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:19 smithi044 ceph-mon[66366]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:127/600: open fd = 3 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:128/600: open fd = 3 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.340 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:129/600: open fd = 3 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:130/600: open fd = 3 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:131/600: open fd = 3 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.341 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:132/600: open fd = 3 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:133/600: open fd = 3 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:134/600: open fd = 3 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.342 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:135/600: open fd = 3 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:136/600: open fd = 3 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:137/600: open fd = 3 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:138/600: open fd = 3 2024-03-19T00:42:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:139/600: open fd = 3 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:140/600: open fd = 3 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:141/600: open fd = 3 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.344 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:142/600: open fd = 3 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:143/600: open fd = 3 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:144/600: open fd = 3 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:145/600: open fd = 3 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:146/600: open fd = 3 2024-03-19T00:42:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:147/600: open fd = 3 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:148/600: open fd = 3 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:149/600: open fd = 3 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.347 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:150/600: open fd = 3 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:151/600: open fd = 3 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:152/600: open fd = 3 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.348 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:153/600: open fd = 3 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:154/600: open fd = 3 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:155/600: open fd = 3 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:156/600: open fd = 3 2024-03-19T00:42:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:157/600: open fd = 3 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:158/600: open fd = 3 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:159/600: open fd = 3 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:160/600: open fd = 3 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:161/600: open fd = 3 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:162/600: open fd = 3 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.351 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:163/600: open fd = 3 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:164/600: open fd = 3 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:165/600: open fd = 3 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.352 INFO:tasks.workunit.client.0.smithi007.stdout:166/600: open fd = 3 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:167/600: open fd = 3 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:168/600: open fd = 3 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:169/600: open fd = 3 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.353 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:170/600: open fd = 3 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:171/600: open fd = 3 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:172/600: open fd = 3 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.354 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:173/600: open fd = 3 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:174/600: open fd = 3 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:175/600: open fd = 3 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.355 INFO:tasks.workunit.client.0.smithi007.stdout:176/600: open fd = 3 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:177/600: open fd = 3 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:178/600: open fd = 3 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:179/600: open fd = 3 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:180/600: open fd = 3 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:181/600: open fd = 3 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:182/600: open fd = 3 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.357 INFO:tasks.workunit.client.0.smithi007.stdout:183/600: open fd = 3 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:184/600: open fd = 3 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:185/600: open fd = 3 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:186/600: open fd = 3 2024-03-19T00:42:20.358 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:187/600: open fd = 3 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:188/600: open fd = 3 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:20.359 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:42:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:21 smithi007 ceph-mon[62854]: pgmap v483: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:21 smithi204 ceph-mon[66572]: pgmap v483: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:21 smithi044 ceph-mon[66366]: pgmap v483: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:23 smithi007 ceph-mon[62854]: pgmap v484: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:23 smithi204 ceph-mon[66572]: pgmap v484: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:23 smithi044 ceph-mon[66366]: pgmap v484: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:25.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:25 smithi007 ceph-mon[62854]: pgmap v485: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:26.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:25 smithi204 ceph-mon[66572]: pgmap v485: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:25 smithi044 ceph-mon[66366]: pgmap v485: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:26.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:26 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:42:26.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:26 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:42:27.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:26 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:42:27.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:26 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:42:27.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:26 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:42:27.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:26 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:42:27.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:27 smithi007 ceph-mon[62854]: pgmap v486: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:28.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:27 smithi204 ceph-mon[66572]: pgmap v486: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:27 smithi044 ceph-mon[66366]: pgmap v486: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:29 smithi007 ceph-mon[62854]: pgmap v487: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:29 smithi204 ceph-mon[66572]: pgmap v487: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:29 smithi044 ceph-mon[66366]: pgmap v487: 129 pgs: 129 active+clean; 1.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:31 smithi007 ceph-mon[62854]: pgmap v488: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:31 smithi204 ceph-mon[66572]: pgmap v488: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:31 smithi044 ceph-mon[66366]: pgmap v488: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:42:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:33 smithi007 ceph-mon[62854]: pgmap v489: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:33 smithi204 ceph-mon[66572]: pgmap v489: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:34.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:33 smithi044 ceph-mon[66366]: pgmap v489: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:42:35.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:35 smithi007 ceph-mon[62854]: pgmap v490: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:42:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:35 smithi204 ceph-mon[66572]: pgmap v490: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:42:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:35 smithi044 ceph-mon[66366]: pgmap v490: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:42:37.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:37 smithi007 ceph-mon[62854]: pgmap v491: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:38.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:37 smithi204 ceph-mon[66572]: pgmap v491: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:37 smithi044 ceph-mon[66366]: pgmap v491: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:40.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:39 smithi204 ceph-mon[66572]: pgmap v492: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:40.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:39 smithi044 ceph-mon[66366]: pgmap v492: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:40.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:39 smithi007 ceph-mon[62854]: pgmap v492: 129 pgs: 129 active+clean; 1.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:42.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:41 smithi204 ceph-mon[66572]: pgmap v493: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:42:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:41 smithi044 ceph-mon[66366]: pgmap v493: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:42:42.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:41 smithi007 ceph-mon[62854]: pgmap v493: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:42:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:43 smithi204 ceph-mon[66572]: pgmap v494: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:43 smithi044 ceph-mon[66366]: pgmap v494: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:44.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:43 smithi007 ceph-mon[62854]: pgmap v494: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:45.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:44 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:42:45.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:44 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:42:45.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:44 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:42:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:45 smithi204 ceph-mon[66572]: pgmap v495: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:42:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:42:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:42:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:42:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:45 smithi044 ceph-mon[66366]: pgmap v495: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:42:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:42:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:42:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:42:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:45 smithi007 ceph-mon[62854]: pgmap v495: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:42:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:42:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:42:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:42:47.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:42:47.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:42:47.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 -- 172.21.15.7:0/2472124627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 msgr2=0x7f4d30101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 --2- 172.21.15.7:0/2472124627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30101c70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009ca0 tx=0x7f4d2c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 -- 172.21.15.7:0/2472124627 shutdown_connections 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 --2- 172.21.15.7:0/2472124627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d30103400 0x7f4d30109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 --2- 172.21.15.7:0/2472124627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30102a40 0x7f4d30102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 --2- 172.21.15.7:0/2472124627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30101c70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 -- 172.21.15.7:0/2472124627 >> 172.21.15.7:0/2472124627 conn(0x7f4d300fcde0 msgr2=0x7f4d300ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 -- 172.21.15.7:0/2472124627 shutdown_connections 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.759+0000 7f4d38426640 1 -- 172.21.15.7:0/2472124627 wait complete. 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 Processor -- start 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 -- start start 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d30102a40 0x7f4d30197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30103400 0x7f4d3019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3010d380 con 0x7f4d30101850 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3010d200 con 0x7f4d30103400 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d38426640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3010d500 con 0x7f4d30102a40 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53106/0 (socket says 172.21.15.7:53106) 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 -- 172.21.15.7:0/4201619428 learned_addr learned my addr 172.21.15.7:0/4201619428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d369c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30103400 0x7f4d3019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d359c1640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d30102a40 0x7f4d30197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 -- 172.21.15.7:0/4201619428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d30102a40 msgr2=0x7f4d30197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d30102a40 0x7f4d30197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 -- 172.21.15.7:0/4201619428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30103400 msgr2=0x7f4d3019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30103400 0x7f4d3019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 -- 172.21.15.7:0/4201619428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3019f6a0 con 0x7f4d30101850 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d369c3640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30103400 0x7f4d3019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:47.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.760+0000 7f4d361c2640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30197a80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c000f80 tx=0x7f4d2c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.761+0000 7f4d277fe640 1 -- 172.21.15.7:0/4201619428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c041020 con 0x7f4d30101850 2024-03-19T00:42:47.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.761+0000 7f4d277fe640 1 -- 172.21.15.7:0/4201619428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2c034720 con 0x7f4d30101850 2024-03-19T00:42:47.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.761+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3019f930 con 0x7f4d30101850 2024-03-19T00:42:47.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.761+0000 7f4d277fe640 1 -- 172.21.15.7:0/4201619428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c039040 con 0x7f4d30101850 2024-03-19T00:42:47.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.761+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3019fe70 con 0x7f4d30101850 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d277fe640 1 -- 172.21.15.7:0/4201619428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f4d2c044020 con 0x7f4d30101850 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 Processor -- start 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 -- start start 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeaae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeab440 0x55ad4aeaff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aeb2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad4ae0f500 con 0x55ad4aeaaa50 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ad4adb7be0 con 0x55ad4aeb05b0 2024-03-19T00:42:47.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.764+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad4ae0b5a0 con 0x55ad4aeab440 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d369c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeaae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeab440 0x55ad4aeaff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aeb2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aeb2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53890/0 (socket says 172.21.15.7:53890) 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeab440 0x55ad4aeaff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:35454/0 (socket says 172.21.15.7:35454) 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d369c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeaae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53120/0 (socket says 172.21.15.7:53120) 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 -- 172.21.15.7:0/660684427 learned_addr learned my addr 172.21.15.7:0/660684427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d277fe640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d1c0a2880 0x7f4d1c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 -- 172.21.15.7:0/660684427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeab440 msgr2=0x55ad4aeaff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeab440 0x55ad4aeaff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 -- 172.21.15.7:0/660684427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 msgr2=0x55ad4aeaae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeaae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 -- 172.21.15.7:0/660684427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad4add9000 con 0x55ad4aeb05b0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d369c3640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d1c0a2880 0x7f4d1c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d277fe640 1 -- 172.21.15.7:0/4201619428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4d2c0f0e20 con 0x7f4d30101850 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d359c1640 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aeb2a40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009c90 tx=0x7f4d20011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba89aa445e252db server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d24bfe640 1 -- 172.21.15.7:0/660684427 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2001e070 con 0x55ad4aeb05b0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d24bfe640 1 -- 172.21.15.7:0/660684427 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d20019ae0 con 0x55ad4aeb05b0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.765+0000 7f4d24bfe640 1 -- 172.21.15.7:0/660684427 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000ea00 con 0x55ad4aeb05b0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 -- 172.21.15.7:0/660684427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 msgr2=0x55ad4aeb2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aeb2a40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009c90 tx=0x7f4d20011810 comp rx=0 tx=0).stop 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d369c3640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d1c0a2880 0x7f4d1c0a4d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ad4aeabca0 tx=0x7f4d280046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 -- 172.21.15.7:0/660684427 shutdown_connections 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aeb2a40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeab440 0x55ad4aeaff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/660684427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeaae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 -- 172.21.15.7:0/660684427 >> 172.21.15.7:0/660684427 conn(0x55ad4aea5fc0 msgr2=0x55ad4aea63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 -- 172.21.15.7:0/660684427 shutdown_connections 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 -- 172.21.15.7:0/660684427 wait complete. 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.766+0000 7f4d3b00c740 1 -- start start 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 Processor -- start 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aeb5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad4aeb3560 con 0x55ad4aeab440 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ad4adcb160 con 0x55ad4aeaaa50 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ad4aeb36e0 con 0x55ad4aeb05b0 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d3b00c740 10 client.0 _ll_register_callbacks cb 0x55ad4ae07a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d359c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aeb5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53894/0 (socket says 172.21.15.7:53894) 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d359c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aeb5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53128/0 (socket says 172.21.15.7:53128) 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 -- 172.21.15.7:0/3863202779 learned_addr learned my addr 172.21.15.7:0/3863202779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d369c3640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 -- 172.21.15.7:0/3863202779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 msgr2=0x55ad4aebc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 -- 172.21.15.7:0/3863202779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 msgr2=0x55ad4aeb5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aeb5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 -- 172.21.15.7:0/3863202779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad4abed7d0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aeb5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d361c2640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb4ed0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0bd160 tx=0x7f4d2c0f44f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d5f09873f6e4d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.767+0000 7f4d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3863202779 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0bcdc0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3863202779 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4d2c034ca0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d20019ae0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3863202779 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d2c0b9700 con 0x55ad4aeaaa50 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 10 client.26306 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad4ab233f0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3863202779 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d20019ae0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3863202779 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4d2c03c030 con 0x55ad4aeaaa50 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 10 client.26306 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 10 client.26306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 20 client.26306 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4aec0d20 0x55ad4aec31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 4 client.26306 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad4ae1ca40 con 0x55ad4aec0d20 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.768+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4aec0d20 0x55ad4aec31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.769+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4aec0d20 0x55ad4aec31b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be773e5c93646e72 server_cookie=9b0797f732f7fe9c in_seq=0 out_seq=0 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.769+0000 7f4d19ffb640 10 client.26306 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:47.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.769+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3863202779 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ad4ae1ca40 con 0x55ad4aec0d20 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.769+0000 7f4d19ffb640 10 client.26306 handle_command_reply: tid=0 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.769+0000 7f4d3b00c740 1 client.26306 shutdown 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 msgr2=0x55ad4aeb4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb4ed0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0bd160 tx=0x7f4d2c0f44f0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 shutdown_connections 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4aec0d20 0x55ad4aec31b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aeb5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3863202779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb4ed0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d19ffb640 0 client.26306 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 >> 172.21.15.7:0/3863202779 conn(0x55ad4aea5fc0 msgr2=0x55ad4aea51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 shutdown_connections 2024-03-19T00:42:47.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3863202779 wait complete. 2024-03-19T00:42:47.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.770+0000 7f4d3b00c740 20 client.26306 trim_cache size 0 max 16384 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 Processor -- start 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 -- start start 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aebc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aea4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301aa870 con 0x55ad4aeab440 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301a8140 con 0x55ad4aeb05b0 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.787+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301a5f90 con 0x55ad4aeaaa50 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aebc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aebc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53136/0 (socket says 172.21.15.7:53136) 2024-03-19T00:42:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 -- 172.21.15.7:0/2666756984 learned_addr learned my addr 172.21.15.7:0/2666756984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d369c3640 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aea4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d361c2640 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 -- 172.21.15.7:0/2666756984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeaaa50 msgr2=0x55ad4aeb5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 -- 172.21.15.7:0/2666756984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 msgr2=0x55ad4aea4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aea4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 -- 172.21.15.7:0/2666756984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad4aae30f0 con 0x55ad4aeab440 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d359c1640 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aebc410 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009be0 tx=0x7f4d20009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54419e8e6f1c1db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d1bfff640 1 -- 172.21.15.7:0/2666756984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2001e040 con 0x55ad4aeab440 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d1bfff640 1 -- 172.21.15.7:0/2666756984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2002f030 con 0x55ad4aeab440 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.788+0000 7f4d1bfff640 1 -- 172.21.15.7:0/2666756984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2002b7c0 con 0x55ad4aeab440 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- 172.21.15.7:0/2666756984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 msgr2=0x55ad4aebc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aebc410 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009be0 tx=0x7f4d20009f00 comp rx=0 tx=0).stop 2024-03-19T00:42:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- 172.21.15.7:0/2666756984 shutdown_connections 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeb05b0 0x55ad4aea4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeab440 0x55ad4aebc410 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/2666756984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- 172.21.15.7:0/2666756984 >> 172.21.15.7:0/2666756984 conn(0x55ad4aea5fc0 msgr2=0x55ad4aea63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- 172.21.15.7:0/2666756984 shutdown_connections 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- 172.21.15.7:0/2666756984 wait complete. 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- start start 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 Processor -- start 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeab440 0x55ad4aeba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301a6600 con 0x55ad4aeaaa50 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301a7a70 con 0x55ad4aeab440 2024-03-19T00:42:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301a6cd0 con 0x55ad4aeb05b0 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.789+0000 7f4d3b00c740 10 client.0 _ll_register_callbacks cb 0x7f4d2c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53146/0 (socket says 172.21.15.7:53146) 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 -- 172.21.15.7:0/3387748097 learned_addr learned my addr 172.21.15.7:0/3387748097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d369c3640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeab440 0x55ad4aeba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 -- 172.21.15.7:0/3387748097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 msgr2=0x55ad4aebb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 -- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeab440 msgr2=0x55ad4aeba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeab440 0x55ad4aeba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 -- 172.21.15.7:0/3387748097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad4aebb5f0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeab440 0x55ad4aeba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d361c2640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb9b00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006010 tx=0x7f4d2c0f4970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d3ff0998cc57d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3387748097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0bcdc0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3387748097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4d2c0bcab0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2002f030 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3387748097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d2c0bd230 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d3b00c740 10 client.16464 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.790+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad4af27120 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3387748097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2002f030 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3387748097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4d2c0dd0b0 con 0x55ad4aeaaa50 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 10 client.16464 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 20 client.16464 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4af27850 0x55ad4af29c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 4 client.16464 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad4ae16d90 con 0x55ad4af27850 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4af27850 0x55ad4af29c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d359c1640 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4af27850 0x55ad4af29c70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c20cba649b32b43 server_cookie=bd22da5901d177ec in_seq=0 out_seq=0 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d19ffb640 10 client.16464 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d19ffb640 1 -- 172.21.15.7:0/3387748097 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ad4ae16d90 con 0x55ad4af27850 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d19ffb640 10 client.16464 handle_command_reply: tid=0 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.791+0000 7f4d3b00c740 1 client.16464 shutdown 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 msgr2=0x55ad4aeb9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb9b00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006010 tx=0x7f4d2c0f4970 comp rx=0 tx=0).stop 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 shutdown_connections 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad4aeb05b0 0x55ad4aebb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad4aeab440 0x55ad4aeba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ad4af27850 0x55ad4af29c70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 --2- 172.21.15.7:0/3387748097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad4aeaaa50 0x55ad4aeb9b00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 >> 172.21.15.7:0/3387748097 conn(0x55ad4aea5fc0 msgr2=0x55ad4aea9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d19ffb640 0 client.16464 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 shutdown_connections 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 1 -- 172.21.15.7:0/3387748097 wait complete. 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d3b00c740 20 client.16464 trim_cache size 0 max 16384 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:42:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:42:47.801 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:42:47.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d1c0a2880 msgr2=0x7f4d1c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d1c0a2880 0x7f4d1c0a4d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ad4aeabca0 tx=0x7f4d280046c0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 msgr2=0x7f4d30197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30197a80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c000f80 tx=0x7f4d2c0328a0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 shutdown_connections 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d1c0a2880 0x7f4d1c0a4d40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d30103400 0x7f4d3019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d30102a40 0x7f4d30197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 --2- 172.21.15.7:0/4201619428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d30101850 0x7f4d30197a80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 >> 172.21.15.7:0/4201619428 conn(0x7f4d300fcde0 msgr2=0x7f4d30105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 shutdown_connections 2024-03-19T00:42:47.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.792+0000 7f4d38426640 1 -- 172.21.15.7:0/4201619428 wait complete. 2024-03-19T00:42:47.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:42:47.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 -- 172.21.15.7:0/1747205935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c1035e0 msgr2=0x7fe76c109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 --2- 172.21.15.7:0/1747205935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c1035e0 0x7fe76c109e70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe764009930 tx=0x7fe76402bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 -- 172.21.15.7:0/1747205935 shutdown_connections 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 --2- 172.21.15.7:0/1747205935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c1035e0 0x7fe76c109e70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 --2- 172.21.15.7:0/1747205935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c102c20 0x7fe76c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 --2- 172.21.15.7:0/1747205935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c0ffbb0 0x7fe76c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 -- 172.21.15.7:0/1747205935 >> 172.21.15.7:0/1747205935 conn(0x7fe76c0fb4a0 msgr2=0x7fe76c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 -- 172.21.15.7:0/1747205935 shutdown_connections 2024-03-19T00:42:47.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.906+0000 7fe770fdf640 1 -- 172.21.15.7:0/1747205935 wait complete. 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 Processor -- start 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 -- start start 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c0ffbb0 0x7fe76c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 0x7fe76c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c1035e0 0x7fe76c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c10d370 con 0x7fe76c102c20 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c10d1f0 con 0x7fe76c0ffbb0 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe770fdf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c10d4f0 con 0x7fe76c1035e0 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 0x7fe76c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 0x7fe76c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53162/0 (socket says 172.21.15.7:53162) 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 -- 172.21.15.7:0/2459063641 learned_addr learned my addr 172.21.15.7:0/2459063641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe76a59c640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c0ffbb0 0x7fe76c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe76ad9d640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c1035e0 0x7fe76c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 -- 172.21.15.7:0/2459063641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c1035e0 msgr2=0x7fe76c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c1035e0 0x7fe76c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 -- 172.21.15.7:0/2459063641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c0ffbb0 msgr2=0x7fe76c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c0ffbb0 0x7fe76c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 -- 172.21.15.7:0/2459063641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c19f630 con 0x7fe76c102c20 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe76ad9d640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c1035e0 0x7fe76c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe76a59c640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c0ffbb0 0x7fe76c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe769d9b640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 0x7fe76c197f50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e9f0 tx=0x7fe76000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.907+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76000cd80 con 0x7fe76c102c20 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.908+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7600108e0 con 0x7fe76c102c20 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.908+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c19f920 con 0x7fe76c102c20 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.908+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760018920 con 0x7fe76c102c20 2024-03-19T00:42:47.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.908+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c19fe70 con 0x7fe76c102c20 2024-03-19T00:42:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.909+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76c108d50 con 0x7fe76c102c20 2024-03-19T00:42:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.910+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe760010a90 con 0x7fe76c102c20 2024-03-19T00:42:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.911+0000 7fe75b7fe640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7540a2800 0x7fe7540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.911+0000 7fe76a59c640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7540a2800 0x7fe7540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.911+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7600cf580 con 0x7fe76c102c20 2024-03-19T00:42:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.911+0000 7fe76a59c640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7540a2800 0x7fe7540a4cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe76c069cc0 tx=0x7fe75c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:47.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:47.915+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe76009bc70 con 0x7fe76c102c20 2024-03-19T00:42:47.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:47 smithi007 ceph-mon[62854]: pgmap v496: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:47 smithi204 ceph-mon[66572]: pgmap v496: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:47 smithi044 ceph-mon[66366]: pgmap v496: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:42:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.137+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe76c02d460 con 0x7fe76c102c20 2024-03-19T00:42:48.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.137+0000 7fe75b7fe640 1 -- 172.21.15.7:0/2459063641 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fe76009fd10 con 0x7fe76c102c20 2024-03-19T00:42:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:42:48.138 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:42:48.139 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:42:48.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7540a2800 msgr2=0x7fe7540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7540a2800 0x7fe7540a4cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe76c069cc0 tx=0x7fe75c009500 comp rx=0 tx=0).stop 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 msgr2=0x7fe76c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 0x7fe76c197f50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e9f0 tx=0x7fe76000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 shutdown_connections 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7540a2800 0x7fe7540a4cc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76c1035e0 0x7fe76c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c102c20 0x7fe76c197f50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 --2- 172.21.15.7:0/2459063641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe76c0ffbb0 0x7fe76c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 >> 172.21.15.7:0/2459063641 conn(0x7fe76c0fb4a0 msgr2=0x7fe76c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.139+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 shutdown_connections 2024-03-19T00:42:48.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.140+0000 7fe770fdf640 1 -- 172.21.15.7:0/2459063641 wait complete. 2024-03-19T00:42:48.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:42:48.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/3240289194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4102ab0 msgr2=0x7f6df4102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/3240289194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4102ab0 0x7f6df4102f30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009930 tx=0x7f6de402bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:48.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/3240289194 shutdown_connections 2024-03-19T00:42:48.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/3240289194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6df4103470 0x7f6df4109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/3240289194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4102ab0 0x7f6df4102f30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/3240289194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df40ffae0 0x7f6df40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/3240289194 >> 172.21.15.7:0/3240289194 conn(0x7f6df40fb3d0 msgr2=0x7f6df40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/3240289194 shutdown_connections 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.256+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/3240289194 wait complete. 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 Processor -- start 2024-03-19T00:42:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 -- start start 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6df40ffae0 0x7f6df4102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df4102ab0 0x7f6df4100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 0x7f6df4100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6df410d370 con 0x7f6df4103470 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6df410d1f0 con 0x7f6df4102ab0 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfc6bf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6df410d4f0 con 0x7f6df40ffae0 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 0x7f6df4100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 0x7f6df4100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53166/0 (socket says 172.21.15.7:53166) 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 -- 172.21.15.7:0/2465777819 learned_addr learned my addr 172.21.15.7:0/2465777819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df4102ab0 0x7f6df4100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6df40ffae0 0x7f6df4102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 -- 172.21.15.7:0/2465777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6df40ffae0 msgr2=0x7f6df4102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6df40ffae0 0x7f6df4102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 -- 172.21.15.7:0/2465777819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df4102ab0 msgr2=0x7f6df4100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df4102ab0 0x7f6df4100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 -- 172.21.15.7:0/2465777819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df4101830 con 0x7f6df4103470 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df4102ab0 0x7f6df4100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.257+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 0x7f6df4100fa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00eb10 tx=0x7f6dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.258+0000 7f6deb7fe640 1 -- 172.21.15.7:0/2465777819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec004b20 con 0x7f6df4103470 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.258+0000 7f6deb7fe640 1 -- 172.21.15.7:0/2465777819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dec004cc0 con 0x7f6df4103470 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.258+0000 7f6deb7fe640 1 -- 172.21.15.7:0/2465777819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec010690 con 0x7f6df4103470 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.258+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df41a1bc0 con 0x7f6df4103470 2024-03-19T00:42:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.258+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41a2090 con 0x7f6df4103470 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.260+0000 7f6deb7fe640 1 -- 172.21.15.7:0/2465777819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6dec005020 con 0x7f6df4103470 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.260+0000 7f6dff389740 1 Processor -- start 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.260+0000 7f6dff389740 1 -- start start 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dff389740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dff389740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6deb7fe640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6de00a2800 0x7f6de00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dff389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc620a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dff389740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bc620054b0 con 0x55bc620a1ce0 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dff389740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bc61fadd70 con 0x55bc620a12f0 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dff389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bc62001550 con 0x55bc620a6e50 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6deb7fe640 1 -- 172.21.15.7:0/2465777819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6dec0cf4e0 con 0x7f6df4103470 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc620a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53986/0 (socket says 172.21.15.7:53986) 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6de00a2800 0x7f6de00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6df9c5a640 1 -- 172.21.15.7:0/3790316198 learned_addr learned my addr 172.21.15.7:0/3790316198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc620a92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:35510/0 (socket says 172.21.15.7:35510) 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 -- 172.21.15.7:0/3790316198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 msgr2=0x55bc620a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 -- 172.21.15.7:0/3790316198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 msgr2=0x55bc620a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 -- 172.21.15.7:0/3790316198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc61fd0210 con 0x55bc620a6e50 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6de00a2800 0x7f6de00a4cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6df4069d80 tx=0x7f6df0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.261+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc620a92e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0cccd0 tx=0x7f6dec097760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9601a4c133cb09d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6de8ff9640 1 -- 172.21.15.7:0/3790316198 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec09b480 con 0x55bc620a6e50 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6de8ff9640 1 -- 172.21.15.7:0/3790316198 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dec09ce50 con 0x55bc620a6e50 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6de8ff9640 1 -- 172.21.15.7:0/3790316198 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0a8ec0 con 0x55bc620a6e50 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 -- 172.21.15.7:0/3790316198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 msgr2=0x55bc620a92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc620a92e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0cccd0 tx=0x7f6dec097760 comp rx=0 tx=0).stop 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 -- 172.21.15.7:0/3790316198 shutdown_connections 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc620a92e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 --2- 172.21.15.7:0/3790316198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 -- 172.21.15.7:0/3790316198 >> 172.21.15.7:0/3790316198 conn(0x55bc6209c860 msgr2=0x55bc6209cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 -- 172.21.15.7:0/3790316198 shutdown_connections 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.262+0000 7f6dff389740 1 -- 172.21.15.7:0/3790316198 wait complete. 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 -- start start 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 Processor -- start 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 0x55bc620ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a1ce0 0x55bc620abd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bc620a9e00 con 0x55bc620a12f0 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bc61fc1310 con 0x55bc620a6e50 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bc620a9f80 con 0x55bc620a1ce0 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dff389740 10 client.0 _ll_register_callbacks cb 0x55bc61fb52d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 0x55bc620ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 0x55bc620ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53186/0 (socket says 172.21.15.7:53186) 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 -- 172.21.15.7:0/1072953079 learned_addr learned my addr 172.21.15.7:0/1072953079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a1ce0 0x55bc620abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 -- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a1ce0 msgr2=0x55bc620abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a1ce0 0x55bc620abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 -- 172.21.15.7:0/1072953079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 msgr2=0x55bc620b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 -- 172.21.15.7:0/1072953079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc61d193f0 con 0x55bc620a12f0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 0x55bc620ab7d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6df000ecb0 tx=0x7f6df0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5564f2375808d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.263+0000 7f6dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dddffb640 1 -- 172.21.15.7:0/1072953079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0004220 con 0x55bc620a12f0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dddffb640 1 -- 172.21.15.7:0/1072953079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6df00043c0 con 0x55bc620a12f0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dec09ce50 con 0x55bc620a12f0 2024-03-19T00:42:48.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dddffb640 1 -- 172.21.15.7:0/1072953079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6df0030050 con 0x55bc620a12f0 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 10 client.16482 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc61de37d0 con 0x55bc620a12f0 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dddffb640 1 -- 172.21.15.7:0/1072953079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dec09ce50 con 0x55bc620a12f0 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dddffb640 1 -- 172.21.15.7:0/1072953079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6df0003bd0 con 0x55bc620a12f0 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 10 client.16482 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 10 client.16482 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 20 client.16482 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc620b76c0 0x55bc620b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 4 client.16482 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc620633e0 con 0x55bc620b76c0 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.264+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc620b76c0 0x55bc620b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.265+0000 7f6dddffb640 10 client.16482 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.265+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc620b76c0 0x55bc620b9b60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6d1eee345f14793 server_cookie=bef091c2cc23bc6a in_seq=0 out_seq=0 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dddffb640 1 -- 172.21.15.7:0/1072953079 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc620633e0 con 0x55bc620b76c0 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dddffb640 10 client.16482 handle_command_reply: tid=0 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 client.16482 shutdown 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 msgr2=0x55bc620ab7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 0x55bc620ab7d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6df000ecb0 tx=0x7f6df0004730 comp rx=0 tx=0).stop 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 shutdown_connections 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc620b76c0 0x55bc620b9b60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a1ce0 0x55bc620abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 --2- 172.21.15.7:0/1072953079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a12f0 0x55bc620ab7d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dddffb640 0 client.16482 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 >> 172.21.15.7:0/1072953079 conn(0x55bc6209c860 msgr2=0x55bc6209bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 shutdown_connections 2024-03-19T00:42:48.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 1 -- 172.21.15.7:0/1072953079 wait complete. 2024-03-19T00:42:48.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.266+0000 7f6dff389740 20 client.16482 trim_cache size 0 max 16384 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 Processor -- start 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 -- start start 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc6209b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41aee20 con 0x55bc620a1ce0 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ac6f0 con 0x55bc620a12f0 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dff389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41aa540 con 0x55bc620a6e50 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53200/0 (socket says 172.21.15.7:53200) 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6df9c5a640 1 -- 172.21.15.7:0/434527464 learned_addr learned my addr 172.21.15.7:0/434527464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.279+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc6209b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6df9c5a640 1 -- 172.21.15.7:0/434527464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 msgr2=0x55bc6209b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc6209b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6df9c5a640 1 -- 172.21.15.7:0/434527464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 msgr2=0x55bc620abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6df9c5a640 1 -- 172.21.15.7:0/434527464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc620b6000 con 0x55bc620a1ce0 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b2d10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6de4000c00 tx=0x7f6de4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f63400abc7f2db server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6ddffff640 1 -- 172.21.15.7:0/434527464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de40035f0 con 0x55bc620a1ce0 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6ddffff640 1 -- 172.21.15.7:0/434527464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6de4003c40 con 0x55bc620a1ce0 2024-03-19T00:42:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.280+0000 7f6ddffff640 1 -- 172.21.15.7:0/434527464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4043700 con 0x55bc620a1ce0 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- 172.21.15.7:0/434527464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 msgr2=0x55bc620b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b2d10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6de4000c00 tx=0x7f6de4009560 comp rx=0 tx=0).stop 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- 172.21.15.7:0/434527464 shutdown_connections 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a6e50 0x55bc6209b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b2d10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- 172.21.15.7:0/434527464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a12f0 0x55bc620abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- 172.21.15.7:0/434527464 >> 172.21.15.7:0/434527464 conn(0x55bc6209c860 msgr2=0x55bc620ab3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- 172.21.15.7:0/434527464 shutdown_connections 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- 172.21.15.7:0/434527464 wait complete. 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- start start 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 Processor -- start 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a12f0 0x55bc620b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41aabb0 con 0x55bc620a1ce0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ac020 con 0x55bc620a6e50 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ab280 con 0x55bc620a12f0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dff389740 10 client.0 _ll_register_callbacks cb 0x7f6df000e960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6df9c5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53208/0 (socket says 172.21.15.7:53208) 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6df9c5a640 1 -- 172.21.15.7:0/1469833822 learned_addr learned my addr 172.21.15.7:0/1469833822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a12f0 0x55bc620b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.281+0000 7f6dfac5c640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6df9c5a640 1 -- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a12f0 msgr2=0x55bc620b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a12f0 0x55bc620b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6df9c5a640 1 -- 172.21.15.7:0/1469833822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 msgr2=0x55bc620b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6df9c5a640 1 -- 172.21.15.7:0/1469833822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc620b1ce0 con 0x55bc620a1ce0 2024-03-19T00:42:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6df9c5a640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b0730 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005ec0 tx=0x7f6de4043450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c2be3c53a7be23 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dddffb640 1 -- 172.21.15.7:0/1469833822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4058070 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dddffb640 1 -- 172.21.15.7:0/1469833822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6de4003f30 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de4003c40 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dddffb640 1 -- 172.21.15.7:0/1469833822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6de4064050 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dff389740 10 client.16488 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc620b3010 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dddffb640 1 -- 172.21.15.7:0/1469833822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6de4003c40 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.282+0000 7f6dddffb640 1 -- 172.21.15.7:0/1469833822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6de4034440 con 0x55bc620a1ce0 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 10 client.16488 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 10 client.16488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 20 client.16488 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc6211e190 0x55bc621205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 4 client.16488 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc620129f0 con 0x55bc6211e190 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc6211e190 0x55bc621205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dddffb640 10 client.16488 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dfa45b640 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc6211e190 0x55bc621205b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=885c6d2bb697287d server_cookie=e164b0c5ad93e2ec in_seq=0 out_seq=0 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dddffb640 1 -- 172.21.15.7:0/1469833822 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc620129f0 con 0x55bc6211e190 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dddffb640 10 client.16488 handle_command_reply: tid=0 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 client.16488 shutdown 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 msgr2=0x55bc620b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b0730 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005ec0 tx=0x7f6de4043450 comp rx=0 tx=0).stop 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 shutdown_connections 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55bc6211e190 0x55bc621205b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc620a6e50 0x55bc620b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dddffb640 0 client.16488 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc620a1ce0 0x55bc620b0730 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 --2- 172.21.15.7:0/1469833822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc620a12f0 0x55bc620b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 >> 172.21.15.7:0/1469833822 conn(0x55bc6209c860 msgr2=0x55bc6209fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 shutdown_connections 2024-03-19T00:42:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.283+0000 7f6dff389740 1 -- 172.21.15.7:0/1469833822 wait complete. 2024-03-19T00:42:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dff389740 20 client.16488 trim_cache size 0 max 16384 2024-03-19T00:42:48.292 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6de00a2800 msgr2=0x7f6de00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6de00a2800 0x7f6de00a4cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6df4069d80 tx=0x7f6df0005f90 comp rx=0 tx=0).stop 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 msgr2=0x7f6df4100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 0x7f6df4100fa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00eb10 tx=0x7f6dec00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 shutdown_connections 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6de00a2800 0x7f6de00a4cc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df4103470 0x7f6df4100fa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6df4102ab0 0x7f6df4100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 --2- 172.21.15.7:0/2465777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6df40ffae0 0x7f6df4102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 >> 172.21.15.7:0/2465777819 conn(0x7f6df40fb3d0 msgr2=0x7f6df4105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 shutdown_connections 2024-03-19T00:42:48.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.284+0000 7f6dfc6bf640 1 -- 172.21.15.7:0/2465777819 wait complete. 2024-03-19T00:42:48.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:42:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 -- 172.21.15.7:0/4227735366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 msgr2=0x7f7f38101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/4227735366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38101c70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009900 tx=0x7f7f2c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 -- 172.21.15.7:0/4227735366 shutdown_connections 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/4227735366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f38103400 0x7f7f38109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/4227735366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f38102a40 0x7f7f38102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/4227735366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38101c70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 -- 172.21.15.7:0/4227735366 >> 172.21.15.7:0/4227735366 conn(0x7f7f380fcd80 msgr2=0x7f7f380ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 -- 172.21.15.7:0/4227735366 shutdown_connections 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 -- 172.21.15.7:0/4227735366 wait complete. 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 Processor -- start 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 -- start start 2024-03-19T00:42:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.399+0000 7f7f3f00b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3f00b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f38102a40 0x7f7f38197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3f00b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f38103400 0x7f7f3819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3f00b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810d350 con 0x7f7f38101850 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3f00b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810d1d0 con 0x7f7f38102a40 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3f00b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810d4d0 con 0x7f7f38103400 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53222/0 (socket says 172.21.15.7:53222) 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 -- 172.21.15.7:0/771688448 learned_addr learned my addr 172.21.15.7:0/771688448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f37fff640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f38102a40 0x7f7f38197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f38103400 0x7f7f3819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 -- 172.21.15.7:0/771688448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f38103400 msgr2=0x7f7f3819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f38103400 0x7f7f3819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 -- 172.21.15.7:0/771688448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f38102a40 msgr2=0x7f7f38197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f38102a40 0x7f7f38197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 -- 172.21.15.7:0/771688448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3819f6b0 con 0x7f7f38101850 2024-03-19T00:42:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38197a90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c000c00 tx=0x7f7f2c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f35ffb640 1 -- 172.21.15.7:0/771688448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c043020 con 0x7f7f38101850 2024-03-19T00:42:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f35ffb640 1 -- 172.21.15.7:0/771688448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f2c004350 con 0x7f7f38101850 2024-03-19T00:42:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3819f940 con 0x7f7f38101850 2024-03-19T00:42:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.400+0000 7f7f35ffb640 1 -- 172.21.15.7:0/771688448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c003b10 con 0x7f7f38101850 2024-03-19T00:42:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.401+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3819fe90 con 0x7f7f38101850 2024-03-19T00:42:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.403+0000 7f7f41bf8740 1 Processor -- start 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.403+0000 7f7f35ffb640 1 -- 172.21.15.7:0/771688448 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7f2c046020 con 0x7f7f38101850 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 -- start start 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 0x56354d900b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d9010f0 0x56354d905bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d906260 0x56354d9086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56354d8604b0 con 0x56354d906260 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56354d808d70 con 0x56354d900700 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56354d85c550 con 0x56354d9010f0 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d906260 0x56354d9086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 0x56354d900b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3cda7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d9010f0 0x56354d905bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 0x56354d900b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54046/0 (socket says 172.21.15.7:54046) 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 -- 172.21.15.7:0/3600959950 learned_addr learned my addr 172.21.15.7:0/3600959950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f35ffb640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f280a2880 0x7f7f280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 -- 172.21.15.7:0/3600959950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d9010f0 msgr2=0x56354d905bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d9010f0 0x56354d905bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 -- 172.21.15.7:0/3600959950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d906260 msgr2=0x56354d9086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d906260 0x56354d9086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.404+0000 7f7f3d5a8640 1 -- 172.21.15.7:0/3600959950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354d82b210 con 0x56354d900700 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.405+0000 7f7f35ffb640 1 -- 172.21.15.7:0/771688448 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7f2c0f0ec0 con 0x7f7f38101850 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.405+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f280a2880 0x7f7f280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.405+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 0x56354d900b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f30011570 tx=0x7f7f30011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d53324c6ccb0587 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.405+0000 7f7f22ffd640 1 -- 172.21.15.7:0/3600959950 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30017070 con 0x56354d900700 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.405+0000 7f7f22ffd640 1 -- 172.21.15.7:0/3600959950 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f30004760 con 0x56354d900700 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.405+0000 7f7f22ffd640 1 -- 172.21.15.7:0/3600959950 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30026b20 con 0x56354d900700 2024-03-19T00:42:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3600959950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 msgr2=0x56354d900b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 0x56354d900b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f30011570 tx=0x7f7f30011a30 comp rx=0 tx=0).stop 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f280a2880 0x7f7f280a4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f30021000 tx=0x7f7f30022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3600959950 shutdown_connections 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d906260 0x56354d9086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d9010f0 0x56354d905bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3600959950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d900700 0x56354d900b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3600959950 >> 172.21.15.7:0/3600959950 conn(0x56354d8f7860 msgr2=0x56354d8f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3600959950 shutdown_connections 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3600959950 wait complete. 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- start start 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 Processor -- start 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d90b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d912110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56354d9091b0 con 0x56354d9010f0 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56354d81c310 con 0x56354d906260 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.406+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56354d909330 con 0x56354d900700 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f41bf8740 10 client.0 _ll_register_callbacks cb 0x56354d8102d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d90b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f3cda7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d90b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53242/0 (socket says 172.21.15.7:53242) 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 -- 172.21.15.7:0/4218398290 learned_addr learned my addr 172.21.15.7:0/4218398290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d912110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 -- 172.21.15.7:0/4218398290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 msgr2=0x56354d90ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 -- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 msgr2=0x56354d912110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d912110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 -- 172.21.15.7:0/4218398290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354d5743f0 con 0x56354d9010f0 2024-03-19T00:42:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d912110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f37fff640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d90b0d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7f24012700 tx=0x7f7f24012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd85533e8229f59 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f17fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f17fff640 1 -- 172.21.15.7:0/4218398290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2400f8c0 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f30004760 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f17fff640 1 -- 172.21.15.7:0/4218398290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7f2400fa60 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f17fff640 1 -- 172.21.15.7:0/4218398290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f24025050 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f41bf8740 10 client.16500 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.407+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56354d63e7d0 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f17fff640 1 -- 172.21.15.7:0/4218398290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f30004760 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f17fff640 1 -- 172.21.15.7:0/4218398290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7f24010430 con 0x56354d9010f0 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f41bf8740 10 client.16500 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f41bf8740 10 client.16500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f41bf8740 20 client.16500 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d916a60 0x56354d918ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f41bf8740 4 client.16500 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56354d8be3e0 con 0x56354d916a60 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d916a60 0x56354d918ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f17fff640 10 client.16500 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.408+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d916a60 0x56354d918ec0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc863f8bd15664c0 server_cookie=30c4097e69872f63 in_seq=0 out_seq=0 2024-03-19T00:42:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.409+0000 7f7f17fff640 1 -- 172.21.15.7:0/4218398290 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56354d8be3e0 con 0x56354d916a60 2024-03-19T00:42:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.409+0000 7f7f17fff640 10 client.16500 handle_command_reply: tid=0 2024-03-19T00:42:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.409+0000 7f7f41bf8740 1 client.16500 shutdown 2024-03-19T00:42:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.409+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 msgr2=0x56354d90b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.409+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d90b0d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7f24012700 tx=0x7f7f24012bc0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.409+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 shutdown_connections 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d916a60 0x56354d918ec0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d912110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d90b0d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4218398290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f17fff640 0 client.16500 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 >> 172.21.15.7:0/4218398290 conn(0x56354d8f7860 msgr2=0x56354d8f6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 shutdown_connections 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4218398290 wait complete. 2024-03-19T00:42:48.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.410+0000 7f7f41bf8740 20 client.16500 trim_cache size 0 max 16384 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 Processor -- start 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 -- start start 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 0x56354d9025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d9010f0 0x56354d9120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d906260 0x56354d902af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381aa9e0 con 0x56354d900700 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381a82b0 con 0x56354d9010f0 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.423+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381a6100 con 0x56354d906260 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 0x56354d9025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 0x56354d9025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53254/0 (socket says 172.21.15.7:53254) 2024-03-19T00:42:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 -- 172.21.15.7:0/4284580661 learned_addr learned my addr 172.21.15.7:0/4284580661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f37fff640 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d9010f0 0x56354d9120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d906260 0x56354d902af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 -- 172.21.15.7:0/4284580661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d906260 msgr2=0x56354d902af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d906260 0x56354d902af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 -- 172.21.15.7:0/4284580661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d9010f0 msgr2=0x56354d9120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d9010f0 0x56354d9120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 -- 172.21.15.7:0/4284580661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354d8cd880 con 0x56354d900700 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f37fff640 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d9010f0 0x56354d9120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 0x56354d9025b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0b9e20 tx=0x7f7f2c0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13175086732a570 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f227fc640 1 -- 172.21.15.7:0/4284580661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0bcfe0 con 0x56354d900700 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f227fc640 1 -- 172.21.15.7:0/4284580661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f2c0bdba0 con 0x56354d900700 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.424+0000 7f7f227fc640 1 -- 172.21.15.7:0/4284580661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0bce00 con 0x56354d900700 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4284580661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 msgr2=0x56354d9025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 0x56354d9025b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0b9e20 tx=0x7f7f2c0dd040 comp rx=0 tx=0).stop 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4284580661 shutdown_connections 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d906260 0x56354d902af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d9010f0 0x56354d9120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/4284580661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d900700 0x56354d9025b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4284580661 >> 172.21.15.7:0/4284580661 conn(0x56354d8f7860 msgr2=0x56354d8f7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4284580661 shutdown_connections 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 -- 172.21.15.7:0/4284580661 wait complete. 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 -- start start 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.425+0000 7f7f41bf8740 1 Processor -- start 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d910040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d910fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381a6770 con 0x56354d9010f0 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381a7be0 con 0x56354d906260 2024-03-19T00:42:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381a6e40 con 0x56354d900700 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 10 client.0 _ll_register_callbacks cb 0x7f7f24009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d910040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d910040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53270/0 (socket says 172.21.15.7:53270) 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 -- 172.21.15.7:0/3347277332 learned_addr learned my addr 172.21.15.7:0/3347277332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f3d5a8640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d910fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 -- 172.21.15.7:0/3347277332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 msgr2=0x56354d90fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 -- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 msgr2=0x56354d910fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d910fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 -- 172.21.15.7:0/3347277332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354d9115d0 con 0x56354d9010f0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f37fff640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d910040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7f24010a40 tx=0x7f7f24010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ba49eb85af98c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f17fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f17fff640 1 -- 172.21.15.7:0/3347277332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24010640 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f17fff640 1 -- 172.21.15.7:0/3347277332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7f24022040 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.426+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f2c0bdba0 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f17fff640 1 -- 172.21.15.7:0/3347277332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f24032050 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 10 client.16506 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56354d97d010 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f17fff640 1 -- 172.21.15.7:0/3347277332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f2c0bdba0 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f17fff640 1 -- 172.21.15.7:0/3347277332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7f2400fc30 con 0x56354d9010f0 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 10 client.16506 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 10 client.16506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 20 client.16506 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d97d9e0 0x56354d97fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 4 client.16506 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:42:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56354d86d9f0 con 0x56354d97d9e0 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d97d9e0 0x56354d97fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f17fff640 10 client.16506 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.427+0000 7f7f3cda7640 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d97d9e0 0x56354d97fe00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5f0d259534468c8 server_cookie=b8b38529410ccd7 in_seq=0 out_seq=0 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f17fff640 1 -- 172.21.15.7:0/3347277332 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56354d86d9f0 con 0x56354d97d9e0 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f17fff640 10 client.16506 handle_command_reply: tid=0 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 client.16506 shutdown 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 msgr2=0x56354d910040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d910040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7f24010a40 tx=0x7f7f24010cf0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 shutdown_connections 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56354d97d9e0 0x56354d97fe00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56354d906260 0x56354d910fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f17fff640 0 client.16506 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56354d9010f0 0x56354d910040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 --2- 172.21.15.7:0/3347277332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56354d900700 0x56354d90fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 >> 172.21.15.7:0/3347277332 conn(0x56354d8f7860 msgr2=0x56354d90ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 shutdown_connections 2024-03-19T00:42:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 1 -- 172.21.15.7:0/3347277332 wait complete. 2024-03-19T00:42:48.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f41bf8740 20 client.16506 trim_cache size 0 max 16384 2024-03-19T00:42:48.436 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f280a2880 msgr2=0x7f7f280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f280a2880 0x7f7f280a4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f30021000 tx=0x7f7f30022000 comp rx=0 tx=0).stop 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 msgr2=0x7f7f38197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.428+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38197a90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c000c00 tx=0x7f7f2c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 shutdown_connections 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7f280a2880 0x7f7f280a4d40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f38103400 0x7f7f3819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f38102a40 0x7f7f38197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 --2- 172.21.15.7:0/771688448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38101850 0x7f7f38197a90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 >> 172.21.15.7:0/771688448 conn(0x7f7f380fcd80 msgr2=0x7f7f38105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 shutdown_connections 2024-03-19T00:42:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.429+0000 7f7f3f00b640 1 -- 172.21.15.7:0/771688448 wait complete. 2024-03-19T00:42:48.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.544+0000 7f62447b3640 1 -- 172.21.15.7:0/2292996871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 msgr2=0x7f623c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.544+0000 7f62447b3640 1 --2- 172.21.15.7:0/2292996871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c101c50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c02bc50 comp rx=0 tx=0).stop 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- 172.21.15.7:0/2292996871 shutdown_connections 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 --2- 172.21.15.7:0/2292996871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c1033e0 0x7f623c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 --2- 172.21.15.7:0/2292996871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c102a20 0x7f623c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 --2- 172.21.15.7:0/2292996871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c101c50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- 172.21.15.7:0/2292996871 >> 172.21.15.7:0/2292996871 conn(0x7f623c0fcd60 msgr2=0x7f623c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- 172.21.15.7:0/2292996871 shutdown_connections 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- 172.21.15.7:0/2292996871 wait complete. 2024-03-19T00:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 Processor -- start 2024-03-19T00:42:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- start start 2024-03-19T00:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c102a20 0x7f623c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c1033e0 0x7f623c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10d3d0 con 0x7f623c101830 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10d250 con 0x7f623c1033e0 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f62447b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10d550 con 0x7f623c102a20 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.545+0000 7f624254f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f6241d4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c102a20 0x7f623c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53294/0 (socket says 172.21.15.7:53294) 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 -- 172.21.15.7:0/2560873177 learned_addr learned my addr 172.21.15.7:0/2560873177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f6242d50640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c1033e0 0x7f623c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 -- 172.21.15.7:0/2560873177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c102a20 msgr2=0x7f623c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c102a20 0x7f623c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 -- 172.21.15.7:0/2560873177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c1033e0 msgr2=0x7f623c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c1033e0 0x7f623c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 -- 172.21.15.7:0/2560873177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c19f710 con 0x7f623c101830 2024-03-19T00:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f624254f640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c197af0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f622c000c00 tx=0x7f622c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f62337fe640 1 -- 172.21.15.7:0/2560873177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c043020 con 0x7f623c101830 2024-03-19T00:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f62337fe640 1 -- 172.21.15.7:0/2560873177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f622c0052b0 con 0x7f623c101830 2024-03-19T00:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c19f9a0 con 0x7f623c101830 2024-03-19T00:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f62337fe640 1 -- 172.21.15.7:0/2560873177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c003cd0 con 0x7f623c101830 2024-03-19T00:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.546+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c19ffc0 con 0x7f623c101830 2024-03-19T00:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.548+0000 7f62337fe640 1 -- 172.21.15.7:0/2560873177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f622c046020 con 0x7f623c101830 2024-03-19T00:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 Processor -- start 2024-03-19T00:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 -- start start 2024-03-19T00:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x5613100f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x5613100fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f62337fe640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62280a2880 0x7f62280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x5613101012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56131005d4b0 con 0x5613100f92f0 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561310005d70 con 0x5613100f9ce0 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6247399740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561310059550 con 0x5613100fee50 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6242d50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x5613100f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6241d4e640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62280a2880 0x7f62280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f624254f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x5613100fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6242d50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x5613100f9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53304/0 (socket says 172.21.15.7:53304) 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6242d50640 1 -- 172.21.15.7:0/1541232438 learned_addr learned my addr 172.21.15.7:0/1541232438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f62337fe640 1 -- 172.21.15.7:0/2560873177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f622c0f0f90 con 0x7f623c101830 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.549+0000 7f6242d50640 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x5613101012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6242d50640 1 -- 172.21.15.7:0/1541232438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 msgr2=0x5613101012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6242d50640 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x5613101012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6242d50640 1 -- 172.21.15.7:0/1541232438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 msgr2=0x5613100fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6242d50640 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x5613100fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6242d50640 1 -- 172.21.15.7:0/1541232438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561310028210 con 0x5613100f92f0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f624254f640 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x5613100fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6241d4e640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62280a2880 0x7f62280a4d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f62380095f0 tx=0x7f6238005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6242d50640 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x5613100f9710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f623400ad10 tx=0x7f6234009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2ab40c7e7bd640 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6230ff9640 1 -- 172.21.15.7:0/1541232438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234014040 con 0x5613100f92f0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6230ff9640 1 -- 172.21.15.7:0/1541232438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f623400e030 con 0x5613100f92f0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.550+0000 7f6230ff9640 1 -- 172.21.15.7:0/1541232438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623401abf0 con 0x5613100f92f0 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 -- 172.21.15.7:0/1541232438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 msgr2=0x5613100f9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x5613100f9710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f623400ad10 tx=0x7f6234009b50 comp rx=0 tx=0).stop 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 -- 172.21.15.7:0/1541232438 shutdown_connections 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x5613101012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x5613100fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 --2- 172.21.15.7:0/1541232438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x5613100f9710 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 -- 172.21.15.7:0/1541232438 >> 172.21.15.7:0/1541232438 conn(0x5613100f4860 msgr2=0x5613100f4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 -- 172.21.15.7:0/1541232438 shutdown_connections 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 -- 172.21.15.7:0/1541232438 wait complete. 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 -- start start 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 Processor -- start 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.551+0000 7f6247399740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f9ce0 0x561310103c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 0x56131010acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561310101da0 con 0x5613100fee50 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561310019310 con 0x5613100f92f0 2024-03-19T00:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561310101f20 con 0x5613100f9ce0 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 10 client.0 _ll_register_callbacks cb 0x56131000d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 0x56131010acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f624254f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 0x56131010acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53318/0 (socket says 172.21.15.7:53318) 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 -- 172.21.15.7:0/2917989274 learned_addr learned my addr 172.21.15.7:0/2917989274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6241d4e640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f9ce0 0x561310103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 -- 172.21.15.7:0/2917989274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f9ce0 msgr2=0x561310103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f9ce0 0x561310103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 -- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f92f0 msgr2=0x561310103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 -- 172.21.15.7:0/2917989274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130fd713f0 con 0x5613100fee50 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f624254f640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6242d50640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 0x56131010acc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f623400a010 tx=0x7f623401a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41364b3e2233c369 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6225ffb640 1 -- 172.21.15.7:0/2917989274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623402a060 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6225ffb640 1 -- 172.21.15.7:0/2917989274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6234015c90 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f623400e030 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6225ffb640 1 -- 172.21.15.7:0/2917989274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6234031050 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 10 client.16518 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.552+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56130fe3b7d0 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6225ffb640 1 -- 172.21.15.7:0/2917989274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f623400e030 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6225ffb640 1 -- 172.21.15.7:0/2917989274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f623401b8b0 con 0x5613100fee50 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6247399740 10 client.16518 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6247399740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6247399740 20 client.16518 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6247399740 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56131010f380 0x5613101117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6247399740 4 client.16518 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613100bb3e0 con 0x56131010f380 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f624254f640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56131010f380 0x5613101117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f6225ffb640 10 client.16518 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.553+0000 7f624254f640 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56131010f380 0x5613101117b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad97b7e750f95eac server_cookie=84a61fd35a507f99 in_seq=0 out_seq=0 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.554+0000 7f6225ffb640 1 -- 172.21.15.7:0/2917989274 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613100bb3e0 con 0x56131010f380 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.554+0000 7f6225ffb640 10 client.16518 handle_command_reply: tid=0 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.554+0000 7f6247399740 1 client.16518 shutdown 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 msgr2=0x56131010acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 0x56131010acc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f623400a010 tx=0x7f623401a3d0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 shutdown_connections 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56131010f380 0x5613101117b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100fee50 0x56131010acc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f9ce0 0x561310103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6225ffb640 0 client.16518 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 --2- 172.21.15.7:0/2917989274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 >> 172.21.15.7:0/2917989274 conn(0x5613100f4860 msgr2=0x5613100f3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 shutdown_connections 2024-03-19T00:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 1 -- 172.21.15.7:0/2917989274 wait complete. 2024-03-19T00:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.555+0000 7f6247399740 20 client.16518 trim_cache size 0 max 16384 2024-03-19T00:42:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 Processor -- start 2024-03-19T00:42:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 -- start start 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x56131010ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x561310103c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1acbe0 con 0x5613100f92f0 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1aa4b0 con 0x5613100f9ce0 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6247399740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1a8300 con 0x5613100fee50 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53326/0 (socket says 172.21.15.7:53326) 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 -- 172.21.15.7:0/543708165 learned_addr learned my addr 172.21.15.7:0/543708165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6241d4e640 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x56131010ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6242d50640 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x561310103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 -- 172.21.15.7:0/543708165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 msgr2=0x561310103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x561310103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 -- 172.21.15.7:0/543708165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 msgr2=0x56131010ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x56131010ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f624254f640 1 -- 172.21.15.7:0/543708165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613100ca880 con 0x5613100f92f0 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.568+0000 7f6242d50640 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x561310103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f624254f640 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x561310103740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f622c006010 tx=0x7f622c03abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a81d1b737d7f2cc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6227fff640 1 -- 172.21.15.7:0/543708165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c005460 con 0x5613100f92f0 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6227fff640 1 -- 172.21.15.7:0/543708165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f622c0bdc70 con 0x5613100f92f0 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6227fff640 1 -- 172.21.15.7:0/543708165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c0ee7e0 con 0x5613100f92f0 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 -- 172.21.15.7:0/543708165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 msgr2=0x561310103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x561310103740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f622c006010 tx=0x7f622c03abc0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 -- 172.21.15.7:0/543708165 shutdown_connections 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100fee50 0x561310103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100f9ce0 0x56131010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 --2- 172.21.15.7:0/543708165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f92f0 0x561310103740 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 -- 172.21.15.7:0/543708165 >> 172.21.15.7:0/543708165 conn(0x5613100f4860 msgr2=0x5613100f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 -- 172.21.15.7:0/543708165 shutdown_connections 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.569+0000 7f6247399740 1 -- 172.21.15.7:0/543708165 wait complete. 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 -- start start 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 Processor -- start 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f92f0 0x561310108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 0x561310108580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100fee50 0x561310109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1a8970 con 0x5613100f9ce0 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1a9de0 con 0x5613100fee50 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1a9040 con 0x5613100f92f0 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6247399740 10 client.0 _ll_register_callbacks cb 0x7f623400a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 0x561310108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 0x561310108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53336/0 (socket says 172.21.15.7:53336) 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 -- 172.21.15.7:0/1323753541 learned_addr learned my addr 172.21.15.7:0/1323753541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6242d50640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100fee50 0x561310109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f624254f640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f92f0 0x561310108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 -- 172.21.15.7:0/1323753541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f92f0 msgr2=0x561310108040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f92f0 0x561310108040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 -- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100fee50 msgr2=0x561310109520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100fee50 0x561310109520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 -- 172.21.15.7:0/1323753541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561310109b10 con 0x5613100f9ce0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f624254f640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f92f0 0x561310108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6241d4e640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 0x561310108580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6238003de0 tx=0x7f6238004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8adbb9ae96d26b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.570+0000 7f6225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6225ffb640 1 -- 172.21.15.7:0/1323753541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623802d060 con 0x5613100f9ce0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6225ffb640 1 -- 172.21.15.7:0/1323753541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6238028410 con 0x5613100f9ce0 2024-03-19T00:42:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f622c0bdc70 con 0x5613100f9ce0 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6225ffb640 1 -- 172.21.15.7:0/1323753541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6238034050 con 0x5613100f9ce0 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 10 client.16524 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56131010aeb0 con 0x5613100f9ce0 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6225ffb640 1 -- 172.21.15.7:0/1323753541 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f622c0bdc70 con 0x5613100f9ce0 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6225ffb640 1 -- 172.21.15.7:0/1323753541 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f623801d590 con 0x5613100f9ce0 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 10 client.16524 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 20 client.16524 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561310175e60 0x561310178280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 4 client.16524 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56131006a9f0 con 0x561310175e60 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.571+0000 7f624254f640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561310175e60 0x561310178280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f624254f640 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561310175e60 0x561310178280 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98c142050cf0a0 server_cookie=a4c58b50b7672cf9 in_seq=0 out_seq=0 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6225ffb640 10 client.16524 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6225ffb640 1 -- 172.21.15.7:0/1323753541 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56131006a9f0 con 0x561310175e60 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6225ffb640 10 client.16524 handle_command_reply: tid=0 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 client.16524 shutdown 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 msgr2=0x561310108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 0x561310108580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6238003de0 tx=0x7f6238004730 comp rx=0 tx=0).stop 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 shutdown_connections 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613100fee50 0x561310109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x561310175e60 0x561310178280 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613100f9ce0 0x561310108580 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6225ffb640 0 client.16524 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 --2- 172.21.15.7:0/1323753541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5613100f92f0 0x561310108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 >> 172.21.15.7:0/1323753541 conn(0x5613100f4860 msgr2=0x5613100fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 shutdown_connections 2024-03-19T00:42:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 1 -- 172.21.15.7:0/1323753541 wait complete. 2024-03-19T00:42:48.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.572+0000 7f6247399740 20 client.16524 trim_cache size 0 max 16384 2024-03-19T00:42:48.581 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62280a2880 msgr2=0x7f62280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62280a2880 0x7f62280a4d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f62380095f0 tx=0x7f6238005f90 comp rx=0 tx=0).stop 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 msgr2=0x7f623c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c197af0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f622c000c00 tx=0x7f622c004040 comp rx=0 tx=0).stop 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 shutdown_connections 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62280a2880 0x7f62280a4d40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c1033e0 0x7f623c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c102a20 0x7f623c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 --2- 172.21.15.7:0/2560873177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c101830 0x7f623c197af0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 >> 172.21.15.7:0/2560873177 conn(0x7f623c0fcd60 msgr2=0x7f623c105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 shutdown_connections 2024-03-19T00:42:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:48.573+0000 7f62447b3640 1 -- 172.21.15.7:0/2560873177 wait complete. 2024-03-19T00:42:49.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:48 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2459063641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:42:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:48 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2459063641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:42:49.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:48 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2459063641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:42:49.585 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:42:49.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:42:49.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:42:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 -- 172.21.15.7:0/2765450437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd181000f0 msgr2=0x7efd18100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 --2- 172.21.15.7:0/2765450437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd181000f0 0x7efd18100510 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efd08009900 tx=0x7efd0802bc80 comp rx=0 tx=0).stop 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 -- 172.21.15.7:0/2765450437 shutdown_connections 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 --2- 172.21.15.7:0/2765450437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 0x7efd18101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 --2- 172.21.15.7:0/2765450437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd18100a50 0x7efd18100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 --2- 172.21.15.7:0/2765450437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd181000f0 0x7efd18100510 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 -- 172.21.15.7:0/2765450437 >> 172.21.15.7:0/2765450437 conn(0x7efd180fbcb0 msgr2=0x7efd180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 -- 172.21.15.7:0/2765450437 shutdown_connections 2024-03-19T00:42:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 -- 172.21.15.7:0/2765450437 wait complete. 2024-03-19T00:42:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 Processor -- start 2024-03-19T00:42:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.704+0000 7efd1de75640 1 -- start start 2024-03-19T00:42:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd1de75640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd181000f0 0x7efd18199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd1de75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 0x7efd1819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd1de75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 0x7efd181a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd1de75640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd1810f560 con 0x7efd18100a50 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd1de75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd1810f3e0 con 0x7efd181013f0 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd1de75640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd1810f6e0 con 0x7efd181000f0 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 0x7efd1819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd181000f0 0x7efd18199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 0x7efd1819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53350/0 (socket says 172.21.15.7:53350) 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 -- 172.21.15.7:0/851512866 learned_addr learned my addr 172.21.15.7:0/851512866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd17fff640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 0x7efd181a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 -- 172.21.15.7:0/851512866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd181000f0 msgr2=0x7efd18199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd181000f0 0x7efd18199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 -- 172.21.15.7:0/851512866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 msgr2=0x7efd181a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 0x7efd181a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 -- 172.21.15.7:0/851512866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd181a1900 con 0x7efd18100a50 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd177fe640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd181000f0 0x7efd18199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd17fff640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 0x7efd181a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd16ffd640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 0x7efd1819a230 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00e9f0 tx=0x7efd0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd14ff9640 1 -- 172.21.15.7:0/851512866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00cd80 con 0x7efd18100a50 2024-03-19T00:42:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.705+0000 7efd14ff9640 1 -- 172.21.15.7:0/851512866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd0c0054d0 con 0x7efd18100a50 2024-03-19T00:42:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.706+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd181a1bf0 con 0x7efd18100a50 2024-03-19T00:42:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.706+0000 7efd14ff9640 1 -- 172.21.15.7:0/851512866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c010490 con 0x7efd18100a50 2024-03-19T00:42:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.706+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd181a20f0 con 0x7efd18100a50 2024-03-19T00:42:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd14ff9640 1 -- 172.21.15.7:0/851512866 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7efd0c005000 con 0x7efd18100a50 2024-03-19T00:42:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 Processor -- start 2024-03-19T00:42:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 -- start start 2024-03-19T00:42:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5559a6948160 con 0x5559a69e8f00 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5559a68f0e10 con 0x5559a69e33a0 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.708+0000 7efd20a5b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5559a6944200 con 0x5559a69e3d90 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd14ff9640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd000a2800 0x7efd000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69eb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53354/0 (socket says 172.21.15.7:53354) 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd16ffd640 1 -- 172.21.15.7:0/2020242270 learned_addr learned my addr 172.21.15.7:0/2020242270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd17fff640 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd17fff640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd000a2800 0x7efd000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 -- 172.21.15.7:0/2020242270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 msgr2=0x5559a69e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd14ff9640 1 -- 172.21.15.7:0/851512866 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efd0c0cf230 con 0x7efd18100a50 2024-03-19T00:42:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 -- 172.21.15.7:0/2020242270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 msgr2=0x5559a69eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 -- 172.21.15.7:0/2020242270 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a67247e0 con 0x5559a69e3d90 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd177fe640 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69e8890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd08000c00 tx=0x7efd08009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd3917a969a06960 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efd17fff640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd000a2800 0x7efd000a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd100099e0 tx=0x7efd10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efcfdffb640 1 -- 172.21.15.7:0/2020242270 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08045020 con 0x5559a69e3d90 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efcfdffb640 1 -- 172.21.15.7:0/2020242270 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd08005040 con 0x5559a69e3d90 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.709+0000 7efcfdffb640 1 -- 172.21.15.7:0/2020242270 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080048c0 con 0x5559a69e3d90 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 -- 172.21.15.7:0/2020242270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 msgr2=0x5559a69e8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69e8890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd08000c00 tx=0x7efd08009610 comp rx=0 tx=0).stop 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 -- 172.21.15.7:0/2020242270 shutdown_connections 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69e8890 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 --2- 172.21.15.7:0/2020242270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 -- 172.21.15.7:0/2020242270 >> 172.21.15.7:0/2020242270 conn(0x5559a69de910 msgr2=0x5559a69ded10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 -- 172.21.15.7:0/2020242270 shutdown_connections 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 -- 172.21.15.7:0/2020242270 wait complete. 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.710+0000 7efd20a5b740 1 -- start start 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 Processor -- start 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e8f00 0x5559a69f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5559a69ebe50 con 0x5559a69e3d90 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5559a69041a0 con 0x5559a69e33a0 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5559a69ebfd0 con 0x5559a69e8f00 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd20a5b740 10 client.0 _ll_register_callbacks cb 0x5559a665bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53366/0 (socket says 172.21.15.7:53366) 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69ed7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54186/0 (socket says 172.21.15.7:54186) 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd16ffd640 1 -- 172.21.15.7:0/1625432091 learned_addr learned my addr 172.21.15.7:0/1625432091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd17fff640 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e8f00 0x5559a69f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 -- 172.21.15.7:0/1625432091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e8f00 msgr2=0x5559a69f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e8f00 0x5559a69f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 -- 172.21.15.7:0/1625432091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 msgr2=0x5559a69edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69edd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.711+0000 7efd177fe640 1 -- 172.21.15.7:0/1625432091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a6784f70 con 0x5559a69e33a0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd177fe640 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69ed7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x5559a69df880 tx=0x7efd08004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc51935dee0125a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efcf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efcf6ffd640 1 -- 172.21.15.7:0/1625432091 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08045040 con 0x5559a69e33a0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd08005040 con 0x5559a69e33a0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efcf6ffd640 1 -- 172.21.15.7:0/1625432091 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efd08041480 con 0x5559a69e33a0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efcf6ffd640 1 -- 172.21.15.7:0/1625432091 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd0805b050 con 0x5559a69e33a0 2024-03-19T00:42:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 10 client.26384 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559a661c170 con 0x5559a69e33a0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efcf6ffd640 1 -- 172.21.15.7:0/1625432091 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd08005040 con 0x5559a69e33a0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efcf6ffd640 1 -- 172.21.15.7:0/1625432091 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd0804a4e0 con 0x5559a69e33a0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 10 client.26384 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 10 client.26384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 20 client.26384 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a69f91e0 0x5559a69fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 4 client.26384 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.712+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559a69556a0 con 0x5559a69f91e0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.713+0000 7efd16ffd640 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a69f91e0 0x5559a69fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.713+0000 7efd16ffd640 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a69f91e0 0x5559a69fb5d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc36272ebdd060d server_cookie=555d5869f13b2acd in_seq=0 out_seq=0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.713+0000 7efcf6ffd640 10 client.26384 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.713+0000 7efcf6ffd640 1 -- 172.21.15.7:0/1625432091 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5559a69556a0 con 0x5559a69f91e0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efcf6ffd640 10 client.26384 handle_command_reply: tid=0 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 client.26384 shutdown 2024-03-19T00:42:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 msgr2=0x5559a69ed7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69ed7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x5559a69df880 tx=0x7efd08004970 comp rx=0 tx=0).stop 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 shutdown_connections 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e8f00 0x5559a69f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a69f91e0 0x5559a69fb5d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 --2- 172.21.15.7:0/1625432091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69ed7e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efcf6ffd640 0 client.26384 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 >> 172.21.15.7:0/1625432091 conn(0x5559a69de910 msgr2=0x5559a69ddb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 shutdown_connections 2024-03-19T00:42:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 1 -- 172.21.15.7:0/1625432091 wait complete. 2024-03-19T00:42:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.714+0000 7efd20a5b740 20 client.26384 trim_cache size 0 max 16384 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 Processor -- start 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 -- start start 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69ddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aed30 con 0x5559a69e8f00 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd181ac600 con 0x5559a69e33a0 2024-03-19T00:42:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd20a5b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aa450 con 0x5559a69e3d90 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53380/0 (socket says 172.21.15.7:53380) 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 -- 172.21.15.7:0/3348755085 learned_addr learned my addr 172.21.15.7:0/3348755085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd16ffd640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69ddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd177fe640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 -- 172.21.15.7:0/3348755085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 msgr2=0x5559a69ddef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69ddef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 -- 172.21.15.7:0/3348755085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 msgr2=0x5559a69dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 -- 172.21.15.7:0/3348755085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a69fb940 con 0x5559a69e8f00 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd16ffd640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69ddef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd177fe640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.732+0000 7efd17fff640 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69f4cb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efd10011750 tx=0x7efd1000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd38ed89140579a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efcfd7fa640 1 -- 172.21.15.7:0/3348755085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd100045d0 con 0x5559a69e8f00 2024-03-19T00:42:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efcfd7fa640 1 -- 172.21.15.7:0/3348755085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd10004770 con 0x5559a69e8f00 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efcfd7fa640 1 -- 172.21.15.7:0/3348755085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1001fe50 con 0x5559a69e8f00 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 -- 172.21.15.7:0/3348755085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 msgr2=0x5559a69f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69f4cb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efd10011750 tx=0x7efd1000e000 comp rx=0 tx=0).stop 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 -- 172.21.15.7:0/3348755085 shutdown_connections 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e8f00 0x5559a69f4cb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e3d90 0x5559a69ddef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3348755085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e33a0 0x5559a69dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 -- 172.21.15.7:0/3348755085 >> 172.21.15.7:0/3348755085 conn(0x5559a69de910 msgr2=0x5559a69dd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 -- 172.21.15.7:0/3348755085 shutdown_connections 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 -- 172.21.15.7:0/3348755085 wait complete. 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.733+0000 7efd20a5b740 1 -- start start 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 Processor -- start 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e33a0 0x5559a69f4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e8f00 0x5559a69f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aaac0 con 0x5559a69e3d90 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd181abf30 con 0x5559a69e8f00 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd181ab190 con 0x5559a69e33a0 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd20a5b740 10 client.0 _ll_register_callbacks cb 0x7efd0c096140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53384/0 (socket says 172.21.15.7:53384) 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd17fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e8f00 0x5559a69f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd16ffd640 1 -- 172.21.15.7:0/3049531901 learned_addr learned my addr 172.21.15.7:0/3049531901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd177fe640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e33a0 0x5559a69f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd17fff640 1 -- 172.21.15.7:0/3049531901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e33a0 msgr2=0x5559a69f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd17fff640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e33a0 0x5559a69f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd17fff640 1 -- 172.21.15.7:0/3049531901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 msgr2=0x5559a69f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.734+0000 7efd17fff640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.735+0000 7efd17fff640 1 -- 172.21.15.7:0/3049531901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a69f35b0 con 0x5559a69e8f00 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.735+0000 7efd177fe640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e33a0 0x5559a69f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.735+0000 7efd16ffd640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.735+0000 7efd17fff640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e8f00 0x5559a69f2d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd10011720 tx=0x7efd10011780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668c3d11e7cb8ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.735+0000 7efcf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.737+0000 7efcf6ffd640 1 -- 172.21.15.7:0/3049531901 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10038020 con 0x5559a69e8f00 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.737+0000 7efcf6ffd640 1 -- 172.21.15.7:0/3049531901 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efd1001fd50 con 0x5559a69e8f00 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.737+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd10004770 con 0x5559a69e8f00 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efcf6ffd640 1 -- 172.21.15.7:0/3049531901 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd10043050 con 0x5559a69e8f00 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 10 client.26390 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559a6a5ff40 con 0x5559a69e8f00 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efcf6ffd640 1 -- 172.21.15.7:0/3049531901 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd10004770 con 0x5559a69e8f00 2024-03-19T00:42:49.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efcf6ffd640 1 -- 172.21.15.7:0/3049531901 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd1001d040 con 0x5559a69e8f00 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 10 client.26390 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 10 client.26390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 20 client.26390 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a6a60970 0x5559a6a62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 4 client.26390 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5559a694f9f0 con 0x5559a6a60970 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.738+0000 7efd177fe640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a6a60970 0x5559a6a62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.739+0000 7efcf6ffd640 10 client.26390 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.739+0000 7efd177fe640 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a6a60970 0x5559a6a62d90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20b23228a3582e1 server_cookie=1dc2d9eecfd8ac44 in_seq=0 out_seq=0 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.739+0000 7efcf6ffd640 1 -- 172.21.15.7:0/3049531901 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559a694f9f0 con 0x5559a6a60970 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.739+0000 7efcf6ffd640 10 client.26390 handle_command_reply: tid=0 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.739+0000 7efd20a5b740 1 client.26390 shutdown 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e8f00 msgr2=0x5559a69f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e8f00 0x5559a69f2d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd10011720 tx=0x7efd10011780 comp rx=0 tx=0).stop 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 shutdown_connections 2024-03-19T00:42:49.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559a69e8f00 0x5559a69f2d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5559a6a60970 0x5559a6a62d90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559a69e3d90 0x5559a69f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 --2- 172.21.15.7:0/3049531901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a69e33a0 0x5559a69f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efcf6ffd640 0 client.26390 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 >> 172.21.15.7:0/3049531901 conn(0x5559a69de910 msgr2=0x5559a69ddb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 shutdown_connections 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 1 -- 172.21.15.7:0/3049531901 wait complete. 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd20a5b740 20 client.26390 trim_cache size 0 max 16384 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "19d58dcd-a166-45c1-80cc-f94d6e848aba", 2024-03-19T00:42:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:42:49.746 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:42:49.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd000a2800 msgr2=0x7efd000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd000a2800 0x7efd000a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd100099e0 tx=0x7efd10009500 comp rx=0 tx=0).stop 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 msgr2=0x7efd1819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 0x7efd1819a230 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00e9f0 tx=0x7efd0c00edf0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 shutdown_connections 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efd000a2800 0x7efd000a4cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd181013f0 0x7efd181a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd18100a50 0x7efd1819a230 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 --2- 172.21.15.7:0/851512866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd181000f0 0x7efd18199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.740+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 >> 172.21.15.7:0/851512866 conn(0x7efd180fbcb0 msgr2=0x7efd180fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.741+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 shutdown_connections 2024-03-19T00:42:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.741+0000 7efd1de75640 1 -- 172.21.15.7:0/851512866 wait complete. 2024-03-19T00:42:49.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:42:49.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 -- 172.21.15.7:0/2525391648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 msgr2=0x7f44b8100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 --2- 172.21.15.7:0/2525391648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b8100b60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009ca0 tx=0x7f44b402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 -- 172.21.15.7:0/2525391648 shutdown_connections 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 --2- 172.21.15.7:0/2525391648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b81022f0 0x7f44b8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 --2- 172.21.15.7:0/2525391648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b8101930 0x7f44b8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 --2- 172.21.15.7:0/2525391648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b8100b60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 -- 172.21.15.7:0/2525391648 >> 172.21.15.7:0/2525391648 conn(0x7f44b80fbcd0 msgr2=0x7f44b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 -- 172.21.15.7:0/2525391648 shutdown_connections 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 -- 172.21.15.7:0/2525391648 wait complete. 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.856+0000 7f44bf768640 1 Processor -- start 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 -- start start 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b806aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b8101930 0x7f44b819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b81022f0 0x7f44b819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810c240 con 0x7f44b8100740 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810c0c0 con 0x7f44b81022f0 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bf768640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810c3c0 con 0x7f44b8101930 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b806aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b806aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53400/0 (socket says 172.21.15.7:53400) 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 -- 172.21.15.7:0/691605716 learned_addr learned my addr 172.21.15.7:0/691605716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bcd03640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b8101930 0x7f44b819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bdd05640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b81022f0 0x7f44b819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 -- 172.21.15.7:0/691605716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b8101930 msgr2=0x7f44b819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b8101930 0x7f44b819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 -- 172.21.15.7:0/691605716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b81022f0 msgr2=0x7f44b819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b81022f0 0x7f44b819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 -- 172.21.15.7:0/691605716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b819f700 con 0x7f44b8100740 2024-03-19T00:42:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44bd504640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b806aa30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f44b4032380 tx=0x7f44b4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.857+0000 7f44ae7fc640 1 -- 172.21.15.7:0/691605716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4041020 con 0x7f44b8100740 2024-03-19T00:42:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.858+0000 7f44ae7fc640 1 -- 172.21.15.7:0/691605716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b4003e20 con 0x7f44b8100740 2024-03-19T00:42:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.858+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b819f930 con 0x7f44b8100740 2024-03-19T00:42:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.858+0000 7f44ae7fc640 1 -- 172.21.15.7:0/691605716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4039040 con 0x7f44b8100740 2024-03-19T00:42:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.858+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b819fe20 con 0x7f44b8100740 2024-03-19T00:42:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.860+0000 7f44ae7fc640 1 -- 172.21.15.7:0/691605716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f44b4044020 con 0x7f44b8100740 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.860+0000 7f44c242a740 1 Processor -- start 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.860+0000 7f44c242a740 1 -- start start 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.860+0000 7f44c242a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f1ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44c242a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f21f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44c242a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f24a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44c242a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556211e81500 con 0x556211f1ca50 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44c242a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556211e29be0 con 0x556211f225b0 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44c242a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556211e7d5a0 con 0x556211f1d440 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f1ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f1ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53402/0 (socket says 172.21.15.7:53402) 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 -- 172.21.15.7:0/2082835262 learned_addr learned my addr 172.21.15.7:0/2082835262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bd504640 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f21f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bcd03640 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f24a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44ae7fc640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f44a40a2800 0x7f44a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 -- 172.21.15.7:0/2082835262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 msgr2=0x556211f21f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f21f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 -- 172.21.15.7:0/2082835262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 msgr2=0x556211f24a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f24a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 -- 172.21.15.7:0/2082835262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556211e4b000 con 0x556211f1ca50 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f44a40a2800 0x7f44a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44ae7fc640 1 -- 172.21.15.7:0/691605716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f44b40f0e10 con 0x7f44b8100740 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f1ce70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009da0 tx=0x7f44b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c6bc7d8bea1b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44a37fe640 1 -- 172.21.15.7:0/2082835262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0017050 con 0x556211f1ca50 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44a37fe640 1 -- 172.21.15.7:0/2082835262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b00268f0 con 0x556211f1ca50 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44bdd05640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f44a40a2800 0x7f44a40a4cc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44b0023000 tx=0x7f44b0024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.861+0000 7f44a37fe640 1 -- 172.21.15.7:0/2082835262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000f620 con 0x556211f1ca50 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 -- 172.21.15.7:0/2082835262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 msgr2=0x556211f1ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f1ce70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009da0 tx=0x7f44b0011810 comp rx=0 tx=0).stop 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 -- 172.21.15.7:0/2082835262 shutdown_connections 2024-03-19T00:42:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f24a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f21f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 --2- 172.21.15.7:0/2082835262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f1ce70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 -- 172.21.15.7:0/2082835262 >> 172.21.15.7:0/2082835262 conn(0x556211f17fc0 msgr2=0x556211f183c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 -- 172.21.15.7:0/2082835262 shutdown_connections 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 -- 172.21.15.7:0/2082835262 wait complete. 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.862+0000 7f44c242a740 1 -- start start 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 Processor -- start 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f26ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f27420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556211f25560 con 0x556211f225b0 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556211e3d160 con 0x556211f1d440 2024-03-19T00:42:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556211f256e0 con 0x556211f1ca50 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44c242a740 10 client.0 _ll_register_callbacks cb 0x556211e79a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bcd03640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f27420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f2e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53404/0 (socket says 172.21.15.7:53404) 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 -- 172.21.15.7:0/3675366305 learned_addr learned my addr 172.21.15.7:0/3675366305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bd504640 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f26ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 -- 172.21.15.7:0/3675366305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 msgr2=0x556211f26ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f26ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 -- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 msgr2=0x556211f27420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f27420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 -- 172.21.15.7:0/3675366305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556211c5f7d0 con 0x556211f225b0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.863+0000 7f44bdd05640 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f2e460 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44b0005c20 tx=0x7f44b0011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36900e58fcfc6bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44a0ff9640 1 -- 172.21.15.7:0/3675366305 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b002b040 con 0x556211f225b0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44a0ff9640 1 -- 172.21.15.7:0/3675366305 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f44b000f600 con 0x556211f225b0 2024-03-19T00:42:49.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b00268f0 con 0x556211f225b0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44a0ff9640 1 -- 172.21.15.7:0/3675366305 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44b003f050 con 0x556211f225b0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 10 client.16554 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556211b953f0 con 0x556211f225b0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44a0ff9640 1 -- 172.21.15.7:0/3675366305 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b00268f0 con 0x556211f225b0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44a0ff9640 1 -- 172.21.15.7:0/3675366305 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f44b002e030 con 0x556211f225b0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 10 client.16554 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 20 client.16554 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f32c70 0x556211f35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 4 client.16554 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556211e8ea40 con 0x556211f32c70 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.864+0000 7f44bd504640 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f32c70 0x556211f35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44a0ff9640 10 client.16554 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44bd504640 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f32c70 0x556211f35080 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9cfccf86bf9f6e server_cookie=5548fe388131c3e9 in_seq=0 out_seq=0 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44a0ff9640 1 -- 172.21.15.7:0/3675366305 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556211e8ea40 con 0x556211f32c70 2024-03-19T00:42:49.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44a0ff9640 10 client.16554 handle_command_reply: tid=0 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44c242a740 1 client.16554 shutdown 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 msgr2=0x556211f2e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.865+0000 7f44c242a740 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f2e460 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44b0005c20 tx=0x7f44b0011dc0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 shutdown_connections 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f32c70 0x556211f35080 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f2e460 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f27420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 --2- 172.21.15.7:0/3675366305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f26ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44a0ff9640 0 client.16554 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 >> 172.21.15.7:0/3675366305 conn(0x556211f17fc0 msgr2=0x556211f171e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 shutdown_connections 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 1 -- 172.21.15.7:0/3675366305 wait complete. 2024-03-19T00:42:49.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.866+0000 7f44c242a740 20 client.16554 trim_cache size 0 max 16384 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 Processor -- start 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 -- start start 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f16e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81acb90 con 0x556211f225b0 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81aa460 con 0x556211f1d440 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44c242a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81a82b0 con 0x556211f1ca50 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44bdd05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44bdd05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f173d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53410/0 (socket says 172.21.15.7:53410) 2024-03-19T00:42:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44bdd05640 1 -- 172.21.15.7:0/3833379672 learned_addr learned my addr 172.21.15.7:0/3833379672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44bd504640 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.883+0000 7f44bcd03640 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f16e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44bdd05640 1 -- 172.21.15.7:0/3833379672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 msgr2=0x556211f16950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44bdd05640 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f16950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44bdd05640 1 -- 172.21.15.7:0/3833379672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 msgr2=0x556211f16e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44bdd05640 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f16e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44bdd05640 1 -- 172.21.15.7:0/3833379672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556211b550f0 con 0x556211f225b0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44bdd05640 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f173d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f44b0011ef0 tx=0x7f44b001bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afbd2e9c9ba29fa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44a2ffd640 1 -- 172.21.15.7:0/3833379672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0017040 con 0x556211f225b0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44a2ffd640 1 -- 172.21.15.7:0/3833379672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b0036070 con 0x556211f225b0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44a2ffd640 1 -- 172.21.15.7:0/3833379672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0031980 con 0x556211f225b0 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 -- 172.21.15.7:0/3833379672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 msgr2=0x556211f173d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f173d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f44b0011ef0 tx=0x7f44b001bc10 comp rx=0 tx=0).stop 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 -- 172.21.15.7:0/3833379672 shutdown_connections 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f225b0 0x556211f173d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f1d440 0x556211f16e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 --2- 172.21.15.7:0/3833379672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1ca50 0x556211f16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 -- 172.21.15.7:0/3833379672 >> 172.21.15.7:0/3833379672 conn(0x556211f17fc0 msgr2=0x556211f26ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.884+0000 7f44c242a740 1 -- 172.21.15.7:0/3833379672 shutdown_connections 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 -- 172.21.15.7:0/3833379672 wait complete. 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 -- start start 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 Processor -- start 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f94420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f94960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f9b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81a8920 con 0x556211f1ca50 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81a9d90 con 0x556211f225b0 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81a8ff0 con 0x556211f1d440 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44bd504640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f94420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44c242a740 10 client.0 _ll_register_callbacks cb 0x7f44b0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44bd504640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f94420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53424/0 (socket says 172.21.15.7:53424) 2024-03-19T00:42:49.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44bd504640 1 -- 172.21.15.7:0/145525599 learned_addr learned my addr 172.21.15.7:0/145525599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44bdd05640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f9b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.885+0000 7f44bcd03640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f94960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44bd504640 1 -- 172.21.15.7:0/145525599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 msgr2=0x556211f94960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44bd504640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f94960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44bd504640 1 -- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 msgr2=0x556211f9b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44bd504640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f9b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44bd504640 1 -- 172.21.15.7:0/145525599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556211f9bf90 con 0x556211f1ca50 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44bd504640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f94420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f44b4003fd0 tx=0x7f44b40b9820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254e3d0651d85f4c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44a0ff9640 1 -- 172.21.15.7:0/145525599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40bdb50 con 0x556211f1ca50 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44a0ff9640 1 -- 172.21.15.7:0/145525599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f44b40347b0 con 0x556211f1ca50 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b0036070 con 0x556211f1ca50 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44a0ff9640 1 -- 172.21.15.7:0/145525599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44b40bd450 con 0x556211f1ca50 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 10 client.16560 fetch_fsmap learned FSMap version 27 2024-03-19T00:42:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556211f9d370 con 0x556211f1ca50 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44a0ff9640 1 -- 172.21.15.7:0/145525599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b0036070 con 0x556211f1ca50 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44a0ff9640 1 -- 172.21.15.7:0/145525599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f44b403c030 con 0x556211f1ca50 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 10 client.16560 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 20 client.16560 populate_metadata read hostname 'smithi007' 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f9dd20 0x556211fa0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 4 client.16560 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.886+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556211e88d90 con 0x556211f9dd20 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44bcd03640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f9dd20 0x556211fa0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44a0ff9640 10 client.16560 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44bcd03640 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f9dd20 0x556211fa0140 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be1bd880b3d9687c server_cookie=1f16dc89af7991a in_seq=0 out_seq=0 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44a0ff9640 1 -- 172.21.15.7:0/145525599 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556211e88d90 con 0x556211f9dd20 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44a0ff9640 10 client.16560 handle_command_reply: tid=0 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 client.16560 shutdown 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 msgr2=0x556211f94420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f94420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f44b4003fd0 tx=0x7f44b40b9820 comp rx=0 tx=0).stop 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 shutdown_connections 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556211f225b0 0x556211f9b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556211f1d440 0x556211f94960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556211f9dd20 0x556211fa0140 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 --2- 172.21.15.7:0/145525599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556211f1ca50 0x556211f94420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44a0ff9640 0 client.16560 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 >> 172.21.15.7:0/145525599 conn(0x556211f17fc0 msgr2=0x556211f1b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 shutdown_connections 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 1 -- 172.21.15.7:0/145525599 wait complete. 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.887+0000 7f44c242a740 20 client.16560 trim_cache size 0 max 16384 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:42:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "19d58dcd-a166-45c1-80cc-f94d6e848aba": { 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "19d58dcd-a166-45c1-80cc-f94d6e848aba", 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:42:49.897 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:42:49.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f44a40a2800 msgr2=0x7f44a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f44a40a2800 0x7f44a40a4cc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44b0023000 tx=0x7f44b0024000 comp rx=0 tx=0).stop 2024-03-19T00:42:49.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 msgr2=0x7f44b806aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:42:49.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b806aa30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f44b4032380 tx=0x7f44b4032820 comp rx=0 tx=0).stop 2024-03-19T00:42:49.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 shutdown_connections 2024-03-19T00:42:49.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f44a40a2800 0x7f44a40a4cc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44b81022f0 0x7f44b819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b8101930 0x7f44b819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 --2- 172.21.15.7:0/691605716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44b8100740 0x7f44b806aa30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:42:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 >> 172.21.15.7:0/691605716 conn(0x7f44b80fbcd0 msgr2=0x7f44b8104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:42:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 shutdown_connections 2024-03-19T00:42:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:42:49.888+0000 7f44bf768640 1 -- 172.21.15.7:0/691605716 wait complete. 2024-03-19T00:42:49.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:49 smithi007 ceph-mon[62854]: pgmap v497: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:49.900 INFO:tasks.cephfs.filesystem:scrub status for tag:19d58dcd-a166-45c1-80cc-f94d6e848aba - {'path': '/', 'tag': '19d58dcd-a166-45c1-80cc-f94d6e848aba', 'options': 'recursive,force'} 2024-03-19T00:42:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:49 smithi204 ceph-mon[66572]: pgmap v497: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:49 smithi044 ceph-mon[66366]: pgmap v497: 129 pgs: 129 active+clean; 1.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:50 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:42:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:50 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:50 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:42:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:50 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:42:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:50 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:50 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:50 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:42:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:50 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:50 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:42:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:50 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:42:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:50 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:50 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:50 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:42:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:50 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:50 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:42:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:50 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:42:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:50 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:51.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:50 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:42:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:51 smithi204 ceph-mon[66572]: pgmap v498: 129 pgs: 129 active+clean; 1.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:42:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:51 smithi044 ceph-mon[66366]: pgmap v498: 129 pgs: 129 active+clean; 1.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:42:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:51 smithi007 ceph-mon[62854]: pgmap v498: 129 pgs: 129 active+clean; 1.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:42:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:53 smithi204 ceph-mon[66572]: pgmap v499: 129 pgs: 129 active+clean; 1.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:53 smithi044 ceph-mon[66366]: pgmap v499: 129 pgs: 129 active+clean; 1.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:53 smithi007 ceph-mon[62854]: pgmap v499: 129 pgs: 129 active+clean; 1.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:42:56.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:55 smithi204 ceph-mon[66572]: pgmap v500: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:55 smithi044 ceph-mon[66366]: pgmap v500: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:55 smithi007 ceph-mon[62854]: pgmap v500: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:42:57.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:56 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:42:57.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:56 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:42:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:56 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:42:57.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:56 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:42:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:56 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:42:57.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:56 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:42:58.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:57 smithi204 ceph-mon[66572]: pgmap v501: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:57 smithi044 ceph-mon[66366]: pgmap v501: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:42:58.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:57 smithi007 ceph-mon[62854]: pgmap v501: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:00.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:42:59 smithi204 ceph-mon[66572]: pgmap v502: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:42:59 smithi044 ceph-mon[66366]: pgmap v502: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:42:59 smithi007 ceph-mon[62854]: pgmap v502: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:02.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:01 smithi204 ceph-mon[66572]: pgmap v503: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:02.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:01 smithi044 ceph-mon[66366]: pgmap v503: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:01 smithi007 ceph-mon[62854]: pgmap v503: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:04.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:03 smithi204 ceph-mon[66572]: pgmap v504: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:03 smithi044 ceph-mon[66366]: pgmap v504: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:04.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:03 smithi007 ceph-mon[62854]: pgmap v504: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:43:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:05 smithi204 ceph-mon[66572]: pgmap v505: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:43:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:05 smithi044 ceph-mon[66366]: pgmap v505: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:43:06.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:05 smithi007 ceph-mon[62854]: pgmap v505: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:43:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:07 smithi044 ceph-mon[66366]: pgmap v506: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:08.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:07 smithi007 ceph-mon[62854]: pgmap v506: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:07 smithi204 ceph-mon[66572]: pgmap v506: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:09 smithi044 ceph-mon[66366]: pgmap v507: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:09 smithi007 ceph-mon[62854]: pgmap v507: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:09 smithi204 ceph-mon[66572]: pgmap v507: 129 pgs: 129 active+clean; 1.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:12.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:11 smithi204 ceph-mon[66572]: pgmap v508: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:11 smithi044 ceph-mon[66366]: pgmap v508: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:12.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:11 smithi007 ceph-mon[62854]: pgmap v508: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:13 smithi044 ceph-mon[66366]: pgmap v509: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:13 smithi007 ceph-mon[62854]: pgmap v509: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:13 smithi204 ceph-mon[66572]: pgmap v509: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:15 smithi044 ceph-mon[66366]: pgmap v510: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:43:16.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:15 smithi007 ceph-mon[62854]: pgmap v510: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:43:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:15 smithi204 ceph-mon[66572]: pgmap v510: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:43:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:17 smithi044 ceph-mon[66366]: pgmap v511: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:18.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:17 smithi007 ceph-mon[62854]: pgmap v511: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:17 smithi204 ceph-mon[66572]: pgmap v511: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:19.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:43:19.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3520534793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb0100740 msgr2=0x7f6fb0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3520534793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb0100740 0x7f6fb0100b60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009860 tx=0x7f6fa402bb70 comp rx=0 tx=0).stop 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3520534793 shutdown_connections 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3520534793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb01022f0 0x7f6fb0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3520534793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb0101930 0x7f6fb0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3520534793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb0100740 0x7f6fb0100b60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3520534793 >> 172.21.15.7:0/3520534793 conn(0x7f6fb00fbcb0 msgr2=0x7f6fb00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3520534793 shutdown_connections 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.017+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3520534793 wait complete. 2024-03-19T00:43:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 Processor -- start 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 -- start start 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 0x7f6fb01046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb0101930 0x7f6fb0102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb01022f0 0x7f6fb0103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb010c260 con 0x7f6fb0100740 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb010c0e0 con 0x7f6fb01022f0 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb6ae8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb010c3e0 con 0x7f6fb0101930 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 0x7f6fb01046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 0x7f6fb01046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54394/0 (socket says 172.21.15.7:54394) 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 -- 172.21.15.7:0/3213335064 learned_addr learned my addr 172.21.15.7:0/3213335064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb01022f0 0x7f6fb0103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6faffff640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb0101930 0x7f6fb0102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 -- 172.21.15.7:0/3213335064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb0101930 msgr2=0x7f6fb0102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb0101930 0x7f6fb0102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 -- 172.21.15.7:0/3213335064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb01022f0 msgr2=0x7f6fb0103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb01022f0 0x7f6fb0103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb4884640 1 -- 172.21.15.7:0/3213335064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb0103ae0 con 0x7f6fb0100740 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.018+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb01022f0 0x7f6fb0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.019+0000 7f6fb4884640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 0x7f6fb01046d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4005940 tx=0x7f6fa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.019+0000 7f6fadffb640 1 -- 172.21.15.7:0/3213335064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4041020 con 0x7f6fb0100740 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.019+0000 7f6fadffb640 1 -- 172.21.15.7:0/3213335064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fa40035f0 con 0x7f6fb0100740 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.019+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb01a1ba0 con 0x7f6fb0100740 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.019+0000 7f6fadffb640 1 -- 172.21.15.7:0/3213335064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4039040 con 0x7f6fb0100740 2024-03-19T00:43:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.019+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb01a20f0 con 0x7f6fb0100740 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.021+0000 7f6fadffb640 1 -- 172.21.15.7:0/3213335064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6fa4044020 con 0x7f6fb0100740 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 Processor -- start 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 -- start start 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fadffb640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6fa00a2800 0x7f6fa00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55943f216500 con 0x55943f2b2440 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55943f1bebe0 con 0x55943f2b1a50 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55943f2125a0 con 0x55943f2b75b0 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6faffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb4884640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6fa00a2800 0x7f6fa00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6faffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54406/0 (socket says 172.21.15.7:54406) 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6faffff640 1 -- 172.21.15.7:0/823604929 learned_addr learned my addr 172.21.15.7:0/823604929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb5085640 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.022+0000 7f6fb5085640 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6fadffb640 1 -- 172.21.15.7:0/3213335064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6fa40f0790 con 0x7f6fb0100740 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6faffff640 1 -- 172.21.15.7:0/823604929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 msgr2=0x55943f2b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6faffff640 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2b9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6faffff640 1 -- 172.21.15.7:0/823604929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 msgr2=0x55943f2b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6faffff640 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6faffff640 1 -- 172.21.15.7:0/823604929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943f1e0000 con 0x55943f2b2440 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6fb5085640 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6fb4884640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6fa00a2800 0x7f6fa00a4cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40bca70 tx=0x7f6fa40e77c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6faffff640 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2b6f40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009ca0 tx=0x7f6f9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6834586ce88a45e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6f9affd640 1 -- 172.21.15.7:0/823604929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01e070 con 0x55943f2b2440 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6f9affd640 1 -- 172.21.15.7:0/823604929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f9c019ae0 con 0x55943f2b2440 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.023+0000 7f6f9affd640 1 -- 172.21.15.7:0/823604929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00e8e0 con 0x55943f2b2440 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 -- 172.21.15.7:0/823604929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 msgr2=0x55943f2b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2b6f40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009ca0 tx=0x7f6f9c011810 comp rx=0 tx=0).stop 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 -- 172.21.15.7:0/823604929 shutdown_connections 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2b6f40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/823604929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 -- 172.21.15.7:0/823604929 >> 172.21.15.7:0/823604929 conn(0x55943f2acfc0 msgr2=0x55943f2ad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 -- 172.21.15.7:0/823604929 shutdown_connections 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 -- 172.21.15.7:0/823604929 wait complete. 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 -- start start 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.024+0000 7f6fb97ac740 1 Processor -- start 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55943f2ba560 con 0x55943f2b2440 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55943f1d2160 con 0x55943f2b1a50 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55943f2ba6e0 con 0x55943f2b75b0 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 10 client.0 _ll_register_callbacks cb 0x55943f20ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2bc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54414/0 (socket says 172.21.15.7:54414) 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 -- 172.21.15.7:0/3038052422 learned_addr learned my addr 172.21.15.7:0/3038052422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 -- 172.21.15.7:0/3038052422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 msgr2=0x55943f2c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 -- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 msgr2=0x55943f2bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 -- 172.21.15.7:0/3038052422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943eff47d0 con 0x55943f2b2440 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6faffff640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2bc410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c01c040 tx=0x7f6f9c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7ae72b35d32773 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3038052422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01e050 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3038052422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f9c0052d0 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.025+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9c019ae0 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3038052422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c032050 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 10 client.16572 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55943ef2a3f0 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3038052422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f9c019ae0 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3038052422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6f9c00e6f0 con 0x55943f2b2440 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 10 client.16572 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 20 client.16572 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f2c7ae0 0x55943f2c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 4 client.16572 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:20.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55943f223a40 con 0x55943f2c7ae0 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.026+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f2c7ae0 0x55943f2c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.027+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f2c7ae0 0x55943f2c9f30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5434921b4200dd server_cookie=b78ff77cbb454a9d in_seq=0 out_seq=0 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.027+0000 7f6f8ffff640 10 client.16572 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3038052422 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55943f223a40 con 0x55943f2c7ae0 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6f8ffff640 10 client.16572 handle_command_reply: tid=0 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 client.16572 shutdown 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 msgr2=0x55943f2bc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2bc410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c01c040 tx=0x7f6f9c0098d0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 shutdown_connections 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b75b0 0x55943f2c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f2c7ae0 0x55943f2c9f30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b2440 0x55943f2bc410 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3038052422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6f8ffff640 0 client.16572 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 >> 172.21.15.7:0/3038052422 conn(0x55943f2acfc0 msgr2=0x55943f2ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 shutdown_connections 2024-03-19T00:43:20.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3038052422 wait complete. 2024-03-19T00:43:20.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.028+0000 7f6fb97ac740 20 client.16572 trim_cache size 0 max 16384 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.045+0000 7f6fb97ac740 1 Processor -- start 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 -- start start 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 0x55943f2ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01aed60 con 0x55943f2b75b0 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01ac630 con 0x55943f2b1a50 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01aa480 con 0x55943f2b2440 2024-03-19T00:43:20.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 0x55943f2ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 0x55943f2ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54420/0 (socket says 172.21.15.7:54420) 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 -- 172.21.15.7:0/318449856 learned_addr learned my addr 172.21.15.7:0/318449856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6faffff640 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 -- 172.21.15.7:0/318449856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 msgr2=0x55943f2c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 -- 172.21.15.7:0/318449856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 msgr2=0x55943f2ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 -- 172.21.15.7:0/318449856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943eeea0f0 con 0x55943f2b75b0 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6faffff640 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6fb5085640 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 0x55943f2ab950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8005cc0 tx=0x7f6fa8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebd70dff386fa96 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6f9a7fc640 1 -- 172.21.15.7:0/318449856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004b20 con 0x55943f2b75b0 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6f9a7fc640 1 -- 172.21.15.7:0/318449856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fa8004cc0 con 0x55943f2b75b0 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.046+0000 7f6f9a7fc640 1 -- 172.21.15.7:0/318449856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80054e0 con 0x55943f2b75b0 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 -- 172.21.15.7:0/318449856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 msgr2=0x55943f2ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 0x55943f2ab950 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8005cc0 tx=0x7f6fa8004ab0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 -- 172.21.15.7:0/318449856 shutdown_connections 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b75b0 0x55943f2ab950 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/318449856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b1a50 0x55943f2ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 -- 172.21.15.7:0/318449856 >> 172.21.15.7:0/318449856 conn(0x55943f2acfc0 msgr2=0x55943f2b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 -- 172.21.15.7:0/318449856 shutdown_connections 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 -- 172.21.15.7:0/318449856 wait complete. 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 -- start start 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.047+0000 7f6fb97ac740 1 Processor -- start 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 0x55943f2c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b75b0 0x55943f2c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01aaaf0 con 0x55943f2b1a50 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01abf60 con 0x55943f2b75b0 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01ab1c0 con 0x55943f2b2440 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb97ac740 10 client.0 _ll_register_callbacks cb 0x7f6f9c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 0x55943f2c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 0x55943f2c06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54432/0 (socket says 172.21.15.7:54432) 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 -- 172.21.15.7:0/3151147563 learned_addr learned my addr 172.21.15.7:0/3151147563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b75b0 0x55943f2c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6faffff640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 -- 172.21.15.7:0/3151147563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 msgr2=0x55943f2c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 -- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b75b0 msgr2=0x55943f2c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b75b0 0x55943f2c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 -- 172.21.15.7:0/3151147563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943f2c21c0 con 0x55943f2b1a50 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6faffff640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6fb5085640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 0x55943f2c06d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8011010 tx=0x7f6fa80097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b7e6a1822effc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.048+0000 7f6f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3151147563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8038070 con 0x55943f2b1a50 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa8004cc0 con 0x55943f2b1a50 2024-03-19T00:43:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3151147563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6fa8027650 con 0x55943f2b1a50 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3151147563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fa8045050 con 0x55943f2b1a50 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 10 client.16578 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55943f2c34d0 con 0x55943f2b1a50 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3151147563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fa8004cc0 con 0x55943f2b1a50 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3151147563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6fa800b040 con 0x55943f2b1a50 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 10 client.16578 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 20 client.16578 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f32e240 0x55943f330660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 4 client.16578 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55943f21dd90 con 0x55943f32e240 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.049+0000 7f6faffff640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f32e240 0x55943f330660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6f8ffff640 10 client.16578 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6faffff640 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f32e240 0x55943f330660 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b3ae29b610418b server_cookie=e685ae3f0ad259ce in_seq=0 out_seq=0 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6f8ffff640 1 -- 172.21.15.7:0/3151147563 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55943f21dd90 con 0x55943f32e240 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6f8ffff640 10 client.16578 handle_command_reply: tid=0 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 client.16578 shutdown 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 msgr2=0x55943f2c06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 0x55943f2c06d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8011010 tx=0x7f6fa80097e0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 shutdown_connections 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55943f2b75b0 0x55943f2c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55943f2b2440 0x55943f2c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55943f32e240 0x55943f330660 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6f8ffff640 0 client.16578 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 --2- 172.21.15.7:0/3151147563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55943f2b1a50 0x55943f2c06d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 >> 172.21.15.7:0/3151147563 conn(0x55943f2acfc0 msgr2=0x55943f2b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 shutdown_connections 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 1 -- 172.21.15.7:0/3151147563 wait complete. 2024-03-19T00:43:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.050+0000 7f6fb97ac740 20 client.16578 trim_cache size 0 max 16384 2024-03-19T00:43:20.060 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:43:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:43:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:43:20.060 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6fa00a2800 msgr2=0x7f6fa00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6fa00a2800 0x7f6fa00a4cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40bca70 tx=0x7f6fa40e77c0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 msgr2=0x7f6fb01046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 0x7f6fb01046d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4005940 tx=0x7f6fa40042f0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 shutdown_connections 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6fa00a2800 0x7f6fa00a4cc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb01022f0 0x7f6fb0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fb0101930 0x7f6fb0102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 --2- 172.21.15.7:0/3213335064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb0100740 0x7f6fb01046d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 >> 172.21.15.7:0/3213335064 conn(0x7f6fb00fbcb0 msgr2=0x7f6fb0069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 shutdown_connections 2024-03-19T00:43:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.051+0000 7f6fb6ae8640 1 -- 172.21.15.7:0/3213335064 wait complete. 2024-03-19T00:43:20.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:19 smithi007 ceph-mon[62854]: pgmap v512: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:43:20.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:43:20.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:43:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:19 smithi044 ceph-mon[66366]: pgmap v512: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:43:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 -- 172.21.15.7:0/1988290929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 msgr2=0x7f0388109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 --2- 172.21.15.7:0/1988290929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f0388109980 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0380009860 tx=0x7f038002bb70 comp rx=0 tx=0).stop 2024-03-19T00:43:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 -- 172.21.15.7:0/1988290929 shutdown_connections 2024-03-19T00:43:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 --2- 172.21.15.7:0/1988290929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f0388109980 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 --2- 172.21.15.7:0/1988290929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0388104c60 0x7f0388107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 --2- 172.21.15.7:0/1988290929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 0x7f0388104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 -- 172.21.15.7:0/1988290929 >> 172.21.15.7:0/1988290929 conn(0x7f03880fbcc0 msgr2=0x7f03880fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 -- 172.21.15.7:0/1988290929 shutdown_connections 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.164+0000 7f038d2dc640 1 -- 172.21.15.7:0/1988290929 wait complete. 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 Processor -- start 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 -- start start 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 0x7f0388197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0388104c60 0x7f0388197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f038819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f038810d3a0 con 0x7f0388107590 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f038810d220 con 0x7f0388102330 2024-03-19T00:43:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f038d2dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f038810d520 con 0x7f0388104c60 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.165+0000 7f03877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f038819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03867fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0388104c60 0x7f0388197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f0386ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 0x7f0388197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f038819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54440/0 (socket says 172.21.15.7:54440) 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 -- 172.21.15.7:0/4201045322 learned_addr learned my addr 172.21.15.7:0/4201045322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 -- 172.21.15.7:0/4201045322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0388104c60 msgr2=0x7f0388197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0388104c60 0x7f0388197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 -- 172.21.15.7:0/4201045322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 msgr2=0x7f0388197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 0x7f0388197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 -- 172.21.15.7:0/4201045322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038819f630 con 0x7f0388107590 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f0386ffd640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 0x7f0388197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f03877fe640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f038819efc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f03800057c0 tx=0x7f03800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380041020 con 0x7f0388107590 2024-03-19T00:43:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03800035f0 con 0x7f0388107590 2024-03-19T00:43:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380039040 con 0x7f0388107590 2024-03-19T00:43:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038819f8c0 con 0x7f0388107590 2024-03-19T00:43:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.166+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038819fdb0 con 0x7f0388107590 2024-03-19T00:43:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.167+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03880fd7b0 con 0x7f0388107590 2024-03-19T00:43:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.168+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0380044020 con 0x7f0388107590 2024-03-19T00:43:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.169+0000 7f037ffff640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03740a2800 0x7f03740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.169+0000 7f0386ffd640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03740a2800 0x7f03740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.169+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f03800f0aa0 con 0x7f0388107590 2024-03-19T00:43:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.169+0000 7f0386ffd640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03740a2800 0x7f03740a4cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0378005fd0 tx=0x7f03780095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.172+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f038004b020 con 0x7f0388107590 2024-03-19T00:43:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:19 smithi204 ceph-mon[66572]: pgmap v512: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:43:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.399+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f038802d1f0 con 0x7f0388107590 2024-03-19T00:43:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.399+0000 7f037ffff640 1 -- 172.21.15.7:0/4201045322 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f03800bd780 con 0x7f0388107590 2024-03-19T00:43:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:43:20.401 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:43:20.402 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03740a2800 msgr2=0x7f03740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03740a2800 0x7f03740a4cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0378005fd0 tx=0x7f03780095c0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 msgr2=0x7f038819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f038819efc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f03800057c0 tx=0x7f03800042f0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 shutdown_connections 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f03740a2800 0x7f03740a4cc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0388107590 0x7f038819efc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.401+0000 7f038d2dc640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0388104c60 0x7f0388197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.402+0000 7f038d2dc640 1 --2- 172.21.15.7:0/4201045322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0388102330 0x7f0388197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.402+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 >> 172.21.15.7:0/4201045322 conn(0x7f03880fbcc0 msgr2=0x7f0388102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.402+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 shutdown_connections 2024-03-19T00:43:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.402+0000 7f038d2dc640 1 -- 172.21.15.7:0/4201045322 wait complete. 2024-03-19T00:43:20.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:43:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 -- 172.21.15.7:0/12373756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 msgr2=0x7f8d54101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/12373756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d54101870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009ca0 tx=0x7f8d4c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 -- 172.21.15.7:0/12373756 shutdown_connections 2024-03-19T00:43:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/12373756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d54101870 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/12373756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d54100a50 0x7f8d54100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/12373756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d541000f0 0x7f8d54100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 -- 172.21.15.7:0/12373756 >> 172.21.15.7:0/12373756 conn(0x7f8d540fbc50 msgr2=0x7f8d540fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 -- 172.21.15.7:0/12373756 shutdown_connections 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 -- 172.21.15.7:0/12373756 wait complete. 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 Processor -- start 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 -- start start 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d541000f0 0x7f8d54199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.517+0000 7f8d5c3df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d54100a50 0x7f8d5419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5c3df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d541a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5c3df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410f560 con 0x7f8d541013f0 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5c3df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410f3e0 con 0x7f8d54100a50 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5c3df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410f6e0 con 0x7f8d541000f0 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d541a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d541a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54470/0 (socket says 172.21.15.7:54470) 2024-03-19T00:43:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 -- 172.21.15.7:0/3725879372 learned_addr learned my addr 172.21.15.7:0/3725879372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d541000f0 0x7f8d54199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5997a640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d54100a50 0x7f8d5419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 -- 172.21.15.7:0/3725879372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d541000f0 msgr2=0x7f8d54199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d541000f0 0x7f8d54199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 -- 172.21.15.7:0/3725879372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d54100a50 msgr2=0x7f8d5419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d54100a50 0x7f8d5419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 -- 172.21.15.7:0/3725879372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d541a17a0 con 0x7f8d541013f0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5997a640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d54100a50 0x7f8d5419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5a97c640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d541a1130 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c02c700 tx=0x7f8d4c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d4b7fe640 1 -- 172.21.15.7:0/3725879372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c041020 con 0x7f8d541013f0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d4b7fe640 1 -- 172.21.15.7:0/3725879372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d4c003e20 con 0x7f8d541013f0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d541a1a30 con 0x7f8d541013f0 2024-03-19T00:43:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d4b7fe640 1 -- 172.21.15.7:0/3725879372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c039040 con 0x7f8d541013f0 2024-03-19T00:43:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.518+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d541a1f20 con 0x7f8d541013f0 2024-03-19T00:43:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.520+0000 7f8d4b7fe640 1 -- 172.21.15.7:0/3725879372 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f8d4c044020 con 0x7f8d541013f0 2024-03-19T00:43:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 Processor -- start 2024-03-19T00:43:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 -- start start 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edf62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56118ed524b0 con 0x56118edeece0 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56118ecfad70 con 0x56118edf3e50 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56118ed4e550 con 0x56118edee2f0 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d4b7fe640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d400a2800 0x7f8d400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5a17b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5a17b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54486/0 (socket says 172.21.15.7:54486) 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5a17b640 1 -- 172.21.15.7:0/723257939 learned_addr learned my addr 172.21.15.7:0/723257939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5a97c640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d400a2800 0x7f8d400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.521+0000 7f8d5997a640 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5997a640 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edf62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d4b7fe640 1 -- 172.21.15.7:0/3725879372 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d4c0f0f90 con 0x7f8d541013f0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a17b640 1 -- 172.21.15.7:0/723257939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 msgr2=0x56118edee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a17b640 1 -- 172.21.15.7:0/723257939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 msgr2=0x56118edf62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edf62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a17b640 1 -- 172.21.15.7:0/723257939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118ed1d210 con 0x56118edeece0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf37e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009d20 tx=0x7f8d50011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1162a77b0238ae0a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d5a97c640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d400a2800 0x7f8d400a4cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0c57e0 tx=0x7f8d4c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d48bfe640 1 -- 172.21.15.7:0/723257939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001e070 con 0x56118edeece0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d48bfe640 1 -- 172.21.15.7:0/723257939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d50019ae0 con 0x56118edeece0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.522+0000 7f8d48bfe640 1 -- 172.21.15.7:0/723257939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000ea10 con 0x56118edeece0 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 -- 172.21.15.7:0/723257939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 msgr2=0x56118edf37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf37e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009d20 tx=0x7f8d50011810 comp rx=0 tx=0).stop 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 -- 172.21.15.7:0/723257939 shutdown_connections 2024-03-19T00:43:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edf62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf37e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/723257939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 -- 172.21.15.7:0/723257939 >> 172.21.15.7:0/723257939 conn(0x56118ede9860 msgr2=0x56118ede9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 -- 172.21.15.7:0/723257939 shutdown_connections 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 -- 172.21.15.7:0/723257939 wait complete. 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 -- start start 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.523+0000 7f8d5efc7740 1 Processor -- start 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56118edf6e00 con 0x56118edeece0 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56118ed0e310 con 0x56118edf3e50 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56118edf6f80 con 0x56118edee2f0 2024-03-19T00:43:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 10 client.0 _ll_register_callbacks cb 0x56118ed022d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5a17b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:46642/0 (socket says 172.21.15.7:46642) 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5a17b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54502/0 (socket says 172.21.15.7:54502) 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 -- 172.21.15.7:0/2197308221 learned_addr learned my addr 172.21.15.7:0/2197308221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 -- 172.21.15.7:0/2197308221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 msgr2=0x56118edffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 -- 172.21.15.7:0/2197308221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 msgr2=0x56118edf8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 -- 172.21.15.7:0/2197308221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118ea663f0 con 0x56118edee2f0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400f5a0 tx=0x7f8d440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1773eed21ab8d869 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:43:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2197308221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44011280 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2197308221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d44004050 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.524+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d50019ae0 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2197308221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d4402a050 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 10 client.26353 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56118eb307d0 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2197308221 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d50019ae0 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2197308221 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8d44003d00 con 0x56118edee2f0 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 10 client.26353 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 10 client.26353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 20 client.26353 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee04600 0x56118ee06aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 4 client.26353 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56118edb03e0 con 0x56118ee04600 2024-03-19T00:43:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.525+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee04600 0x56118ee06aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee04600 0x56118ee06aa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=520c686891050277 server_cookie=608e32fbad331a7c in_seq=0 out_seq=0 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d3dffb640 10 client.26353 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2197308221 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56118edb03e0 con 0x56118ee04600 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d3dffb640 10 client.26353 handle_command_reply: tid=0 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 client.26353 shutdown 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 msgr2=0x56118edf8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400f5a0 tx=0x7f8d440046c0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 shutdown_connections 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee04600 0x56118ee06aa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edf3e50 0x56118edffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edf8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2197308221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d3dffb640 0 client.26353 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 >> 172.21.15.7:0/2197308221 conn(0x56118ede9860 msgr2=0x56118ede8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 shutdown_connections 2024-03-19T00:43:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2197308221 wait complete. 2024-03-19T00:43:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.526+0000 7f8d5efc7740 20 client.26353 trim_cache size 0 max 16384 2024-03-19T00:43:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 Processor -- start 2024-03-19T00:43:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 -- start start 2024-03-19T00:43:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edeece0 0x56118edffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 0x56118edf8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541aec20 con 0x56118edf3e50 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541ac4f0 con 0x56118edeece0 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541aa340 con 0x56118edee2f0 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5997a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 0x56118edf8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5997a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 0x56118edf8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54504/0 (socket says 172.21.15.7:54504) 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edeece0 0x56118edffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5997a640 1 -- 172.21.15.7:0/1529063701 learned_addr learned my addr 172.21.15.7:0/1529063701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5997a640 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 -- 172.21.15.7:0/1529063701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 msgr2=0x56118edf8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 -- 172.21.15.7:0/1529063701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 msgr2=0x56118edf8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 0x56118edf8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 -- 172.21.15.7:0/1529063701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118edbf880 con 0x56118edeece0 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5997a640 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 0x56118edf8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.540+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edeece0 0x56118edffd70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x56118edea7d0 tx=0x7f8d50019860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf2fee5c0b1371c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1529063701 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5002a070 con 0x56118edeece0 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1529063701 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d5001f040 con 0x56118edeece0 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1529063701 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001e040 con 0x56118edeece0 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 -- 172.21.15.7:0/1529063701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edeece0 msgr2=0x56118edffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edeece0 0x56118edffd70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x56118edea7d0 tx=0x7f8d50019860 comp rx=0 tx=0).stop 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 -- 172.21.15.7:0/1529063701 shutdown_connections 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edf3e50 0x56118edf8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edeece0 0x56118edffd70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/1529063701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edee2f0 0x56118edf8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 -- 172.21.15.7:0/1529063701 >> 172.21.15.7:0/1529063701 conn(0x56118ede9860 msgr2=0x56118ede4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 -- 172.21.15.7:0/1529063701 shutdown_connections 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 -- 172.21.15.7:0/1529063701 wait complete. 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.541+0000 7f8d5efc7740 1 -- start start 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 Processor -- start 2024-03-19T00:43:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 0x56118edfd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edfd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edf3e50 0x56118edfe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541aa9b0 con 0x56118edeece0 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541abe20 con 0x56118edee2f0 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541ab080 con 0x56118edf3e50 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5efc7740 10 client.0 _ll_register_callbacks cb 0x7f8d440119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 0x56118edfd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5a17b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edfd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 0x56118edfd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49086/0 (socket says 172.21.15.7:49086) 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 -- 172.21.15.7:0/2398213359 learned_addr learned my addr 172.21.15.7:0/2398213359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edf3e50 0x56118edfe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 -- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edf3e50 msgr2=0x56118edfe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edf3e50 0x56118edfe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 -- 172.21.15.7:0/2398213359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 msgr2=0x56118edfd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edfd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 -- 172.21.15.7:0/2398213359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118edfec90 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edfd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d5997a640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 0x56118edfd150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d44010430 tx=0x7f8d44009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48dda4f1fb7ad54b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.542+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2398213359 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440108b0 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d5001f040 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2398213359 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d44010b00 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2398213359 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d44010d00 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 10 client.26450 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56118edfff90 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2398213359 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d5001f040 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2398213359 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8d4402f410 con 0x56118edee2f0 2024-03-19T00:43:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 10 client.26450 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 10 client.26450 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 20 client.26450 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee6b0a0 0x56118ee6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 4 client.26450 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56118ed5f9f0 con 0x56118ee6b0a0 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.543+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee6b0a0 0x56118ee6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5a17b640 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee6b0a0 0x56118ee6d4c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60b74321bc5744e server_cookie=df53153591e2392d in_seq=0 out_seq=0 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d3dffb640 10 client.26450 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d3dffb640 1 -- 172.21.15.7:0/2398213359 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56118ed5f9f0 con 0x56118ee6b0a0 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d3dffb640 10 client.26450 handle_command_reply: tid=0 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 client.26450 shutdown 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 msgr2=0x56118edfd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 0x56118edfd150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d44010430 tx=0x7f8d44009eb0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 shutdown_connections 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x56118ee6b0a0 0x56118ee6d4c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56118edf3e50 0x56118edfe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56118edeece0 0x56118edfd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d3dffb640 0 client.26450 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 --2- 172.21.15.7:0/2398213359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56118edee2f0 0x56118edfd150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 >> 172.21.15.7:0/2398213359 conn(0x56118ede9860 msgr2=0x56118edf8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 shutdown_connections 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 1 -- 172.21.15.7:0/2398213359 wait complete. 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.544+0000 7f8d5efc7740 20 client.26450 trim_cache size 0 max 16384 2024-03-19T00:43:20.553 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:43:20.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d400a2800 msgr2=0x7f8d400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d400a2800 0x7f8d400a4cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0c57e0 tx=0x7f8d4c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 msgr2=0x7f8d541a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d541a1130 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c02c700 tx=0x7f8d4c0328a0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 shutdown_connections 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d400a2800 0x7f8d400a4cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541013f0 0x7f8d541a1130 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d54100a50 0x7f8d5419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 --2- 172.21.15.7:0/3725879372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d541000f0 0x7f8d54199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 >> 172.21.15.7:0/3725879372 conn(0x7f8d540fbc50 msgr2=0x7f8d540fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 shutdown_connections 2024-03-19T00:43:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.545+0000 7f8d5c3df640 1 -- 172.21.15.7:0/3725879372 wait complete. 2024-03-19T00:43:20.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 -- 172.21.15.7:0/1281601057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8100a20 msgr2=0x7fb4d8100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/1281601057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009860 tx=0x7fb4cc02bb70 comp rx=0 tx=0).stop 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 -- 172.21.15.7:0/1281601057 shutdown_connections 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/1281601057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d81013e0 0x7fb4d8108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/1281601057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/1281601057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d8101d80 0x7fb4d81021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 -- 172.21.15.7:0/1281601057 >> 172.21.15.7:0/1281601057 conn(0x7fb4d80fb470 msgr2=0x7fb4d80fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 -- 172.21.15.7:0/1281601057 shutdown_connections 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 -- 172.21.15.7:0/1281601057 wait complete. 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 Processor -- start 2024-03-19T00:43:20.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.660+0000 7fb4ddd97640 1 -- start start 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d81013e0 0x7fb4d819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8101d80 0x7fb4d819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d810c2a0 con 0x7fb4d8100a20 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d810c120 con 0x7fb4d8101d80 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d810c420 con 0x7fb4d81013e0 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54542/0 (socket says 172.21.15.7:54542) 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 -- 172.21.15.7:0/2338114703 learned_addr learned my addr 172.21.15.7:0/2338114703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d81013e0 0x7fb4d819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8101d80 0x7fb4d819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 -- 172.21.15.7:0/2338114703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d81013e0 msgr2=0x7fb4d819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d81013e0 0x7fb4d819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 -- 172.21.15.7:0/2338114703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8101d80 msgr2=0x7fb4d819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8101d80 0x7fb4d819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 -- 172.21.15.7:0/2338114703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d819f710 con 0x7fb4d8100a20 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800e9c0 tx=0x7fb4c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d4ff9640 1 -- 172.21.15.7:0/2338114703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800cde0 con 0x7fb4d8100a20 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d4ff9640 1 -- 172.21.15.7:0/2338114703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4c80108e0 con 0x7fb4d8100a20 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d819f9a0 con 0x7fb4d8100a20 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4d4ff9640 1 -- 172.21.15.7:0/2338114703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8018990 con 0x7fb4d8100a20 2024-03-19T00:43:20.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.661+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d819fec0 con 0x7fb4d8100a20 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4d4ff9640 1 -- 172.21.15.7:0/2338114703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb4c801f020 con 0x7fb4d8100a20 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 Processor -- start 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 -- start start 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caecc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caed42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5603cae304b0 con 0x5603caecc2f0 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5603cadd8d70 con 0x5603caeccce0 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4e0984740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5603cae2c550 con 0x5603caed1e50 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caecc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.664+0000 7fb4d4ff9640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb4c00a2800 0x7fb4c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49118/0 (socket says 172.21.15.7:49118) 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caed42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d77fe640 1 -- 172.21.15.7:0/2361994214 learned_addr learned my addr 172.21.15.7:0/2361994214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb4c00a2800 0x7fb4c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d4ff9640 1 -- 172.21.15.7:0/2338114703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb4c80cef70 con 0x7fb4d8100a20 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2361994214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 msgr2=0x5603caed17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2361994214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 msgr2=0x5603caecc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caecc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2361994214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cadfb210 con 0x5603caed1e50 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caed42e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02c0b0 tx=0x7fb4cc009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f240b8efa55bce5b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb4c00a2800 0x7fb4c00a4cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5603caecd540 tx=0x7fb4d0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4bdffb640 1 -- 172.21.15.7:0/2361994214 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc043020 con 0x5603caed1e50 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4bdffb640 1 -- 172.21.15.7:0/2361994214 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc0346c0 con 0x5603caed1e50 2024-03-19T00:43:20.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.665+0000 7fb4bdffb640 1 -- 172.21.15.7:0/2361994214 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc039040 con 0x5603caed1e50 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 -- 172.21.15.7:0/2361994214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 msgr2=0x5603caed42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caed42e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02c0b0 tx=0x7fb4cc009490 comp rx=0 tx=0).stop 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 -- 172.21.15.7:0/2361994214 shutdown_connections 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caed42e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2361994214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caecc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 -- 172.21.15.7:0/2361994214 >> 172.21.15.7:0/2361994214 conn(0x5603caec7860 msgr2=0x5603caec7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 -- 172.21.15.7:0/2361994214 shutdown_connections 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 -- 172.21.15.7:0/2361994214 wait complete. 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.666+0000 7fb4e0984740 1 -- start start 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 Processor -- start 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caecc2f0 0x5603caed67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 0x5603caeddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5603caed4da0 con 0x5603caed1e50 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5603cadec310 con 0x5603caeccce0 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5603caed4f20 con 0x5603caecc2f0 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4e0984740 10 client.0 _ll_register_callbacks cb 0x5603cade02d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 0x5603caeddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caecc2f0 0x5603caed67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 0x5603caeddce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54568/0 (socket says 172.21.15.7:54568) 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 -- 172.21.15.7:0/3395308272 learned_addr learned my addr 172.21.15.7:0/3395308272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 -- 172.21.15.7:0/3395308272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caecc2f0 msgr2=0x5603caed67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caecc2f0 0x5603caed67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 -- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 msgr2=0x5603caed6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 -- 172.21.15.7:0/3395308272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cab443f0 con 0x5603caed1e50 2024-03-19T00:43:20.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 0x5603caeddce0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000df90 tx=0x7fb4d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58938d051bd2374 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.667+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3395308272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0004b20 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3395308272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb4d0004cc0 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc0346c0 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3395308272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4d0003970 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 10 client.16614 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603cac0e7d0 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3395308272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc0346c0 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3395308272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb4d0027070 con 0x5603caed1e50 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 10 client.16614 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 10 client.16614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 20 client.16614 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caee2680 0x5603caee4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 4 client.16614 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603cae8e3e0 con 0x5603caee2680 2024-03-19T00:43:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.668+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caee2680 0x5603caee4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caee2680 0x5603caee4ae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad11da28a9ce87ec server_cookie=fee87d32a749215a in_seq=0 out_seq=0 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4b6ffd640 10 client.16614 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3395308272 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5603cae8e3e0 con 0x5603caee2680 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4b6ffd640 10 client.16614 handle_command_reply: tid=0 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 client.16614 shutdown 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 msgr2=0x5603caeddce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 0x5603caeddce0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000df90 tx=0x7fb4d0004a40 comp rx=0 tx=0).stop 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 shutdown_connections 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caee2680 0x5603caee4ae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caed1e50 0x5603caeddce0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.669+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caeccce0 0x5603caed6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.670+0000 7fb4b6ffd640 0 client.16614 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.670+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3395308272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caecc2f0 0x5603caed67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.670+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 >> 172.21.15.7:0/3395308272 conn(0x5603caec7860 msgr2=0x5603caec6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.670+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 shutdown_connections 2024-03-19T00:43:20.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.670+0000 7fb4e0984740 1 -- 172.21.15.7:0/3395308272 wait complete. 2024-03-19T00:43:20.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.670+0000 7fb4e0984740 20 client.16614 trim_cache size 0 max 16384 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 Processor -- start 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 -- start start 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caecc2f0 0x5603caec7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 0x5603caeddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caece240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81acc10 con 0x5603caeccce0 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81aa4e0 con 0x5603caecc2f0 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4e0984740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81a8330 con 0x5603caed1e50 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 0x5603caeddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 0x5603caeddca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54578/0 (socket says 172.21.15.7:54578) 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2597925099 learned_addr learned my addr 172.21.15.7:0/2597925099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caecc2f0 0x5603caec7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caece240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2597925099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 msgr2=0x5603caece240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caece240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2597925099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caecc2f0 msgr2=0x5603caec7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caecc2f0 0x5603caec7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 -- 172.21.15.7:0/2597925099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603caecd0c0 con 0x5603caeccce0 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.683+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 0x5603caeddca0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02c180 tx=0x7fb4cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c041fb6156b9a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4bd7fa640 1 -- 172.21.15.7:0/2597925099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc043030 con 0x5603caeccce0 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4bd7fa640 1 -- 172.21.15.7:0/2597925099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc0346c0 con 0x5603caeccce0 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4bd7fa640 1 -- 172.21.15.7:0/2597925099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc04e070 con 0x5603caeccce0 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 -- 172.21.15.7:0/2597925099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 msgr2=0x5603caeddca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 0x5603caeddca0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02c180 tx=0x7fb4cc000bc0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 -- 172.21.15.7:0/2597925099 shutdown_connections 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caed1e50 0x5603caece240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caeccce0 0x5603caeddca0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 --2- 172.21.15.7:0/2597925099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caecc2f0 0x5603caec7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 -- 172.21.15.7:0/2597925099 >> 172.21.15.7:0/2597925099 conn(0x5603caec7860 msgr2=0x5603caec2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 -- 172.21.15.7:0/2597925099 shutdown_connections 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 -- 172.21.15.7:0/2597925099 wait complete. 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.684+0000 7fb4e0984740 1 -- start start 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 Processor -- start 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caedadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caeccce0 0x5603caedb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caed1e50 0x5603caedc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81a89a0 con 0x5603caecc2f0 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81a9e10 con 0x5603caed1e50 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81a9070 con 0x5603caeccce0 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4e0984740 10 client.0 _ll_register_callbacks cb 0x7fb4d000e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caedadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caedadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54586/0 (socket says 172.21.15.7:54586) 2024-03-19T00:43:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caed1e50 0x5603caedc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 -- 172.21.15.7:0/3305460314 learned_addr learned my addr 172.21.15.7:0/3305460314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caeccce0 0x5603caedb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 -- 172.21.15.7:0/3305460314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caeccce0 msgr2=0x5603caedb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caeccce0 0x5603caedb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 -- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caed1e50 msgr2=0x5603caedc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caed1e50 0x5603caedc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 -- 172.21.15.7:0/3305460314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603caedc930 con 0x5603caecc2f0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d7fff640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caed1e50 0x5603caedc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4d77fe640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caedadf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb4c809b540 tx=0x7fb4c8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d6594ec1acb701 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.685+0000 7fb4b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3305460314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80b4430 con 0x5603caecc2f0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3305460314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb4c8097c20 con 0x5603caecc2f0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc0346c0 con 0x5603caecc2f0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3305460314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4c809b5b0 con 0x5603caecc2f0 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 10 client.16620 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603caeddd00 con 0x5603caecc2f0 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3305460314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc0346c0 con 0x5603caecc2f0 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3305460314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb4c80d10b0 con 0x5603caecc2f0 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 10 client.16620 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 10 client.16620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 20 client.16620 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caf48f50 0x5603caf4b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 4 client.16620 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603cae3d9f0 con 0x5603caf48f50 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.686+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caf48f50 0x5603caf4b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4d6ffd640 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caf48f50 0x5603caf4b370 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d22bb3a31db7b23 server_cookie=2531ff2a5606e997 in_seq=0 out_seq=0 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4b6ffd640 10 client.16620 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4b6ffd640 1 -- 172.21.15.7:0/3305460314 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603cae3d9f0 con 0x5603caf48f50 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4b6ffd640 10 client.16620 handle_command_reply: tid=0 2024-03-19T00:43:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 client.16620 shutdown 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 msgr2=0x5603caedadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caedadf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb4c809b540 tx=0x7fb4c8086040 comp rx=0 tx=0).stop 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 shutdown_connections 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5603caf48f50 0x5603caf4b370 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603caed1e50 0x5603caedc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4b6ffd640 0 client.16620 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5603caeccce0 0x5603caedb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 --2- 172.21.15.7:0/3305460314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5603caecc2f0 0x5603caedadf0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 >> 172.21.15.7:0/3305460314 conn(0x5603caec7860 msgr2=0x5603caec7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 shutdown_connections 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 1 -- 172.21.15.7:0/3305460314 wait complete. 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.687+0000 7fb4e0984740 20 client.16620 trim_cache size 0 max 16384 2024-03-19T00:43:20.695 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:43:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb4c00a2800 msgr2=0x7fb4c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb4c00a2800 0x7fb4c00a4cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5603caecd540 tx=0x7fb4d0004ab0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 msgr2=0x7fb4d8100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800e9c0 tx=0x7fb4c800ee80 comp rx=0 tx=0).stop 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 shutdown_connections 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb4c00a2800 0x7fb4c00a4cc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4d8101d80 0x7fb4d819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d81013e0 0x7fb4d819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 --2- 172.21.15.7:0/2338114703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d8100a20 0x7fb4d8100480 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 >> 172.21.15.7:0/2338114703 conn(0x7fb4d80fb470 msgr2=0x7fb4d8104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 shutdown_connections 2024-03-19T00:43:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.688+0000 7fb4ddd97640 1 -- 172.21.15.7:0/2338114703 wait complete. 2024-03-19T00:43:20.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 -- 172.21.15.7:0/3813508825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c1033d0 msgr2=0x7f057c109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 --2- 172.21.15.7:0/3813508825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c1033d0 0x7f057c109c60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0574009ca0 tx=0x7f057402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 -- 172.21.15.7:0/3813508825 shutdown_connections 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 --2- 172.21.15.7:0/3813508825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c1033d0 0x7f057c109c60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 --2- 172.21.15.7:0/3813508825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f057c102a10 0x7f057c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 --2- 172.21.15.7:0/3813508825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f057c101820 0x7f057c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 -- 172.21.15.7:0/3813508825 >> 172.21.15.7:0/3813508825 conn(0x7f057c0fcd50 msgr2=0x7f057c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 -- 172.21.15.7:0/3813508825 shutdown_connections 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 -- 172.21.15.7:0/3813508825 wait complete. 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.804+0000 7f05847c6640 1 Processor -- start 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 -- start start 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 0x7f057c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f057c102a10 0x7f057c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f057c1033d0 0x7f057c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f057c10d330 con 0x7f057c101820 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f057c10d1b0 con 0x7f057c1033d0 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f057c10d4b0 con 0x7f057c102a10 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 0x7f057c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 0x7f057c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54608/0 (socket says 172.21.15.7:54608) 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 -- 172.21.15.7:0/2830474571 learned_addr learned my addr 172.21.15.7:0/2830474571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0581d61640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f057c102a10 0x7f057c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582d63640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f057c1033d0 0x7f057c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 -- 172.21.15.7:0/2830474571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f057c102a10 msgr2=0x7f057c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f057c102a10 0x7f057c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 -- 172.21.15.7:0/2830474571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f057c1033d0 msgr2=0x7f057c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f057c1033d0 0x7f057c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 -- 172.21.15.7:0/2830474571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c19f5b0 con 0x7f057c101820 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f0582562640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 0x7f057c1979e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f057800d950 tx=0x7f057800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05737fe640 1 -- 172.21.15.7:0/2830474571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578017020 con 0x7f057c101820 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05737fe640 1 -- 172.21.15.7:0/2830474571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0578009c30 con 0x7f057c101820 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c19f8a0 con 0x7f057c101820 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.805+0000 7f05737fe640 1 -- 172.21.15.7:0/2830474571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057800f6b0 con 0x7f057c101820 2024-03-19T00:43:20.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.806+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c19fe40 con 0x7f057c101820 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05737fe640 1 -- 172.21.15.7:0/2830474571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f057801a020 con 0x7f057c101820 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 Processor -- start 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 -- start start 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05737fe640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05680a2880 0x7f05680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc39710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 0x5647bcc3e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc3ee50 0x5647bcc412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5647bcb9d4b0 con 0x5647bcc3ee50 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5647bcb45d70 con 0x5647bcc392f0 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05873a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5647bcb99550 con 0x5647bcc39ce0 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f0582d63640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05680a2880 0x7f05680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.808+0000 7f05737fe640 1 -- 172.21.15.7:0/2830474571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f05780ce960 con 0x7f057c101820 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0582562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc39710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 0x5647bcc3e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 0x5647bcc3e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:46768/0 (socket says 172.21.15.7:46768) 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 -- 172.21.15.7:0/3888025377 learned_addr learned my addr 172.21.15.7:0/3888025377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0582562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc39710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49180/0 (socket says 172.21.15.7:49180) 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0582d63640 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc3ee50 0x5647bcc412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 -- 172.21.15.7:0/3888025377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 msgr2=0x5647bcc39710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc39710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 -- 172.21.15.7:0/3888025377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc3ee50 msgr2=0x5647bcc412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc3ee50 0x5647bcc412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 -- 172.21.15.7:0/3888025377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647bcb68210 con 0x5647bcc39ce0 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0582d63640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05680a2880 0x7f05680a4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0574006010 tx=0x7f0574032ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0581d61640 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 0x5647bcc3e7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f056c009c10 tx=0x7f056c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8efe5940fd346e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0570ff9640 1 -- 172.21.15.7:0/3888025377 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c004050 con 0x5647bcc39ce0 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0570ff9640 1 -- 172.21.15.7:0/3888025377 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f056c0054d0 con 0x5647bcc39ce0 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.809+0000 7f0570ff9640 1 -- 172.21.15.7:0/3888025377 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c004ce0 con 0x5647bcc39ce0 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 -- 172.21.15.7:0/3888025377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 msgr2=0x5647bcc3e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 0x5647bcc3e7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f056c009c10 tx=0x7f056c011810 comp rx=0 tx=0).stop 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 -- 172.21.15.7:0/3888025377 shutdown_connections 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc3ee50 0x5647bcc412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc39ce0 0x5647bcc3e7e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 --2- 172.21.15.7:0/3888025377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc39710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 -- 172.21.15.7:0/3888025377 >> 172.21.15.7:0/3888025377 conn(0x5647bcc34860 msgr2=0x5647bcc34c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 -- 172.21.15.7:0/3888025377 shutdown_connections 2024-03-19T00:43:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 -- 172.21.15.7:0/3888025377 wait complete. 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 -- start start 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.810+0000 7f05873a8740 1 Processor -- start 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 0x5647bcc43790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc39ce0 0x5647bcc43cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc4ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5647bcc41e00 con 0x5647bcc392f0 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5647bcb59310 con 0x5647bcc39ce0 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5647bcc41f80 con 0x5647bcc3ee50 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f05873a8740 10 client.0 _ll_register_callbacks cb 0x5647bcb4d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 0x5647bcc43790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 0x5647bcc43790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54630/0 (socket says 172.21.15.7:54630) 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 -- 172.21.15.7:0/1434796632 learned_addr learned my addr 172.21.15.7:0/1434796632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0581d61640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc39ce0 0x5647bcc43cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582d63640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc4ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 -- 172.21.15.7:0/1434796632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 msgr2=0x5647bcc4ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc4ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 -- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc39ce0 msgr2=0x5647bcc43cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc39ce0 0x5647bcc43cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 -- 172.21.15.7:0/1434796632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647bc8b13f0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0581d61640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc39ce0 0x5647bcc43cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0582562640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 0x5647bcc43790 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f057809aca0 tx=0x7f05780072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6efcecd58285cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0565ffb640 1 -- 172.21.15.7:0/1434796632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578019cf0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.811+0000 7f0565ffb640 1 -- 172.21.15.7:0/1434796632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0578022be0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056c0054d0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f0565ffb640 1 -- 172.21.15.7:0/1434796632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f05780224f0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 10 client.16632 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647bc97b7d0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f0565ffb640 1 -- 172.21.15.7:0/1434796632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f056c0054d0 con 0x5647bcc392f0 2024-03-19T00:43:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f0565ffb640 1 -- 172.21.15.7:0/1434796632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0578024060 con 0x5647bcc392f0 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 10 client.16632 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 20 client.16632 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bcc4f630 0x5647bcc51ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 4 client.16632 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647bcbfb3e0 con 0x5647bcc4f630 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.812+0000 7f0581d61640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bcc4f630 0x5647bcc51ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f0581d61640 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bcc4f630 0x5647bcc51ac0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd93ca2bed50d91 server_cookie=a98d4b29d050b19a in_seq=0 out_seq=0 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f0565ffb640 10 client.16632 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f0565ffb640 1 -- 172.21.15.7:0/1434796632 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5647bcbfb3e0 con 0x5647bcc4f630 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f0565ffb640 10 client.16632 handle_command_reply: tid=0 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 client.16632 shutdown 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 msgr2=0x5647bcc43790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 0x5647bcc43790 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f057809aca0 tx=0x7f05780072b0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 shutdown_connections 2024-03-19T00:43:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc4ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc39ce0 0x5647bcc43cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bcc4f630 0x5647bcc51ac0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 --2- 172.21.15.7:0/1434796632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc392f0 0x5647bcc43790 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f0565ffb640 0 client.16632 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 >> 172.21.15.7:0/1434796632 conn(0x5647bcc34860 msgr2=0x5647bcc33ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.813+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 shutdown_connections 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.814+0000 7f05873a8740 1 -- 172.21.15.7:0/1434796632 wait complete. 2024-03-19T00:43:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.814+0000 7f05873a8740 20 client.16632 trim_cache size 0 max 16384 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 Processor -- start 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 -- start start 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc4acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc33a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1a42d0 con 0x5647bcc39ce0 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1a1ba0 con 0x5647bcc392f0 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f05873a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f057c073350 con 0x5647bcc3ee50 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0581d61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc4acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0581d61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc4acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54642/0 (socket says 172.21.15.7:54642) 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0581d61640 1 -- 172.21.15.7:0/2351601929 learned_addr learned my addr 172.21.15.7:0/2351601929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582d63640 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc33a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582562640 1 -- 172.21.15.7:0/2351601929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 msgr2=0x5647bcc33a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582562640 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc33a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582562640 1 -- 172.21.15.7:0/2351601929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 msgr2=0x5647bcc4acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582562640 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc4acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0582562640 1 -- 172.21.15.7:0/2351601929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647bcc3e430 con 0x5647bcc392f0 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.827+0000 7f0581d61640 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc4acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:20.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f0582562640 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc43c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0578000c00 tx=0x7f05780972c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f77cd4f06b179d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f0567fff640 1 -- 172.21.15.7:0/2351601929 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057809afa0 con 0x5647bcc392f0 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f0567fff640 1 -- 172.21.15.7:0/2351601929 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05780a3740 con 0x5647bcc392f0 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f0567fff640 1 -- 172.21.15.7:0/2351601929 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578095d70 con 0x5647bcc392f0 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 -- 172.21.15.7:0/2351601929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 msgr2=0x5647bcc43c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc43c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0578000c00 tx=0x7f05780972c0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 -- 172.21.15.7:0/2351601929 shutdown_connections 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc3ee50 0x5647bcc33a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc4acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 --2- 172.21.15.7:0/2351601929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc392f0 0x5647bcc43c30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 -- 172.21.15.7:0/2351601929 >> 172.21.15.7:0/2351601929 conn(0x5647bcc34860 msgr2=0x5647bcc33310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 -- 172.21.15.7:0/2351601929 shutdown_connections 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 -- 172.21.15.7:0/2351601929 wait complete. 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.828+0000 7f05873a8740 1 -- start start 2024-03-19T00:43:20.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 Processor -- start 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc392f0 0x5647bcc48160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc3ee50 0x5647bcc49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1a00d0 con 0x5647bcc39ce0 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1a14d0 con 0x5647bcc3ee50 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1a0730 con 0x5647bcc392f0 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f05873a8740 10 client.0 _ll_register_callbacks cb 0x7f05780071c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54650/0 (socket says 172.21.15.7:54650) 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0582562640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc392f0 0x5647bcc48160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 -- 172.21.15.7:0/3608746261 learned_addr learned my addr 172.21.15.7:0/3608746261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0582d63640 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc3ee50 0x5647bcc49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 -- 172.21.15.7:0/3608746261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc392f0 msgr2=0x5647bcc48160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc392f0 0x5647bcc48160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 -- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc3ee50 msgr2=0x5647bcc49660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc3ee50 0x5647bcc49660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 -- 172.21.15.7:0/3608746261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647bcc49c50 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0581d61640 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc486a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f056c009b60 tx=0x7f056c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4699b6eb11d81a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.829+0000 7f0565ffb640 1 -- 172.21.15.7:0/3608746261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c040d10 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f0565ffb640 1 -- 172.21.15.7:0/3608746261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f056c040330 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05780a3740 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f0565ffb640 1 -- 172.21.15.7:0/3608746261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f056c04f050 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 10 client.16638 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647bcc4af80 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f0565ffb640 1 -- 172.21.15.7:0/3608746261 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05780a3740 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f0565ffb640 1 -- 172.21.15.7:0/3608746261 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f056c003bc0 con 0x5647bcc39ce0 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 10 client.16638 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 20 client.16638 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bccb5d50 0x5647bccb8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 4 client.16638 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647bcbaa9f0 con 0x5647bccb5d50 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.830+0000 7f0582562640 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bccb5d50 0x5647bccb8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f0582562640 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bccb5d50 0x5647bccb8170 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7def3563cfa644d server_cookie=f4eb10e4159b5b44 in_seq=0 out_seq=0 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f0565ffb640 10 client.16638 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f0565ffb640 1 -- 172.21.15.7:0/3608746261 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647bcbaa9f0 con 0x5647bccb5d50 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f0565ffb640 10 client.16638 handle_command_reply: tid=0 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 client.16638 shutdown 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 msgr2=0x5647bcc486a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc486a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f056c009b60 tx=0x7f056c005db0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 shutdown_connections 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647bcc3ee50 0x5647bcc49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5647bccb5d50 0x5647bccb8170 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5647bcc39ce0 0x5647bcc486a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f0565ffb640 0 client.16638 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 --2- 172.21.15.7:0/3608746261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5647bcc392f0 0x5647bcc48160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 >> 172.21.15.7:0/3608746261 conn(0x5647bcc34860 msgr2=0x5647bcc38350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 shutdown_connections 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 1 -- 172.21.15.7:0/3608746261 wait complete. 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05873a8740 20 client.16638 trim_cache size 0 max 16384 2024-03-19T00:43:20.841 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05680a2880 msgr2=0x7f05680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05847c6640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05680a2880 0x7f05680a4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0574006010 tx=0x7f0574032ec0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 msgr2=0x7f057c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.831+0000 7f05847c6640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 0x7f057c1979e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f057800d950 tx=0x7f057800de10 comp rx=0 tx=0).stop 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 shutdown_connections 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05680a2880 0x7f05680a4d40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f057c1033d0 0x7f057c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f057c102a10 0x7f057c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 --2- 172.21.15.7:0/2830474571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f057c101820 0x7f057c1979e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 >> 172.21.15.7:0/2830474571 conn(0x7f057c0fcd50 msgr2=0x7f057c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 shutdown_connections 2024-03-19T00:43:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:20.832+0000 7f05847c6640 1 -- 172.21.15.7:0/2830474571 wait complete. 2024-03-19T00:43:20.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:20 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4201045322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:43:21.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:20 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4201045322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:43:21.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:20 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4201045322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:43:21.844 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:43:21.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:43:21.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 -- 172.21.15.7:0/2725800324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa250101930 msgr2=0x7fa250101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 --2- 172.21.15.7:0/2725800324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa250101930 0x7fa250101db0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009900 tx=0x7fa24c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 -- 172.21.15.7:0/2725800324 shutdown_connections 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 --2- 172.21.15.7:0/2725800324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2501022f0 0x7fa250108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 --2- 172.21.15.7:0/2725800324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa250101930 0x7fa250101db0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 --2- 172.21.15.7:0/2725800324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 0x7fa250100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 -- 172.21.15.7:0/2725800324 >> 172.21.15.7:0/2725800324 conn(0x7fa2500fbcb0 msgr2=0x7fa2500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 -- 172.21.15.7:0/2725800324 shutdown_connections 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.962+0000 7fa2584c6640 1 -- 172.21.15.7:0/2725800324 wait complete. 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 Processor -- start 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 -- start start 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 0x7fa250197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa250101930 0x7fa250197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 0x7fa25019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa25010c250 con 0x7fa2501022f0 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa25010c0d0 con 0x7fa250101930 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa2584c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa25010c3d0 con 0x7fa250100740 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 0x7fa25019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa255a61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa250101930 0x7fa250197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 0x7fa25019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54668/0 (socket says 172.21.15.7:54668) 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 -- 172.21.15.7:0/1072222996 learned_addr learned my addr 172.21.15.7:0/1072222996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256262640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 0x7fa250197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 -- 172.21.15.7:0/1072222996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 msgr2=0x7fa250197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 0x7fa250197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 -- 172.21.15.7:0/1072222996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa250101930 msgr2=0x7fa250197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa250101930 0x7fa250197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256a63640 1 -- 172.21.15.7:0/1072222996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25019f6d0 con 0x7fa2501022f0 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa255a61640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa250101930 0x7fa250197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.963+0000 7fa256262640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 0x7fa250197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.964+0000 7fa256a63640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 0x7fa25019f060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa24800efd0 tx=0x7fa24800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.964+0000 7fa2477fe640 1 -- 172.21.15.7:0/1072222996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248003b70 con 0x7fa2501022f0 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.964+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25019f9c0 con 0x7fa2501022f0 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.964+0000 7fa2477fe640 1 -- 172.21.15.7:0/1072222996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa248003d10 con 0x7fa2501022f0 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.964+0000 7fa2477fe640 1 -- 172.21.15.7:0/1072222996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248010980 con 0x7fa2501022f0 2024-03-19T00:43:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.964+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25019fee0 con 0x7fa2501022f0 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa2477fe640 1 -- 172.21.15.7:0/1072222996 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa248004020 con 0x7fa2501022f0 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 Processor -- start 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 -- start start 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ecec550160 con 0x55ecec5f0f00 2024-03-19T00:43:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ecec4f8e10 con 0x55ecec5eb3a0 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.967+0000 7fa25b191740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ecec54c200 con 0x55ecec5ebd90 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa255a61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa255a61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5eb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49242/0 (socket says 172.21.15.7:49242) 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa255a61640 1 -- 172.21.15.7:0/792060473 learned_addr learned my addr 172.21.15.7:0/792060473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa255a61640 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa2477fe640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa23c0a2800 0x7fa23c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 -- 172.21.15.7:0/792060473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 msgr2=0x55ecec5eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 -- 172.21.15.7:0/792060473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 msgr2=0x55ecec5f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 -- 172.21.15.7:0/792060473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecec32c7e0 con 0x55ecec5ebd90 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256a63640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa23c0a2800 0x7fa23c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa256262640 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f0890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa240009fa0 tx=0x7fa2400118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2387fa66dcd82d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa2477fe640 1 -- 172.21.15.7:0/1072222996 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa2480cf220 con 0x7fa2501022f0 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa244bfe640 1 -- 172.21.15.7:0/792060473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24001e070 con 0x55ecec5ebd90 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa244bfe640 1 -- 172.21.15.7:0/792060473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24000f9f0 con 0x55ecec5ebd90 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.968+0000 7fa244bfe640 1 -- 172.21.15.7:0/792060473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240019990 con 0x55ecec5ebd90 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 -- 172.21.15.7:0/792060473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 msgr2=0x55ecec5f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa256a63640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa23c0a2800 0x7fa23c0a4cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa248098230 tx=0x7fa2480973e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f0890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa240009fa0 tx=0x7fa2400118d0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 -- 172.21.15.7:0/792060473 shutdown_connections 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f0890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 --2- 172.21.15.7:0/792060473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 -- 172.21.15.7:0/792060473 >> 172.21.15.7:0/792060473 conn(0x55ecec5e6910 msgr2=0x55ecec5e6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 -- 172.21.15.7:0/792060473 shutdown_connections 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.969+0000 7fa25b191740 1 -- 172.21.15.7:0/792060473 wait complete. 2024-03-19T00:43:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 -- start start 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 Processor -- start 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5f0f00 0x55ecec5fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ecec5f3eb0 con 0x55ecec5eb3a0 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ecec50c1a0 con 0x55ecec5f0f00 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ecec5f4030 con 0x55ecec5ebd90 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa25b191740 10 client.0 _ll_register_callbacks cb 0x55ecec263c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54682/0 (socket says 172.21.15.7:54682) 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 -- 172.21.15.7:0/2836054754 learned_addr learned my addr 172.21.15.7:0/2836054754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa256262640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5f0f00 0x55ecec5fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 -- 172.21.15.7:0/2836054754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 msgr2=0x55ecec5f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 -- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5f0f00 msgr2=0x55ecec5fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5f0f00 0x55ecec5fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.970+0000 7fa255a61640 1 -- 172.21.15.7:0/2836054754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecec38cf70 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa255a61640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa24c036040 tx=0x7fa24c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6381a4359a828647 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa239ffb640 1 -- 172.21.15.7:0/2836054754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c04a030 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa239ffb640 1 -- 172.21.15.7:0/2836054754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa24c03a040 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24000f9f0 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa239ffb640 1 -- 172.21.15.7:0/2836054754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa24c057050 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 10 client.16650 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecec224170 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa239ffb640 1 -- 172.21.15.7:0/2836054754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24000f9f0 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa239ffb640 1 -- 172.21.15.7:0/2836054754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa24c003920 con 0x55ecec5eb3a0 2024-03-19T00:43:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 10 client.16650 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 20 client.16650 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec601630 0x55ecec603aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 4 client.16650 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.971+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecec55d6a0 con 0x55ecec601630 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.972+0000 7fa256262640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec601630 0x55ecec603aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.972+0000 7fa256262640 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec601630 0x55ecec603aa0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfbc3a9cbe7a008 server_cookie=ea987764552e71ad in_seq=0 out_seq=0 2024-03-19T00:43:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.972+0000 7fa239ffb640 10 client.16650 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa239ffb640 1 -- 172.21.15.7:0/2836054754 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ecec55d6a0 con 0x55ecec601630 2024-03-19T00:43:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa239ffb640 10 client.16650 handle_command_reply: tid=0 2024-03-19T00:43:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 client.16650 shutdown 2024-03-19T00:43:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 msgr2=0x55ecec5f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa24c036040 tx=0x7fa24c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 shutdown_connections 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5f0f00 0x55ecec5fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec601630 0x55ecec603aa0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 --2- 172.21.15.7:0/2836054754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5880 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 >> 172.21.15.7:0/2836054754 conn(0x55ecec5e6910 msgr2=0x55ecec5e5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa239ffb640 0 client.16650 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 shutdown_connections 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 1 -- 172.21.15.7:0/2836054754 wait complete. 2024-03-19T00:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.973+0000 7fa25b191740 20 client.16650 trim_cache size 0 max 16384 2024-03-19T00:43:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 Processor -- start 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 -- start start 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 0x55ecec5fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5f0f00 0x55ecec5ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501acb80 con 0x55ecec5ebd90 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501aa450 con 0x55ecec5eb3a0 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa25b191740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501a82a0 con 0x55ecec5f0f00 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 0x55ecec5fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 0x55ecec5fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54694/0 (socket says 172.21.15.7:54694) 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 -- 172.21.15.7:0/3490291798 learned_addr learned my addr 172.21.15.7:0/3490291798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa255a61640 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa255a61640 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5f0f00 0x55ecec5ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 -- 172.21.15.7:0/3490291798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5f0f00 msgr2=0x55ecec5ea280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5f0f00 0x55ecec5ea280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 -- 172.21.15.7:0/3490291798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 msgr2=0x55ecec5f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.991+0000 7fa256262640 1 -- 172.21.15.7:0/3490291798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecec5e64b0 con 0x55ecec5ebd90 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa256262640 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 0x55ecec5fcdc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa240009ef0 tx=0x7fa240011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96abc97675c245d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa23bfff640 1 -- 172.21.15.7:0/3490291798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24001e040 con 0x55ecec5ebd90 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa23bfff640 1 -- 172.21.15.7:0/3490291798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24000f9f0 con 0x55ecec5ebd90 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa23bfff640 1 -- 172.21.15.7:0/3490291798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400272f0 con 0x55ecec5ebd90 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 -- 172.21.15.7:0/3490291798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 msgr2=0x55ecec5fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 0x55ecec5fcdc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa240009ef0 tx=0x7fa240011f90 comp rx=0 tx=0).stop 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 -- 172.21.15.7:0/3490291798 shutdown_connections 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5f0f00 0x55ecec5ea280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5ebd90 0x55ecec5fcdc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 --2- 172.21.15.7:0/3490291798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 -- 172.21.15.7:0/3490291798 >> 172.21.15.7:0/3490291798 conn(0x55ecec5e6910 msgr2=0x55ecec5e9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 -- 172.21.15.7:0/3490291798 shutdown_connections 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.992+0000 7fa25b191740 1 -- 172.21.15.7:0/3490291798 wait complete. 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 -- start start 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 Processor -- start 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501a8910 con 0x55ecec5f0f00 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501a9d80 con 0x55ecec5eb3a0 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501a8fe0 con 0x55ecec5ebd90 2024-03-19T00:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa25b191740 10 client.0 _ll_register_callbacks cb 0x7fa24c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa255a61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa255a61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5fc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54696/0 (socket says 172.21.15.7:54696) 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa255a61640 1 -- 172.21.15.7:0/2156067830 learned_addr learned my addr 172.21.15.7:0/2156067830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa256262640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa255a61640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa256262640 1 -- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 msgr2=0x55ecec5fac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa256262640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5fac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa256262640 1 -- 172.21.15.7:0/2156067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 msgr2=0x55ecec5fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa256262640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa256262640 1 -- 172.21.15.7:0/2156067830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecec5fc730 con 0x55ecec5ebd90 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.993+0000 7fa255a61640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa256262640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5fb180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2400054d0 tx=0x7fa240009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71a0f0f533d4c29e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa239ffb640 1 -- 172.21.15.7:0/2156067830 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240036020 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24000f9f0 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa239ffb640 1 -- 172.21.15.7:0/2156067830 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa240005600 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa239ffb640 1 -- 172.21.15.7:0/2156067830 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa24001e040 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 10 client.26413 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecec668120 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa239ffb640 1 -- 172.21.15.7:0/2156067830 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24000f9f0 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa239ffb640 1 -- 172.21.15.7:0/2156067830 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa2400035f0 con 0x55ecec5ebd90 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 10 client.26413 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 20 client.26413 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec668920 0x55ecec66ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 4 client.26413 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.994+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecec5579f0 con 0x55ecec668920 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.995+0000 7fa255a61640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec668920 0x55ecec66ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.995+0000 7fa255a61640 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec668920 0x55ecec66ad40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35f2762eaac8555 server_cookie=e92596bc210387a1 in_seq=0 out_seq=0 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.995+0000 7fa239ffb640 10 client.26413 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa239ffb640 1 -- 172.21.15.7:0/2156067830 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecec5579f0 con 0x55ecec668920 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa239ffb640 10 client.26413 handle_command_reply: tid=0 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 client.26413 shutdown 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 msgr2=0x55ecec5fb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5fb180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2400054d0 tx=0x7fa240009570 comp rx=0 tx=0).stop 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 shutdown_connections 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ecec668920 0x55ecec66ad40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecec5f0f00 0x55ecec5fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecec5ebd90 0x55ecec5fb180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa239ffb640 0 client.26413 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 --2- 172.21.15.7:0/2156067830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ecec5eb3a0 0x55ecec5fac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 >> 172.21.15.7:0/2156067830 conn(0x55ecec5e6910 msgr2=0x55ecec5ea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 shutdown_connections 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 1 -- 172.21.15.7:0/2156067830 wait complete. 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.996+0000 7fa25b191740 20 client.26413 trim_cache size 0 max 16384 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2e1b49b7-65df-4f9f-8b94-c17db6759471", 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:43:22.005 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:43:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa23c0a2800 msgr2=0x7fa23c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa23c0a2800 0x7fa23c0a4cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa248098230 tx=0x7fa2480973e0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 msgr2=0x7fa25019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 0x7fa25019f060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa24800efd0 tx=0x7fa24800c6a0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 shutdown_connections 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa23c0a2800 0x7fa23c0a4cc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2501022f0 0x7fa25019f060 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa250101930 0x7fa250197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 --2- 172.21.15.7:0/1072222996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa250100740 0x7fa250197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 >> 172.21.15.7:0/1072222996 conn(0x7fa2500fbcb0 msgr2=0x7fa250068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 shutdown_connections 2024-03-19T00:43:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:21.997+0000 7fa2584c6640 1 -- 172.21.15.7:0/1072222996 wait complete. 2024-03-19T00:43:22.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:43:22.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:43:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:21 smithi044 ceph-mon[66366]: pgmap v513: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 -- 172.21.15.7:0/863445015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 msgr2=0x7efc68101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 --2- 172.21.15.7:0/863445015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68101db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009900 tx=0x7efc5c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:43:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 -- 172.21.15.7:0/863445015 shutdown_connections 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 --2- 172.21.15.7:0/863445015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc681022f0 0x7efc68108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 --2- 172.21.15.7:0/863445015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68101db0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 --2- 172.21.15.7:0/863445015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc68100740 0x7efc68100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 -- 172.21.15.7:0/863445015 >> 172.21.15.7:0/863445015 conn(0x7efc680fbcb0 msgr2=0x7efc680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 -- 172.21.15.7:0/863445015 shutdown_connections 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.112+0000 7efc6d081640 1 -- 172.21.15.7:0/863445015 wait complete. 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 Processor -- start 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 -- start start 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68100740 0x7efc68197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc681022f0 0x7efc6819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810d390 con 0x7efc68101930 2024-03-19T00:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810d210 con 0x7efc68100740 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.113+0000 7efc6d081640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810d510 con 0x7efc681022f0 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54714/0 (socket says 172.21.15.7:54714) 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68100740 0x7efc68197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 -- 172.21.15.7:0/2578520295 learned_addr learned my addr 172.21.15.7:0/2578520295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6759e640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc681022f0 0x7efc6819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 -- 172.21.15.7:0/2578520295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc681022f0 msgr2=0x7efc6819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc681022f0 0x7efc6819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 -- 172.21.15.7:0/2578520295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68100740 msgr2=0x7efc68197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68100740 0x7efc68197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 -- 172.21.15.7:0/2578520295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6819f6c0 con 0x7efc68101930 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc66d9d640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68100740 0x7efc68197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6659c640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68197fe0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006010 tx=0x7efc5c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc4ffff640 1 -- 172.21.15.7:0/2578520295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c043020 con 0x7efc68101930 2024-03-19T00:43:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc4ffff640 1 -- 172.21.15.7:0/2578520295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc5c0043d0 con 0x7efc68101930 2024-03-19T00:43:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6819f950 con 0x7efc68101930 2024-03-19T00:43:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc4ffff640 1 -- 172.21.15.7:0/2578520295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c003b80 con 0x7efc68101930 2024-03-19T00:43:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.114+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6819fea0 con 0x7efc68101930 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc4ffff640 1 -- 172.21.15.7:0/2578520295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7efc5c046020 con 0x7efc68101930 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 Processor -- start 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 -- start start 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 0x5618cdf8ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf91a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5618cdeee500 con 0x5618cdf8a440 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5618cde96be0 con 0x5618cdf8f5b0 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5618cdeea5a0 con 0x5618cdf89a50 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 0x5618cdf8ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 0x5618cdf8ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54730/0 (socket says 172.21.15.7:54730) 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 -- 172.21.15.7:0/3891933914 learned_addr learned my addr 172.21.15.7:0/3891933914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6659c640 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf91a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6759e640 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 -- 172.21.15.7:0/3891933914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 msgr2=0x5618cdf89e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf89e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc4ffff640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efc500a2800 0x7efc500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 -- 172.21.15.7:0/3891933914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 msgr2=0x5618cdf91a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf91a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc66d9d640 1 -- 172.21.15.7:0/3891933914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618cdeb8000 con 0x5618cdf8a440 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc6759e640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efc500a2800 0x7efc500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.117+0000 7efc4ffff640 1 -- 172.21.15.7:0/2578520295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efc5c0f0fd0 con 0x7efc68101930 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc66d9d640 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 0x5618cdf8ef40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efc580114f0 tx=0x7efc580119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f4c230ec21ff0c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc4d7fa640 1 -- 172.21.15.7:0/3891933914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5801e070 con 0x5618cdf8a440 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc4d7fa640 1 -- 172.21.15.7:0/3891933914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc580035f0 con 0x5618cdf8a440 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc4d7fa640 1 -- 172.21.15.7:0/3891933914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800f820 con 0x5618cdf8a440 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3891933914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 msgr2=0x5618cdf8ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6759e640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efc500a2800 0x7efc500a4cc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5618cdf8aca0 tx=0x7efc60009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 0x5618cdf8ef40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efc580114f0 tx=0x7efc580119b0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3891933914 shutdown_connections 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf91a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8a440 0x5618cdf8ef40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3891933914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf89e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3891933914 >> 172.21.15.7:0/3891933914 conn(0x5618cdf84fc0 msgr2=0x5618cdf853c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.118+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3891933914 shutdown_connections 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3891933914 wait complete. 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 -- start start 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 Processor -- start 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf93f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf94440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5618cdf92560 con 0x5618cdf89a50 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5618cdeaa160 con 0x5618cdf8a440 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5618cdf926e0 con 0x5618cdf8f5b0 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf93f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6fc6d740 10 client.0 _ll_register_callbacks cb 0x5618cdee6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf93f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54740/0 (socket says 172.21.15.7:54740) 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 -- 172.21.15.7:0/1633181695 learned_addr learned my addr 172.21.15.7:0/1633181695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6659c640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf94440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc6759e640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 -- 172.21.15.7:0/1633181695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8f5b0 msgr2=0x5618cdf9b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 -- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 msgr2=0x5618cdf94440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.119+0000 7efc66d9d640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf94440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc66d9d640 1 -- 172.21.15.7:0/1633181695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618cdccc7d0 con 0x5618cdf89a50 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc66d9d640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf93f00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efc580114c0 tx=0x7efc58004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261e3fc7ff627236 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc4aff5640 1 -- 172.21.15.7:0/1633181695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58027020 con 0x5618cdf89a50 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc4aff5640 1 -- 172.21.15.7:0/1633181695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efc58005040 con 0x5618cdf89a50 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc580035f0 con 0x5618cdf89a50 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc4aff5640 1 -- 172.21.15.7:0/1633181695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc58032050 con 0x5618cdf89a50 2024-03-19T00:43:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 10 client.16668 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618cdc023f0 con 0x5618cdf89a50 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc4aff5640 1 -- 172.21.15.7:0/1633181695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc580035f0 con 0x5618cdf89a50 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc4aff5640 1 -- 172.21.15.7:0/1633181695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efc5800f5c0 con 0x5618cdf89a50 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 10 client.16668 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 20 client.16668 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618cdf9fac0 0x5618cdfa1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 4 client.16668 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.120+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618cdefba40 con 0x5618cdf9fac0 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.121+0000 7efc6659c640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618cdf9fac0 0x5618cdfa1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.121+0000 7efc4aff5640 10 client.16668 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.121+0000 7efc6659c640 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618cdf9fac0 0x5618cdfa1f60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96bb5b58f1e958e2 server_cookie=811ae77a963ce8e2 in_seq=0 out_seq=0 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc4aff5640 1 -- 172.21.15.7:0/1633181695 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5618cdefba40 con 0x5618cdf9fac0 2024-03-19T00:43:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc4aff5640 10 client.16668 handle_command_reply: tid=0 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 client.16668 shutdown 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 msgr2=0x5618cdf93f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf93f00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efc580114c0 tx=0x7efc58004230 comp rx=0 tx=0).stop 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 shutdown_connections 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf94440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618cdf9fac0 0x5618cdfa1f60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1633181695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf93f00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 >> 172.21.15.7:0/1633181695 conn(0x5618cdf84fc0 msgr2=0x5618cdf841e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc4aff5640 0 client.16668 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 shutdown_connections 2024-03-19T00:43:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1633181695 wait complete. 2024-03-19T00:43:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.122+0000 7efc6fc6d740 20 client.16668 trim_cache size 0 max 16384 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.139+0000 7efc6fc6d740 1 Processor -- start 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.139+0000 7efc6fc6d740 1 -- start start 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 0x5618cdf93520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc681acad0 con 0x5618cdf8f5b0 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc681aa3a0 con 0x5618cdf8a440 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a81f0 con 0x5618cdf89a50 2024-03-19T00:43:22.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 0x5618cdf93520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 0x5618cdf93520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54742/0 (socket says 172.21.15.7:54742) 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 -- 172.21.15.7:0/1115458103 learned_addr learned my addr 172.21.15.7:0/1115458103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc66d9d640 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6659c640 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 -- 172.21.15.7:0/1115458103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 msgr2=0x5618cdf801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 -- 172.21.15.7:0/1115458103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 msgr2=0x5618cdf9b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf9b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 -- 172.21.15.7:0/1115458103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618cdbc20f0 con 0x5618cdf8f5b0 2024-03-19T00:43:22.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc66d9d640 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc6759e640 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 0x5618cdf93520 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efc60003ef0 tx=0x7efc6001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbef7b5e1f5e76e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc4cff9640 1 -- 172.21.15.7:0/1115458103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60004b20 con 0x5618cdf8f5b0 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc4cff9640 1 -- 172.21.15.7:0/1115458103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc60004cc0 con 0x5618cdf8f5b0 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.140+0000 7efc4cff9640 1 -- 172.21.15.7:0/1115458103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60005450 con 0x5618cdf8f5b0 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1115458103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 msgr2=0x5618cdf93520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 0x5618cdf93520 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efc60003ef0 tx=0x7efc6001a000 comp rx=0 tx=0).stop 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1115458103 shutdown_connections 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf8f5b0 0x5618cdf93520 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8a440 0x5618cdf9b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/1115458103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf89a50 0x5618cdf801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1115458103 >> 172.21.15.7:0/1115458103 conn(0x5618cdf84fc0 msgr2=0x5618cdf88ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1115458103 shutdown_connections 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 -- 172.21.15.7:0/1115458103 wait complete. 2024-03-19T00:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 -- start start 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.141+0000 7efc6fc6d740 1 Processor -- start 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf98c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8a440 0x5618cdf99150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a8860 con 0x5618cdf89a50 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a9cd0 con 0x5618cdf8f5b0 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc681a8f30 con 0x5618cdf8a440 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 10 client.0 _ll_register_callbacks cb 0x7efc5801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf98c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf98c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54752/0 (socket says 172.21.15.7:54752) 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 -- 172.21.15.7:0/3819822862 learned_addr learned my addr 172.21.15.7:0/3819822862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6759e640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6659c640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8a440 0x5618cdf99150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 -- 172.21.15.7:0/3819822862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8a440 msgr2=0x5618cdf99150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8a440 0x5618cdf99150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 -- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 msgr2=0x5618cdf9a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 -- 172.21.15.7:0/3819822862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618cdf9a750 con 0x5618cdf89a50 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc66d9d640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf98c10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efc5800fab0 tx=0x7efc58005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735056f2f1b939c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc4aff5640 1 -- 172.21.15.7:0/3819822862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58027040 con 0x5618cdf89a50 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc4aff5640 1 -- 172.21.15.7:0/3819822862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efc58005040 con 0x5618cdf89a50 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc60004cc0 con 0x5618cdf89a50 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc4aff5640 1 -- 172.21.15.7:0/3819822862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc5801e070 con 0x5618cdf89a50 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 10 client.16674 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.142+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618ce005ec0 con 0x5618cdf89a50 2024-03-19T00:43:22.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc4aff5640 1 -- 172.21.15.7:0/3819822862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc60004cc0 con 0x5618cdf89a50 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc4aff5640 1 -- 172.21.15.7:0/3819822862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efc58004630 con 0x5618cdf89a50 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 10 client.16674 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 20 client.16674 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618ce0068e0 0x5618ce008d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 4 client.16674 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618cdef5d90 con 0x5618ce0068e0 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6659c640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618ce0068e0 0x5618ce008d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc4aff5640 10 client.16674 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6659c640 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618ce0068e0 0x5618ce008d00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b00a56d02d7ec0a server_cookie=5ff65238494816dd in_seq=0 out_seq=0 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc4aff5640 1 -- 172.21.15.7:0/3819822862 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5618cdef5d90 con 0x5618ce0068e0 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc4aff5640 10 client.16674 handle_command_reply: tid=0 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 1 client.16674 shutdown 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 msgr2=0x5618cdf98c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.143+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf98c10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efc5800fab0 tx=0x7efc58005db0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 shutdown_connections 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618cdf8f5b0 0x5618cdf9a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618cdf8a440 0x5618cdf99150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5618ce0068e0 0x5618ce008d00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 --2- 172.21.15.7:0/3819822862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5618cdf89a50 0x5618cdf98c10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc4aff5640 0 client.16674 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 >> 172.21.15.7:0/3819822862 conn(0x5618cdf84fc0 msgr2=0x5618cdf83950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 shutdown_connections 2024-03-19T00:43:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 1 -- 172.21.15.7:0/3819822862 wait complete. 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6fc6d740 20 client.16674 trim_cache size 0 max 16384 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout: "2e1b49b7-65df-4f9f-8b94-c17db6759471": { 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2e1b49b7-65df-4f9f-8b94-c17db6759471", 2024-03-19T00:43:22.154 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:43:22.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:43:22.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:43:22.155 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:43:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efc500a2800 msgr2=0x7efc500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efc500a2800 0x7efc500a4cc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5618cdf8aca0 tx=0x7efc60009500 comp rx=0 tx=0).stop 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 msgr2=0x7efc68197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68197fe0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006010 tx=0x7efc5c005e40 comp rx=0 tx=0).stop 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 shutdown_connections 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7efc500a2800 0x7efc500a4cc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc681022f0 0x7efc6819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc68101930 0x7efc68197fe0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 --2- 172.21.15.7:0/2578520295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68100740 0x7efc68197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 >> 172.21.15.7:0/2578520295 conn(0x7efc680fbcb0 msgr2=0x7efc68104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 shutdown_connections 2024-03-19T00:43:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:22.144+0000 7efc6d081640 1 -- 172.21.15.7:0/2578520295 wait complete. 2024-03-19T00:43:22.157 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:21 smithi007 ceph-mon[62854]: pgmap v513: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:22.158 INFO:tasks.cephfs.filesystem:scrub status for tag:2e1b49b7-65df-4f9f-8b94-c17db6759471 - {'path': '/', 'tag': '2e1b49b7-65df-4f9f-8b94-c17db6759471', 'options': 'recursive,force'} 2024-03-19T00:43:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:21 smithi204 ceph-mon[66572]: pgmap v513: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:189/600: open fd = 3 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:190/600: open fd = 3 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:191/600: open fd = 3 2024-03-19T00:43:22.783 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:192/600: open fd = 3 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:193/600: open fd = 3 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.784 INFO:tasks.workunit.client.0.smithi007.stdout:194/600: open fd = 3 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:195/600: open fd = 3 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:196/600: open fd = 3 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.785 INFO:tasks.workunit.client.0.smithi007.stdout:197/600: open fd = 3 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:198/600: open fd = 3 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:199/600: open fd = 3 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.786 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:200/600: open fd = 3 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:201/600: open fd = 3 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:202/600: open fd = 3 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:203/600: open fd = 3 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:204/600: open fd = 3 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:205/600: open fd = 3 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:206/600: open fd = 3 2024-03-19T00:43:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:207/600: open fd = 3 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:208/600: open fd = 3 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:209/600: open fd = 3 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.789 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:210/600: open fd = 3 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:211/600: open fd = 3 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:212/600: open fd = 3 2024-03-19T00:43:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:213/600: open fd = 3 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:214/600: open fd = 3 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:215/600: open fd = 3 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:216/600: open fd = 3 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:217/600: open fd = 3 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:218/600: open fd = 3 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:219/600: open fd = 3 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:220/600: open fd = 3 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.793 INFO:tasks.workunit.client.0.smithi007.stdout:221/600: open fd = 3 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:222/600: open fd = 3 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:223/600: open fd = 3 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.794 INFO:tasks.workunit.client.0.smithi007.stdout:224/600: open fd = 3 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:225/600: open fd = 3 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:226/600: open fd = 3 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:227/600: open fd = 3 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:228/600: open fd = 3 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:229/600: open fd = 3 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.796 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:230/600: open fd = 3 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:231/600: open fd = 3 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:232/600: open fd = 3 2024-03-19T00:43:22.797 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:233/600: open fd = 3 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:234/600: open fd = 3 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:235/600: open fd = 3 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:236/600: open fd = 3 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:237/600: open fd = 3 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:238/600: open fd = 3 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:239/600: open fd = 3 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:240/600: open fd = 3 2024-03-19T00:43:22.803 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:241/600: open fd = 3 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:242/600: open fd = 3 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.804 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:243/600: open fd = 3 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:244/600: open fd = 3 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:245/600: open fd = 3 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.805 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:246/600: open fd = 3 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:247/600: open fd = 3 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.806 INFO:tasks.workunit.client.0.smithi007.stdout:248/600: open fd = 3 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:249/600: open fd = 3 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:250/600: open fd = 3 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:22.807 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:43:23.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:22 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:43:23.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:22 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:22 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:43:23.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:22 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:43:23.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:22 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:22 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:22 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:43:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:22 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:22 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:43:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:22 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:43:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:22 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:22 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:22 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:43:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:22 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:22 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:43:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:22 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:43:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:22 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:22 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:23 smithi044 ceph-mon[66366]: pgmap v514: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:24.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:23 smithi007 ceph-mon[62854]: pgmap v514: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:23 smithi204 ceph-mon[66572]: pgmap v514: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:25 smithi044 ceph-mon[66366]: pgmap v515: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:26.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:25 smithi007 ceph-mon[62854]: pgmap v515: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:25 smithi204 ceph-mon[66572]: pgmap v515: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:27 smithi044 ceph-mon[66366]: pgmap v516: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-03-19T00:43:28.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:27 smithi007 ceph-mon[62854]: pgmap v516: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-03-19T00:43:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:27 smithi204 ceph-mon[66572]: pgmap v516: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-03-19T00:43:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:29 smithi044 ceph-mon[66366]: pgmap v517: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2024-03-19T00:43:30.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:29 smithi007 ceph-mon[62854]: pgmap v517: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2024-03-19T00:43:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:29 smithi204 ceph-mon[66572]: pgmap v517: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2024-03-19T00:43:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:31 smithi044 ceph-mon[66366]: pgmap v518: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:31 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:43:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:31 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:43:32.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:31 smithi007 ceph-mon[62854]: pgmap v518: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:32.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:31 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:43:32.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:31 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:43:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:31 smithi204 ceph-mon[66572]: pgmap v518: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:43:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:31 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:43:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:31 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:43:34.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:33 smithi044 ceph-mon[66366]: pgmap v519: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:34.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:33 smithi007 ceph-mon[62854]: pgmap v519: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:33 smithi204 ceph-mon[66572]: pgmap v519: 129 pgs: 129 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:35 smithi044 ceph-mon[66366]: pgmap v520: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-03-19T00:43:36.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:35 smithi007 ceph-mon[62854]: pgmap v520: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-03-19T00:43:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:35 smithi204 ceph-mon[66572]: pgmap v520: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-03-19T00:43:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:37 smithi044 ceph-mon[66366]: pgmap v521: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:43:38.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:37 smithi007 ceph-mon[62854]: pgmap v521: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:43:38.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:37 smithi204 ceph-mon[66572]: pgmap v521: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:39 smithi044 ceph-mon[66366]: pgmap v522: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:43:40.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:39 smithi007 ceph-mon[62854]: pgmap v522: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:43:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:39 smithi204 ceph-mon[66572]: pgmap v522: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-19T00:43:42.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:41 smithi204 ceph-mon[66572]: pgmap v523: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 14 op/s 2024-03-19T00:43:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:41 smithi044 ceph-mon[66366]: pgmap v523: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 14 op/s 2024-03-19T00:43:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:41 smithi007 ceph-mon[62854]: pgmap v523: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 14 op/s 2024-03-19T00:43:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:43 smithi044 ceph-mon[66366]: pgmap v524: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-03-19T00:43:44.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:43 smithi007 ceph-mon[62854]: pgmap v524: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-03-19T00:43:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:43 smithi204 ceph-mon[66572]: pgmap v524: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-03-19T00:43:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:45 smithi044 ceph-mon[66366]: pgmap v525: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:43:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:45 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:43:46.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:45 smithi007 ceph-mon[62854]: pgmap v525: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:43:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:45 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:43:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:45 smithi204 ceph-mon[66572]: pgmap v525: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:43:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:45 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:43:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:46 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:43:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:46 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:43:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:46 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:43:47.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:46 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:43:47.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:46 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:43:47.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:46 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:43:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:46 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:43:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:46 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:43:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:46 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:43:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:47 smithi204 ceph-mon[66572]: pgmap v526: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:43:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:47 smithi044 ceph-mon[66366]: pgmap v526: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:43:48.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:47 smithi007 ceph-mon[62854]: pgmap v526: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:43:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:49 smithi204 ceph-mon[66572]: pgmap v527: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-03-19T00:43:50.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:49 smithi007 ceph-mon[62854]: pgmap v527: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-03-19T00:43:50.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:49 smithi044 ceph-mon[66366]: pgmap v527: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-03-19T00:43:52.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:43:52.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:43:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:51 smithi204 ceph-mon[66572]: pgmap v528: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-03-19T00:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 -- 172.21.15.7:0/1851149989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d81033e0 msgr2=0x7fb8d8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1851149989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d81033e0 0x7fb8d8109c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009be0 tx=0x7fb8d002bef0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 -- 172.21.15.7:0/1851149989 shutdown_connections 2024-03-19T00:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1851149989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d81033e0 0x7fb8d8109c70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1851149989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8102a20 0x7fb8d8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1851149989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d8101830 0x7fb8d8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 -- 172.21.15.7:0/1851149989 >> 172.21.15.7:0/1851149989 conn(0x7fb8d80fcda0 msgr2=0x7fb8d80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 -- 172.21.15.7:0/1851149989 shutdown_connections 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.278+0000 7fb8deb09640 1 -- 172.21.15.7:0/1851149989 wait complete. 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 Processor -- start 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 -- start start 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 0x7fb8d8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d8102a20 0x7fb8d8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81033e0 0x7fb8d819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810d360 con 0x7fb8d8101830 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810d1e0 con 0x7fb8d8102a20 2024-03-19T00:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8deb09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810d4e0 con 0x7fb8d81033e0 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8dc8a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 0x7fb8d8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8dc8a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 0x7fb8d8197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44670/0 (socket says 172.21.15.7:44670) 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/1150186632 learned_addr learned my addr 172.21.15.7:0/1150186632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d8102a20 0x7fb8d8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.279+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81033e0 0x7fb8d819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/1150186632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81033e0 msgr2=0x7fb8d819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81033e0 0x7fb8d819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/1150186632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d8102a20 msgr2=0x7fb8d8197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d8102a20 0x7fb8d8197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/1150186632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d819f660 con 0x7fb8d8101830 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 0x7fb8d8197a40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00e9c0 tx=0x7fb8cc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8d5ffb640 1 -- 172.21.15.7:0/1150186632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00ccc0 con 0x7fb8d8101830 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d819f950 con 0x7fb8d8101830 2024-03-19T00:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8d5ffb640 1 -- 172.21.15.7:0/1150186632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8cc0035f0 con 0x7fb8d8101830 2024-03-19T00:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8d5ffb640 1 -- 172.21.15.7:0/1150186632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00ce60 con 0x7fb8d8101830 2024-03-19T00:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.280+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d819fea0 con 0x7fb8d8101830 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.282+0000 7fb8d5ffb640 1 -- 172.21.15.7:0/1150186632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb8cc0037a0 con 0x7fb8d8101830 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 Processor -- start 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8d5ffb640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 -- start start 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 0x55f5672d8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8d5ffb640 1 -- 172.21.15.7:0/1150186632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb8cc0cf2f0 con 0x7fb8d8101830 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672dd9c0 0x55f5672dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f567238500 con 0x55f5672d8850 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f5671e0be0 con 0x55f5672d7e60 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f5672345a0 con 0x55f5672dd9c0 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 0x55f5672d8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 0x55f5672d8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45946/0 (socket says 172.21.15.7:45946) 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/3261538594 learned_addr learned my addr 172.21.15.7:0/3261538594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672dd9c0 0x55f5672dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/3261538594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672dd9c0 msgr2=0x55f5672dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672dd9c0 0x55f5672dfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/3261538594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 msgr2=0x55f5672dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dc8a5640 1 -- 172.21.15.7:0/3261538594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f567202000 con 0x55f5672d7e60 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672dd9c0 0x55f5672dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:43:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.283+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0004af0 tx=0x7fb8d0033780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 0x55f5672d8280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc09b860 tx=0x7fb8cc09b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42a905cbc2fc5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8c2ffd640 1 -- 172.21.15.7:0/3261538594 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc084e60 con 0x55f5672d7e60 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8c2ffd640 1 -- 172.21.15.7:0/3261538594 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8cc09b8d0 con 0x55f5672d7e60 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8c2ffd640 1 -- 172.21.15.7:0/3261538594 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0860c0 con 0x55f5672d7e60 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3261538594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 msgr2=0x55f5672d8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 0x55f5672d8280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc09b860 tx=0x7fb8cc09b7d0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3261538594 shutdown_connections 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672dd9c0 0x55f5672dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3261538594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672d7e60 0x55f5672d8280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3261538594 >> 172.21.15.7:0/3261538594 conn(0x55f5672cefc0 msgr2=0x55f5672cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3261538594 shutdown_connections 2024-03-19T00:43:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.284+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3261538594 wait complete. 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 -- start start 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 Processor -- start 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672dd9c0 0x55f5672e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f5672e08b0 con 0x55f5672d8850 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f5671f4160 con 0x55f5672dd9c0 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f5672e0a30 con 0x55f5672d7e60 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8e17d2740 10 client.0 _ll_register_callbacks cb 0x55f567230a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44678/0 (socket says 172.21.15.7:44678) 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8dc8a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8d7fff640 1 -- 172.21.15.7:0/1797069797 learned_addr learned my addr 172.21.15.7:0/1797069797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.285+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672dd9c0 0x55f5672e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8d7fff640 1 -- 172.21.15.7:0/1797069797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 msgr2=0x55f5672e2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8d7fff640 1 -- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672dd9c0 msgr2=0x55f5672e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672dd9c0 0x55f5672e9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8d7fff640 1 -- 172.21.15.7:0/1797069797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5670167d0 con 0x55f5672d8850 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672dd9c0 0x55f5672e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e27c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40126b0 tx=0x7fb8c4012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7905cfe975145a61 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8b7fff640 1 -- 172.21.15.7:0/1797069797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400f8c0 con 0x55f5672d8850 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8b7fff640 1 -- 172.21.15.7:0/1797069797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb8c400fa60 con 0x55f5672d8850 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc09b8d0 con 0x55f5672d8850 2024-03-19T00:43:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8b7fff640 1 -- 172.21.15.7:0/1797069797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8c400fc60 con 0x55f5672d8850 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8e17d2740 10 client.16686 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f566f4c3f0 con 0x55f5672d8850 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.286+0000 7fb8b7fff640 1 -- 172.21.15.7:0/1797069797 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8cc09b8d0 con 0x55f5672d8850 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8b7fff640 1 -- 172.21.15.7:0/1797069797 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c401c420 con 0x55f5672d8850 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8e17d2740 10 client.16686 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8e17d2740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8e17d2740 20 client.16686 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672ee120 0x55f5672f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8e17d2740 4 client.16686 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f567245a40 con 0x55f5672ee120 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672ee120 0x55f5672f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672ee120 0x55f5672f05c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd2dd46c6332205 server_cookie=575ef2ff57cc5c93 in_seq=0 out_seq=0 2024-03-19T00:43:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.287+0000 7fb8b7fff640 10 client.16686 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8b7fff640 1 -- 172.21.15.7:0/1797069797 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f567245a40 con 0x55f5672ee120 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8b7fff640 10 client.16686 handle_command_reply: tid=0 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 client.16686 shutdown 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 msgr2=0x55f5672e27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e27c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40126b0 tx=0x7fb8c4012b70 comp rx=0 tx=0).stop 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 shutdown_connections 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672dd9c0 0x55f5672e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672ee120 0x55f5672f05c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e27c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/1797069797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8b7fff640 0 client.16686 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 >> 172.21.15.7:0/1797069797 conn(0x55f5672cefc0 msgr2=0x55f5672ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 shutdown_connections 2024-03-19T00:43:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 1 -- 172.21.15.7:0/1797069797 wait complete. 2024-03-19T00:43:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.288+0000 7fb8e17d2740 20 client.16686 trim_cache size 0 max 16384 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 Processor -- start 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 -- start start 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81aa990 con 0x55f5672d8850 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81a8260 con 0x55f5672cefc0 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81a60b0 con 0x55f5672d7e60 2024-03-19T00:43:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8dd0a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8dd0a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44686/0 (socket says 172.21.15.7:44686) 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/3082747199 learned_addr learned my addr 172.21.15.7:0/3082747199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.306+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/3082747199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 msgr2=0x55f5672da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/3082747199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 msgr2=0x55f5672d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672d9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/3082747199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f566f0c0f0 con 0x55f5672d8850 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e9750 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009be0 tx=0x7fb8d0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc99da93df564d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8c27fc640 1 -- 172.21.15.7:0/3082747199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d002c890 con 0x55f5672d8850 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8c27fc640 1 -- 172.21.15.7:0/3082747199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d0004040 con 0x55f5672d8850 2024-03-19T00:43:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.307+0000 7fb8c27fc640 1 -- 172.21.15.7:0/3082747199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0034690 con 0x55f5672d8850 2024-03-19T00:43:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3082747199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 msgr2=0x55f5672e9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e9750 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009be0 tx=0x7fb8d0046000 comp rx=0 tx=0).stop 2024-03-19T00:43:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3082747199 shutdown_connections 2024-03-19T00:43:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e9750 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/3082747199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3082747199 >> 172.21.15.7:0/3082747199 conn(0x55f5672cf710 msgr2=0x55f5672e2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3082747199 shutdown_connections 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- 172.21.15.7:0/3082747199 wait complete. 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- start start 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 Processor -- start 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81a6720 con 0x55f5672d8850 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81a7b90 con 0x55f5672cefc0 2024-03-19T00:43:52.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81a6df0 con 0x55f5672d7e60 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.308+0000 7fb8e17d2740 10 client.0 _ll_register_callbacks cb 0x7fb8c4009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44688/0 (socket says 172.21.15.7:44688) 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/455591625 learned_addr learned my addr 172.21.15.7:0/455591625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8d7fff640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/455591625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 msgr2=0x55f5672e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 msgr2=0x55f5672e6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672e6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 -- 172.21.15.7:0/455591625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5672e8820 con 0x55f5672d8850 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8dd0a6640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e8230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb8d002c430 tx=0x7fb8d0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bbae44a86e1647 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:52.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8b7fff640 1 -- 172.21.15.7:0/455591625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d005d070 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d0004040 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8b7fff640 1 -- 172.21.15.7:0/455591625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb8d002cca0 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8b7fff640 1 -- 172.21.15.7:0/455591625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8d0065050 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8e17d2740 10 client.16692 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.309+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5672e9b80 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8b7fff640 1 -- 172.21.15.7:0/455591625 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d0004040 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8b7fff640 1 -- 172.21.15.7:0/455591625 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8d0060030 con 0x55f5672d8850 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8e17d2740 10 client.16692 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8e17d2740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8e17d2740 20 client.16692 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672dd9c0 0x55f567356ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8e17d2740 4 client.16692 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f56723fd90 con 0x55f5672dd9c0 2024-03-19T00:43:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672dd9c0 0x55f567356ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8b7fff640 10 client.16692 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.310+0000 7fb8dc8a5640 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672dd9c0 0x55f567356ce0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53da5f3bfbdc97f4 server_cookie=22d8144124d3c4d4 in_seq=0 out_seq=0 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8b7fff640 1 -- 172.21.15.7:0/455591625 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f56723fd90 con 0x55f5672dd9c0 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8b7fff640 10 client.16692 handle_command_reply: tid=0 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 client.16692 shutdown 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 msgr2=0x55f5672e8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e8230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb8d002c430 tx=0x7fb8d0005bd0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 shutdown_connections 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f5672dd9c0 0x55f567356ce0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5672d8850 0x55f5672e8230 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5672d7e60 0x55f5672e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8b7fff640 0 client.16692 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 --2- 172.21.15.7:0/455591625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f5672cefc0 0x55f5672e6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 >> 172.21.15.7:0/455591625 conn(0x55f5672cf710 msgr2=0x55f5672e2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 shutdown_connections 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 1 -- 172.21.15.7:0/455591625 wait complete. 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8e17d2740 20 client.16692 trim_cache size 0 max 16384 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:43:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:43:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:43:52.320 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:43:52.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8c80a2800 msgr2=0x7fb8c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0004af0 tx=0x7fb8d0033780 comp rx=0 tx=0).stop 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 msgr2=0x7fb8d8197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.311+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 0x7fb8d8197a40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00e9c0 tx=0x7fb8cc00edc0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 shutdown_connections 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d81033e0 0x7fb8d819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d8102a20 0x7fb8d8197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 --2- 172.21.15.7:0/1150186632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8d8101830 0x7fb8d8197a40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 >> 172.21.15.7:0/1150186632 conn(0x7fb8d80fcda0 msgr2=0x7fb8d8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 shutdown_connections 2024-03-19T00:43:52.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.312+0000 7fb8deb09640 1 -- 172.21.15.7:0/1150186632 wait complete. 2024-03-19T00:43:52.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:51 smithi007 ceph-mon[62854]: pgmap v528: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-03-19T00:43:52.323 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:43:52.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:43:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:51 smithi044 ceph-mon[66366]: pgmap v528: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-03-19T00:43:52.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 -- 172.21.15.7:0/3163997688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734102300 msgr2=0x7f9734108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 --2- 172.21.15.7:0/3163997688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734102300 0x7f9734108b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f972c009be0 tx=0x7f972c02bef0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 -- 172.21.15.7:0/3163997688 shutdown_connections 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 --2- 172.21.15.7:0/3163997688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734102300 0x7f9734108b90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 --2- 172.21.15.7:0/3163997688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734101940 0x7f9734101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 --2- 172.21.15.7:0/3163997688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9734100750 0x7f9734100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 -- 172.21.15.7:0/3163997688 >> 172.21.15.7:0/3163997688 conn(0x7f97340fbcc0 msgr2=0x7f97340fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 -- 172.21.15.7:0/3163997688 shutdown_connections 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.425+0000 7f9738e0b640 1 -- 172.21.15.7:0/3163997688 wait complete. 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 Processor -- start 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 -- start start 2024-03-19T00:43:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 0x7f97341979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 0x7f9734197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9734102300 0x7f973419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f973410c260 con 0x7f9734101940 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f973410c0e0 con 0x7f9734100750 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9738e0b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f973410c3e0 con 0x7f9734102300 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9731d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 0x7f9734197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f973259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 0x7f97341979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f973259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 0x7f97341979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46010/0 (socket says 172.21.15.7:46010) 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f973259c640 1 -- 172.21.15.7:0/2049493055 learned_addr learned my addr 172.21.15.7:0/2049493055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9731d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 0x7f9734197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44700/0 (socket says 172.21.15.7:44700) 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.426+0000 7f9732d9d640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9734102300 0x7f973419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f973259c640 1 -- 172.21.15.7:0/2049493055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9734102300 msgr2=0x7f973419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f973259c640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9734102300 0x7f973419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f973259c640 1 -- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 msgr2=0x7f9734197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f973259c640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 0x7f9734197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f973259c640 1 -- 172.21.15.7:0/2049493055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973419f600 con 0x7f9734100750 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f9731d9b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 0x7f9734197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f973259c640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 0x7f97341979e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f972400e9c0 tx=0x7f972400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972400cd80 con 0x7f9734100750 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97240035f0 con 0x7f9734100750 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973419f8f0 con 0x7f9734100750 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724010690 con 0x7f9734100750 2024-03-19T00:43:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.427+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973419fe40 con 0x7f9734100750 2024-03-19T00:43:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.428+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97340686a0 con 0x7f9734100750 2024-03-19T00:43:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.429+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f97240037a0 con 0x7f9734100750 2024-03-19T00:43:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.430+0000 7f97237fe640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f971c0a2800 0x7f971c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.430+0000 7f9731d9b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f971c0a2800 0x7f971c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.430+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f97240cf1f0 con 0x7f9734100750 2024-03-19T00:43:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.430+0000 7f9731d9b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f971c0a2800 0x7f971c0a4cc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f97280096c0 tx=0x7f9728009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.434+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f972409bed0 con 0x7f9734100750 2024-03-19T00:43:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.655+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97341a00d0 con 0x7f9734100750 2024-03-19T00:43:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.656+0000 7f97237fe640 1 -- 172.21.15.7:0/2049493055 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f972409b7d0 con 0x7f9734100750 2024-03-19T00:43:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:43:52.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:43:52.657 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:43:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f971c0a2800 msgr2=0x7f971c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f971c0a2800 0x7f971c0a4cc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f97280096c0 tx=0x7f9728009260 comp rx=0 tx=0).stop 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 msgr2=0x7f97341979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 0x7f97341979e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f972400e9c0 tx=0x7f972400ee80 comp rx=0 tx=0).stop 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 shutdown_connections 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f971c0a2800 0x7f971c0a4cc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9734102300 0x7f973419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734101940 0x7f9734197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 --2- 172.21.15.7:0/2049493055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9734100750 0x7f97341979e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 >> 172.21.15.7:0/2049493055 conn(0x7f97340fbcc0 msgr2=0x7f9734068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 shutdown_connections 2024-03-19T00:43:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.658+0000 7f9738e0b640 1 -- 172.21.15.7:0/2049493055 wait complete. 2024-03-19T00:43:52.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:43:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 -- 172.21.15.7:0/964774294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4950102340 msgr2=0x7f4950104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 --2- 172.21.15.7:0/964774294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4950102340 0x7f4950104730 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4940009900 tx=0x7f494002bc30 comp rx=0 tx=0).stop 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 -- 172.21.15.7:0/964774294 shutdown_connections 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 --2- 172.21.15.7:0/964774294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49501075a0 0x7f4950109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 --2- 172.21.15.7:0/964774294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950104c70 0x7f4950107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 --2- 172.21.15.7:0/964774294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4950102340 0x7f4950104730 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 -- 172.21.15.7:0/964774294 >> 172.21.15.7:0/964774294 conn(0x7f49500fbcd0 msgr2=0x7f49500fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 -- 172.21.15.7:0/964774294 shutdown_connections 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.774+0000 7f4954ae8640 1 -- 172.21.15.7:0/964774294 wait complete. 2024-03-19T00:43:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 Processor -- start 2024-03-19T00:43:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 -- start start 2024-03-19T00:43:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 0x7f4950197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4950104c70 0x7f4950197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 0x7f495019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f495010d380 con 0x7f49501075a0 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f495010d200 con 0x7f4950102340 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f4954ae8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f495010d500 con 0x7f4950104c70 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.775+0000 7f494ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 0x7f495019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 0x7f4950197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 0x7f495019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44720/0 (socket says 172.21.15.7:44720) 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 0x7f4950197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46036/0 (socket says 172.21.15.7:46036) 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4950104c70 0x7f4950197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 -- 172.21.15.7:0/3714553544 learned_addr learned my addr 172.21.15.7:0/3714553544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 -- 172.21.15.7:0/3714553544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4950104c70 msgr2=0x7f4950197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4950104c70 0x7f4950197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 -- 172.21.15.7:0/3714553544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 msgr2=0x7f4950197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 0x7f4950197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 -- 172.21.15.7:0/3714553544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495019f6d0 con 0x7f49501075a0 2024-03-19T00:43:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494e59c640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 0x7f4950197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f494ed9d640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 0x7f495019f060 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f494800eb10 tx=0x7f494800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f4947fff640 1 -- 172.21.15.7:0/3714553544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948004b20 con 0x7f49501075a0 2024-03-19T00:43:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495019f9c0 con 0x7f49501075a0 2024-03-19T00:43:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f4947fff640 1 -- 172.21.15.7:0/3714553544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4948004cc0 con 0x7f49501075a0 2024-03-19T00:43:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f4947fff640 1 -- 172.21.15.7:0/3714553544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948010690 con 0x7f49501075a0 2024-03-19T00:43:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.776+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495019ff10 con 0x7f49501075a0 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.778+0000 7f4947fff640 1 -- 172.21.15.7:0/3714553544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f4948005020 con 0x7f49501075a0 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 Processor -- start 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 -- start start 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 0x559f7da7eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da83bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da84260 0x559f7da866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f7d9de4b0 con 0x559f7da7f0f0 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559f7d986d70 con 0x559f7da7e700 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f49577ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559f7d9da550 con 0x559f7da84260 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f4947fff640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f493c0a2800 0x7f493c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f494e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da83bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 0x559f7da7eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 0x559f7da7eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46038/0 (socket says 172.21.15.7:46038) 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f494e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da83bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f4945d9b640 1 -- 172.21.15.7:0/2243017206 learned_addr learned my addr 172.21.15.7:0/2243017206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f494ed9d640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f493c0a2800 0x7f493c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.779+0000 7f4945d9b640 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da84260 0x559f7da866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4947fff640 1 -- 172.21.15.7:0/3714553544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f49480cf350 con 0x7f49501075a0 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4945d9b640 1 -- 172.21.15.7:0/2243017206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da84260 msgr2=0x559f7da866f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4945d9b640 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da84260 0x559f7da866f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4945d9b640 1 -- 172.21.15.7:0/2243017206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 msgr2=0x559f7da83bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4945d9b640 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da83bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4945d9b640 1 -- 172.21.15.7:0/2243017206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7d9a9210 con 0x559f7da7e700 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4945d9b640 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 0x559f7da7eb20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f49380093e0 tx=0x7f4938009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0bded9847b7a9b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f494ed9d640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f493c0a2800 0x7f493c0a4cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f49480a3c00 tx=0x7f49480963c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4944d99640 1 -- 172.21.15.7:0/2243017206 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800f790 con 0x559f7da7e700 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4944d99640 1 -- 172.21.15.7:0/2243017206 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f493800fde0 con 0x559f7da7e700 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.780+0000 7f4944d99640 1 -- 172.21.15.7:0/2243017206 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493801fd20 con 0x559f7da7e700 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- 172.21.15.7:0/2243017206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 msgr2=0x559f7da7eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 0x559f7da7eb20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f49380093e0 tx=0x7f4938009370 comp rx=0 tx=0).stop 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- 172.21.15.7:0/2243017206 shutdown_connections 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da84260 0x559f7da866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da83bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- 172.21.15.7:0/2243017206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da7e700 0x559f7da7eb20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- 172.21.15.7:0/2243017206 >> 172.21.15.7:0/2243017206 conn(0x559f7da75860 msgr2=0x559f7da75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- 172.21.15.7:0/2243017206 shutdown_connections 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- 172.21.15.7:0/2243017206 wait complete. 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- start start 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 Processor -- start 2024-03-19T00:43:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da89130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f7da87210 con 0x559f7da7e700 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559f7d99a310 con 0x559f7da84260 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559f7da87390 con 0x559f7da7f0f0 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f49577ac740 10 client.0 _ll_register_callbacks cb 0x559f7d98e2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da88bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44742/0 (socket says 172.21.15.7:44742) 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.781+0000 7f4945d9b640 1 -- 172.21.15.7:0/252163059 learned_addr learned my addr 172.21.15.7:0/252163059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f494e59c640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da89130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 -- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 msgr2=0x559f7da89130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da89130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 -- 172.21.15.7:0/252163059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 msgr2=0x559f7da90170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da90170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 -- 172.21.15.7:0/252163059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7d6f23f0 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4945d9b640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da88bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4938005bc0 tx=0x7f4938023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8879674197e83 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4935ffb640 1 -- 172.21.15.7:0/252163059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938004040 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f493800fde0 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4935ffb640 1 -- 172.21.15.7:0/252163059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f49380041e0 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4935ffb640 1 -- 172.21.15.7:0/252163059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f49380043e0 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f49577ac740 10 client.16710 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f7d7bc7d0 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.782+0000 7f4935ffb640 1 -- 172.21.15.7:0/252163059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f493800fde0 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f4935ffb640 1 -- 172.21.15.7:0/252163059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f493801d020 con 0x559f7da7e700 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f49577ac740 10 client.16710 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f49577ac740 10 client.16710 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f49577ac740 20 client.16710 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f49577ac740 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7da96a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f49577ac740 4 client.16710 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f7da3c3e0 con 0x559f7da94660 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f494e59c640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7da96a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f4935ffb640 10 client.16710 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.783+0000 7f494e59c640 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7da96a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b8fa259579a0e7a server_cookie=959c69464917e6e3 in_seq=0 out_seq=0 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f4935ffb640 1 -- 172.21.15.7:0/252163059 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559f7da3c3e0 con 0x559f7da94660 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f4935ffb640 10 client.16710 handle_command_reply: tid=0 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 client.16710 shutdown 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 msgr2=0x559f7da88bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da88bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4938005bc0 tx=0x7f4938023040 comp rx=0 tx=0).stop 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 shutdown_connections 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7da96a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f4935ffb640 0 client.16710 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da90170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da89130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 --2- 172.21.15.7:0/252163059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da88bf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 >> 172.21.15.7:0/252163059 conn(0x559f7da75860 msgr2=0x559f7da74ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 shutdown_connections 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 1 -- 172.21.15.7:0/252163059 wait complete. 2024-03-19T00:43:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.784+0000 7f49577ac740 20 client.16710 trim_cache size 0 max 16384 2024-03-19T00:43:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.797+0000 7f49577ac740 1 Processor -- start 2024-03-19T00:43:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.797+0000 7f49577ac740 1 -- start start 2024-03-19T00:43:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f49577ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7e700 0x559f7da74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f49577ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da90130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f49577ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da74850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f49577ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49501aa9b0 con 0x559f7da7f0f0 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f49577ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a8280 con 0x559f7da84260 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f49577ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a60d0 con 0x559f7da7e700 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da90130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da90130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44752/0 (socket says 172.21.15.7:44752) 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 -- 172.21.15.7:0/4096858151 learned_addr learned my addr 172.21.15.7:0/4096858151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f4945d9b640 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da74850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f4945d9b640 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7e700 0x559f7da74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 -- 172.21.15.7:0/4096858151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7e700 msgr2=0x559f7da74310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7e700 0x559f7da74310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 -- 172.21.15.7:0/4096858151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 msgr2=0x559f7da74850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da74850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 -- 172.21.15.7:0/4096858151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7da87cd0 con 0x559f7da7f0f0 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f494e59c640 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da90130 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f494002c170 tx=0x7f4940005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792939de9e899bfd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f4937fff640 1 -- 172.21.15.7:0/4096858151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940004280 con 0x559f7da7f0f0 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f4937fff640 1 -- 172.21.15.7:0/4096858151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4940004420 con 0x559f7da7f0f0 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.798+0000 7f4937fff640 1 -- 172.21.15.7:0/4096858151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400053d0 con 0x559f7da7f0f0 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- 172.21.15.7:0/4096858151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 msgr2=0x559f7da90130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da90130 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f494002c170 tx=0x7f4940005fd0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- 172.21.15.7:0/4096858151 shutdown_connections 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da74850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7f0f0 0x559f7da90130 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- 172.21.15.7:0/4096858151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7e700 0x559f7da74310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- 172.21.15.7:0/4096858151 >> 172.21.15.7:0/4096858151 conn(0x559f7da75860 msgr2=0x559f7da88800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- 172.21.15.7:0/4096858151 shutdown_connections 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- 172.21.15.7:0/4096858151 wait complete. 2024-03-19T00:43:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- start start 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 Processor -- start 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da8d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da8d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da8e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a6740 con 0x559f7da7e700 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a7bb0 con 0x559f7da84260 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a6e10 con 0x559f7da7f0f0 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f49577ac740 10 client.0 _ll_register_callbacks cb 0x7f49480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da8d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da8d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44764/0 (socket says 172.21.15.7:44764) 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.799+0000 7f4945d9b640 1 -- 172.21.15.7:0/1947525271 learned_addr learned my addr 172.21.15.7:0/1947525271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f494e59c640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da8d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da8e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 -- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 msgr2=0x559f7da8d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da8d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 -- 172.21.15.7:0/1947525271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 msgr2=0x559f7da8e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da8e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 -- 172.21.15.7:0/1947525271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7da8ee10 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4945d9b640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da8d2d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f493801f750 tx=0x7f4938023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dabde5086ad535 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4935ffb640 1 -- 172.21.15.7:0/1947525271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493801f900 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4935ffb640 1 -- 172.21.15.7:0/1947525271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f493801faa0 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4940004420 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4935ffb640 1 -- 172.21.15.7:0/1947525271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4938011250 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f49577ac740 10 client.16716 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f7da90110 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4935ffb640 1 -- 172.21.15.7:0/1947525271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4940004420 con 0x559f7da7e700 2024-03-19T00:43:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.800+0000 7f4935ffb640 1 -- 172.21.15.7:0/1947525271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f493801d070 con 0x559f7da7e700 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 10 client.16716 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 10 client.16716 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 20 client.16716 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7dafce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 4 client.16716 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f7d9eb9f0 con 0x559f7da94660 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f494e59c640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7dafce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f494e59c640 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7dafce10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f294da154aa3d5ae server_cookie=a79091dc6a70b84f in_seq=0 out_seq=0 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f4935ffb640 10 client.16716 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f4935ffb640 1 -- 172.21.15.7:0/1947525271 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f7d9eb9f0 con 0x559f7da94660 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f4935ffb640 10 client.16716 handle_command_reply: tid=0 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 client.16716 shutdown 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 msgr2=0x559f7da8d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da8d2d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f493801f750 tx=0x7f4938023040 comp rx=0 tx=0).stop 2024-03-19T00:43:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 shutdown_connections 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559f7da94660 0x559f7dafce10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559f7da84260 0x559f7da8e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f4935ffb640 0 client.16716 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f7da7f0f0 0x559f7da8d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 --2- 172.21.15.7:0/1947525271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f7da7e700 0x559f7da8d2d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 >> 172.21.15.7:0/1947525271 conn(0x559f7da75860 msgr2=0x559f7da78ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 shutdown_connections 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.801+0000 7f49577ac740 1 -- 172.21.15.7:0/1947525271 wait complete. 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f49577ac740 20 client.16716 trim_cache size 0 max 16384 2024-03-19T00:43:52.809 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:43:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f493c0a2800 msgr2=0x7f493c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f493c0a2800 0x7f493c0a4cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f49480a3c00 tx=0x7f49480963c0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 msgr2=0x7f495019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 0x7f495019f060 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f494800eb10 tx=0x7f494800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 shutdown_connections 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f493c0a2800 0x7f493c0a4cc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49501075a0 0x7f495019f060 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4950104c70 0x7f4950197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 --2- 172.21.15.7:0/3714553544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4950102340 0x7f4950197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 >> 172.21.15.7:0/3714553544 conn(0x7f49500fbcd0 msgr2=0x7f4950102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 shutdown_connections 2024-03-19T00:43:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.802+0000 7f4954ae8640 1 -- 172.21.15.7:0/3714553544 wait complete. 2024-03-19T00:43:52.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 -- 172.21.15.7:0/1663154904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b41022e0 msgr2=0x7f87b4108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 --2- 172.21.15.7:0/1663154904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b41022e0 0x7f87b4108b70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009ca0 tx=0x7f87ac02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 -- 172.21.15.7:0/1663154904 shutdown_connections 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 --2- 172.21.15.7:0/1663154904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b41022e0 0x7f87b4108b70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 --2- 172.21.15.7:0/1663154904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4101920 0x7f87b4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 --2- 172.21.15.7:0/1663154904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b4100730 0x7f87b4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 -- 172.21.15.7:0/1663154904 >> 172.21.15.7:0/1663154904 conn(0x7f87b40fbc80 msgr2=0x7f87b40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 -- 172.21.15.7:0/1663154904 shutdown_connections 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 -- 172.21.15.7:0/1663154904 wait complete. 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.917+0000 7f87bc15b640 1 Processor -- start 2024-03-19T00:43:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 -- start start 2024-03-19T00:43:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4100730 0x7f87b4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 0x7f87b4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b41022e0 0x7f87b419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410c240 con 0x7f87b4101920 2024-03-19T00:43:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410c0c0 con 0x7f87b4100730 2024-03-19T00:43:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87bc15b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410c3c0 con 0x7f87b41022e0 2024-03-19T00:43:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 0x7f87b4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 0x7f87b4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44780/0 (socket says 172.21.15.7:44780) 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 -- 172.21.15.7:0/3279308539 learned_addr learned my addr 172.21.15.7:0/3279308539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b41022e0 0x7f87b419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4100730 0x7f87b4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 -- 172.21.15.7:0/3279308539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b41022e0 msgr2=0x7f87b419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b41022e0 0x7f87b419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 -- 172.21.15.7:0/3279308539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4100730 msgr2=0x7f87b4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4100730 0x7f87b4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 -- 172.21.15.7:0/3279308539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b419f630 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4100730 0x7f87b4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87b96f6640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 0x7f87b4197f50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f87a400ea60 tx=0x7f87a400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87aaffd640 1 -- 172.21.15.7:0/3279308539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400cde0 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.918+0000 7f87aaffd640 1 -- 172.21.15.7:0/3279308539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87a40035f0 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.919+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b419f920 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.919+0000 7f87aaffd640 1 -- 172.21.15.7:0/3279308539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4010690 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.919+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b419fe70 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.921+0000 7f87aaffd640 1 -- 172.21.15.7:0/3279308539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f87a40037a0 con 0x7f87b4101920 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.921+0000 7f87bee26740 1 Processor -- start 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.921+0000 7f87bee26740 1 -- start start 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87bee26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d1493ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87bee26740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 0x555d1493fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87bee26740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d14940260 0x555d149426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87bee26740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555d1489a4b0 con 0x555d1493b0f0 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87bee26740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d14842d70 con 0x555d14940260 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87bee26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555d14896550 con 0x555d1493a700 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87aaffd640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87a00a2800 0x7f87a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 0x555d1493fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 0x555d1493fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44788/0 (socket says 172.21.15.7:44788) 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d1493ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 -- 172.21.15.7:0/3926417182 learned_addr learned my addr 172.21.15.7:0/3926417182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b96f6640 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d14940260 0x555d149426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87a00a2800 0x7f87a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87aaffd640 1 -- 172.21.15.7:0/3279308539 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f87a40cf050 con 0x7f87b4101920 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 -- 172.21.15.7:0/3926417182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 msgr2=0x555d1493ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d1493ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 -- 172.21.15.7:0/3926417182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d14940260 msgr2=0x555d149426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d14940260 0x555d149426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 -- 172.21.15.7:0/3926417182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d14865210 con 0x555d1493b0f0 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d1493ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b96f6640 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d14940260 0x555d149426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 0x555d1493fbf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f87b00114c0 tx=0x7f87b0011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf51bc7ec1317d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87a00a2800 0x7f87a00a4cc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f87ac006010 tx=0x7f87ac0324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.922+0000 7f879ffff640 1 -- 172.21.15.7:0/3926417182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004020 con 0x555d1493b0f0 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f879ffff640 1 -- 172.21.15.7:0/3926417182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87b00054d0 con 0x555d1493b0f0 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f879ffff640 1 -- 172.21.15.7:0/3926417182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004c10 con 0x555d1493b0f0 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 -- 172.21.15.7:0/3926417182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 msgr2=0x555d1493fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 0x555d1493fbf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f87b00114c0 tx=0x7f87b0011980 comp rx=0 tx=0).stop 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 -- 172.21.15.7:0/3926417182 shutdown_connections 2024-03-19T00:43:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d14940260 0x555d149426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d1493b0f0 0x555d1493fbf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 --2- 172.21.15.7:0/3926417182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d1493ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 -- 172.21.15.7:0/3926417182 >> 172.21.15.7:0/3926417182 conn(0x555d14931860 msgr2=0x555d14931c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 -- 172.21.15.7:0/3926417182 shutdown_connections 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.923+0000 7f87bee26740 1 -- 172.21.15.7:0/3926417182 wait complete. 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 -- start start 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 Processor -- start 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d14944b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493b0f0 0x555d149450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555d14943210 con 0x555d14940260 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d14856310 con 0x555d1493b0f0 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555d14943390 con 0x555d1493a700 2024-03-19T00:43:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87bee26740 10 client.0 _ll_register_callbacks cb 0x555d1484a2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44804/0 (socket says 172.21.15.7:44804) 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1264025195 learned_addr learned my addr 172.21.15.7:0/1264025195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d14944b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87b96f6640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493b0f0 0x555d149450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1264025195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 msgr2=0x555d14944b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.924+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d14944b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493b0f0 msgr2=0x555d149450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493b0f0 0x555d149450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1264025195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d145ae3f0 con 0x555d14940260 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494c110 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009a60 tx=0x7f87ac0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2e82496637ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f879d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f879d7fa640 1 -- 172.21.15.7:0/1264025195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac053060 con 0x555d14940260 2024-03-19T00:43:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b00054d0 con 0x555d14940260 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f879d7fa640 1 -- 172.21.15.7:0/1264025195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f87ac004490 con 0x555d14940260 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f879d7fa640 1 -- 172.21.15.7:0/1264025195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87ac039040 con 0x555d14940260 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 10 client.16728 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d146787d0 con 0x555d14940260 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f879d7fa640 1 -- 172.21.15.7:0/1264025195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87b00054d0 con 0x555d14940260 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f879d7fa640 1 -- 172.21.15.7:0/1264025195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f87ac04e700 con 0x555d14940260 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 10 client.16728 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 10 client.16728 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 20 client.16728 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d14950a90 0x555d14952ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 4 client.16728 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.925+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d148f83e0 con 0x555d14950a90 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.926+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d14950a90 0x555d14952ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.926+0000 7f879d7fa640 10 client.16728 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.926+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d14950a90 0x555d14952ed0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdc29afbef9290ae server_cookie=82aa0e844a980fba in_seq=0 out_seq=0 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f879d7fa640 1 -- 172.21.15.7:0/1264025195 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555d148f83e0 con 0x555d14950a90 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f879d7fa640 10 client.16728 handle_command_reply: tid=0 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 client.16728 shutdown 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 msgr2=0x555d1494c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494c110 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009a60 tx=0x7f87ac0095c0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 shutdown_connections 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d14950a90 0x555d14952ed0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494c110 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493b0f0 0x555d149450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 --2- 172.21.15.7:0/1264025195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493a700 0x555d14944b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f879d7fa640 0 client.16728 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 >> 172.21.15.7:0/1264025195 conn(0x555d14931860 msgr2=0x555d14930ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 shutdown_connections 2024-03-19T00:43:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 1 -- 172.21.15.7:0/1264025195 wait complete. 2024-03-19T00:43:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.927+0000 7f87bee26740 20 client.16728 trim_cache size 0 max 16384 2024-03-19T00:43:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 Processor -- start 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 -- start start 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14944590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d14930310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41aa980 con 0x555d14940260 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a8250 con 0x555d1493a700 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87bee26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a60a0 con 0x555d1493b0f0 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d14930310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d14930310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44808/0 (socket says 172.21.15.7:44808) 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 -- 172.21.15.7:0/899701501 learned_addr learned my addr 172.21.15.7:0/899701501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14944590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87b96f6640 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 -- 172.21.15.7:0/899701501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 msgr2=0x555d1494c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 -- 172.21.15.7:0/899701501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 msgr2=0x555d14944590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14944590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 -- 172.21.15.7:0/899701501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d14931400 con 0x555d14940260 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d14930310 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009430 tx=0x7f87ac02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba2e9ebbace81f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f879f7fe640 1 -- 172.21.15.7:0/899701501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac053070 con 0x555d14940260 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f879f7fe640 1 -- 172.21.15.7:0/899701501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87ac004410 con 0x555d14940260 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.941+0000 7f879f7fe640 1 -- 172.21.15.7:0/899701501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac04e6b0 con 0x555d14940260 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 -- 172.21.15.7:0/899701501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 msgr2=0x555d14930310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d14930310 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009430 tx=0x7f87ac02c890 comp rx=0 tx=0).stop 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 -- 172.21.15.7:0/899701501 shutdown_connections 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d14930310 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 --2- 172.21.15.7:0/899701501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14944590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 -- 172.21.15.7:0/899701501 >> 172.21.15.7:0/899701501 conn(0x555d14931860 msgr2=0x555d14943e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 -- 172.21.15.7:0/899701501 shutdown_connections 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 -- 172.21.15.7:0/899701501 wait complete. 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 -- start start 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.942+0000 7f87bee26740 1 Processor -- start 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14949d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a6710 con 0x555d14940260 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a7b80 con 0x555d1493a700 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a6de0 con 0x555d1493b0f0 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 10 client.0 _ll_register_callbacks cb 0x7f87b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44818/0 (socket says 172.21.15.7:44818) 2024-03-19T00:43:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14949d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87b96f6640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1397120149 learned_addr learned my addr 172.21.15.7:0/1397120149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1397120149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 msgr2=0x555d1494a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 msgr2=0x555d14949d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14949d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 -- 172.21.15.7:0/1397120149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d14907880 con 0x555d14940260 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14949d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87ba6f8640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494b280 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009a60 tx=0x7f87ac02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3de7a2b570d399 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f879d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f879d7fa640 1 -- 172.21.15.7:0/1397120149 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac062020 con 0x555d14940260 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f879d7fa640 1 -- 172.21.15.7:0/1397120149 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f87ac004b40 con 0x555d14940260 2024-03-19T00:43:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.943+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac004410 con 0x555d14940260 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f879d7fa640 1 -- 172.21.15.7:0/1397120149 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87ac053040 con 0x555d14940260 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 10 client.16734 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d149b7120 con 0x555d14940260 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f879d7fa640 1 -- 172.21.15.7:0/1397120149 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87ac004410 con 0x555d14940260 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f879d7fa640 1 -- 172.21.15.7:0/1397120149 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f87ac05ec30 con 0x555d14940260 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 10 client.16734 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 10 client.16734 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 20 client.16734 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d149b78d0 0x555d149b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 4 client.16734 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d148a79f0 con 0x555d149b78d0 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d149b78d0 0x555d149b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f879d7fa640 10 client.16734 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.944+0000 7f87b9ef7640 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d149b78d0 0x555d149b9cf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64ad6a79d7a0b06 server_cookie=b64ad56f7cdc88e8 in_seq=0 out_seq=0 2024-03-19T00:43:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f879d7fa640 1 -- 172.21.15.7:0/1397120149 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d148a79f0 con 0x555d149b78d0 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f879d7fa640 10 client.16734 handle_command_reply: tid=0 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 client.16734 shutdown 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 msgr2=0x555d1494b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494b280 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009a60 tx=0x7f87ac02c900 comp rx=0 tx=0).stop 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 shutdown_connections 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x555d149b78d0 0x555d149b9cf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d14940260 0x555d1494b280 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1493b0f0 0x555d1494a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 --2- 172.21.15.7:0/1397120149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d1493a700 0x555d14949d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f879d7fa640 0 client.16734 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 >> 172.21.15.7:0/1397120149 conn(0x555d14931860 msgr2=0x555d14934ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 shutdown_connections 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 1 -- 172.21.15.7:0/1397120149 wait complete. 2024-03-19T00:43:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bee26740 20 client.16734 trim_cache size 0 max 16384 2024-03-19T00:43:52.953 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:43:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87a00a2800 msgr2=0x7f87a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bc15b640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87a00a2800 0x7f87a00a4cc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f87ac006010 tx=0x7f87ac0324f0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 msgr2=0x7f87b4197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.945+0000 7f87bc15b640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 0x7f87b4197f50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f87a400ea60 tx=0x7f87a400ef20 comp rx=0 tx=0).stop 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 shutdown_connections 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f87a00a2800 0x7f87a00a4cc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b41022e0 0x7f87b419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b4101920 0x7f87b4197f50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 --2- 172.21.15.7:0/3279308539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b4100730 0x7f87b4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 >> 172.21.15.7:0/3279308539 conn(0x7f87b40fbc80 msgr2=0x7f87b40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 shutdown_connections 2024-03-19T00:43:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:52.946+0000 7f87bc15b640 1 -- 172.21.15.7:0/3279308539 wait complete. 2024-03-19T00:43:52.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:43:53.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3789866572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100730 msgr2=0x7ff314100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3789866572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100730 0x7ff314100b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009ca0 tx=0x7ff30c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3789866572 shutdown_connections 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3789866572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3141022e0 0x7ff314108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3789866572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314101920 0x7ff314101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3789866572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100730 0x7ff314100b50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3789866572 >> 172.21.15.7:0/3789866572 conn(0x7ff3140fbc80 msgr2=0x7ff3140fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3789866572 shutdown_connections 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.061+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3789866572 wait complete. 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 Processor -- start 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 -- start start 2024-03-19T00:43:53.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314100730 0x7ff314196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 0x7ff314196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3141022e0 0x7ff31419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff31410c230 con 0x7ff314101920 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff31410c0b0 con 0x7ff3141022e0 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff31ab3b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff31410c3b0 con 0x7ff314100730 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 0x7ff314196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 0x7ff314196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44838/0 (socket says 172.21.15.7:44838) 2024-03-19T00:43:53.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff313fff640 1 -- 172.21.15.7:0/3652945822 learned_addr learned my addr 172.21.15.7:0/3652945822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314100730 0x7ff314196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.062+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3141022e0 0x7ff31419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff313fff640 1 -- 172.21.15.7:0/3652945822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314100730 msgr2=0x7ff314196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff313fff640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314100730 0x7ff314196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff313fff640 1 -- 172.21.15.7:0/3652945822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3141022e0 msgr2=0x7ff31419df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff313fff640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3141022e0 0x7ff31419df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff313fff640 1 -- 172.21.15.7:0/3652945822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31419e590 con 0x7ff314101920 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff313fff640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 0x7ff314196eb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff30800d950 tx=0x7ff30800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff311ffb640 1 -- 172.21.15.7:0/3652945822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308017030 con 0x7ff314101920 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff311ffb640 1 -- 172.21.15.7:0/3652945822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff308004020 con 0x7ff314101920 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31419e880 con 0x7ff314101920 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff311ffb640 1 -- 172.21.15.7:0/3652945822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080047a0 con 0x7ff314101920 2024-03-19T00:43:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.063+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31419ee20 con 0x7ff314101920 2024-03-19T00:43:53.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.065+0000 7ff311ffb640 1 -- 172.21.15.7:0/3652945822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7ff3080041d0 con 0x7ff314101920 2024-03-19T00:43:53.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.065+0000 7ff31d803740 1 Processor -- start 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 -- start start 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb74920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562d1cad64b0 con 0x562d1cb7a060 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d1ca7ed70 con 0x562d1cb74500 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff31d803740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d1cad2550 con 0x562d1cb74ef0 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff311ffb640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2fc0a2880 0x7ff2fc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb7c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44842/0 (socket says 172.21.15.7:44842) 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 -- 172.21.15.7:0/3776012354 learned_addr learned my addr 172.21.15.7:0/3776012354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb74920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2fc0a2880 0x7ff2fc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff311ffb640 1 -- 172.21.15.7:0/3652945822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff3080ce7d0 con 0x7ff314101920 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 -- 172.21.15.7:0/3776012354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 msgr2=0x562d1cb799f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb799f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 -- 172.21.15.7:0/3776012354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 msgr2=0x562d1cb74920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb74920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff313fff640 1 -- 172.21.15.7:0/3776012354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1caa1210 con 0x562d1cb7a060 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.066+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb74920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff313fff640 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb7c4f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff30809af70 tx=0x7ff30800ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0421adf66df3728 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2fc0a2880 0x7ff2fc0a4d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x562d1cb75750 tx=0x7ff304005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff2faffd640 1 -- 172.21.15.7:0/3776012354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30809b4b0 con 0x562d1cb7a060 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff2faffd640 1 -- 172.21.15.7:0/3776012354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff30809ef00 con 0x562d1cb7a060 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff2faffd640 1 -- 172.21.15.7:0/3776012354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080cee20 con 0x562d1cb7a060 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 -- 172.21.15.7:0/3776012354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 msgr2=0x562d1cb7c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb7c4f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff30809af70 tx=0x7ff30800ff50 comp rx=0 tx=0).stop 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 -- 172.21.15.7:0/3776012354 shutdown_connections 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb7c4f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 --2- 172.21.15.7:0/3776012354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb74920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 -- 172.21.15.7:0/3776012354 >> 172.21.15.7:0/3776012354 conn(0x562d1cb6d860 msgr2=0x562d1cb6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 -- 172.21.15.7:0/3776012354 shutdown_connections 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.067+0000 7ff31d803740 1 -- 172.21.15.7:0/3776012354 wait complete. 2024-03-19T00:43:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 -- start start 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 Processor -- start 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb7e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb7eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562d1cb7cfb0 con 0x562d1cb7a060 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d1ca92310 con 0x562d1cb74500 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d1cb7d130 con 0x562d1cb74ef0 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff31d803740 10 client.0 _ll_register_callbacks cb 0x562d1ca862d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff3190d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff3190d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb85f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44850/0 (socket says 172.21.15.7:44850) 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff3190d8640 1 -- 172.21.15.7:0/3216066257 learned_addr learned my addr 172.21.15.7:0/3216066257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff313fff640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb7eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.068+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb7e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3190d8640 1 -- 172.21.15.7:0/3216066257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 msgr2=0x562d1cb7eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb7eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3190d8640 1 -- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 msgr2=0x562d1cb7e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb7e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3190d8640 1 -- 172.21.15.7:0/3216066257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1c7ea3f0 con 0x562d1cb7a060 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb7e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff3190d8640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb85f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff304019100 tx=0x7ff3040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4b58c4f235dc30 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff2f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff2f3fff640 1 -- 172.21.15.7:0/3216066257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400fd00 con 0x562d1cb7a060 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff2f3fff640 1 -- 172.21.15.7:0/3216066257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff304005020 con 0x562d1cb7a060 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30809ef00 con 0x562d1cb7a060 2024-03-19T00:43:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff2f3fff640 1 -- 172.21.15.7:0/3216066257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff304032050 con 0x562d1cb7a060 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff31d803740 10 client.16746 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d1c8b47d0 con 0x562d1cb7a060 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff2f3fff640 1 -- 172.21.15.7:0/3216066257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff30809ef00 con 0x562d1cb7a060 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff2f3fff640 1 -- 172.21.15.7:0/3216066257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff304004b60 con 0x562d1cb7a060 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff31d803740 10 client.16746 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff31d803740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.069+0000 7ff31d803740 20 client.16746 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.070+0000 7ff31d803740 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cb8a7d0 0x562d1cb8cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.070+0000 7ff31d803740 4 client.16746 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.070+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1cb343e0 con 0x562d1cb8a7d0 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.070+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cb8a7d0 0x562d1cb8cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.070+0000 7ff2f3fff640 10 client.16746 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.070+0000 7ff3188d7640 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cb8a7d0 0x562d1cb8cc90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db8e494b46ab236 server_cookie=4c47931bacf8ed4a in_seq=0 out_seq=0 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff2f3fff640 1 -- 172.21.15.7:0/3216066257 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562d1cb343e0 con 0x562d1cb8a7d0 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff2f3fff640 10 client.16746 handle_command_reply: tid=0 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 client.16746 shutdown 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 msgr2=0x562d1cb85f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb85f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff304019100 tx=0x7ff3040046c0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 shutdown_connections 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cb8a7d0 0x562d1cb8cc90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb85f30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb7eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff2f3fff640 0 client.16746 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:53.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 --2- 172.21.15.7:0/3216066257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb7e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 >> 172.21.15.7:0/3216066257 conn(0x562d1cb6d860 msgr2=0x562d1cb6cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:53.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 shutdown_connections 2024-03-19T00:43:53.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 1 -- 172.21.15.7:0/3216066257 wait complete. 2024-03-19T00:43:53.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.071+0000 7ff31d803740 20 client.16746 trim_cache size 0 max 16384 2024-03-19T00:43:53.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 Processor -- start 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 -- start start 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb6c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb6c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141aaa10 con 0x562d1cb7a060 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141a82e0 con 0x562d1cb74500 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff31d803740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141a6130 con 0x562d1cb74ef0 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb6c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb6c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44852/0 (socket says 172.21.15.7:44852) 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 -- 172.21.15.7:0/4267006425 learned_addr learned my addr 172.21.15.7:0/4267006425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:53.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3188d7640 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb6c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff313fff640 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 -- 172.21.15.7:0/4267006425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 msgr2=0x562d1cb85ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb85ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 -- 172.21.15.7:0/4267006425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 msgr2=0x562d1cb6c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb6c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 -- 172.21.15.7:0/4267006425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1cb752d0 con 0x562d1cb7a060 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff313fff640 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb85ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.085+0000 7ff3190d8640 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb6c850 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff304000c00 tx=0x7ff3040045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ddd7fcb5a847e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff2fa7fc640 1 -- 172.21.15.7:0/4267006425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30402e070 con 0x562d1cb7a060 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff2fa7fc640 1 -- 172.21.15.7:0/4267006425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff304004c50 con 0x562d1cb7a060 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff2fa7fc640 1 -- 172.21.15.7:0/4267006425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304005400 con 0x562d1cb7a060 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 -- 172.21.15.7:0/4267006425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 msgr2=0x562d1cb6c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb6c850 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff304000c00 tx=0x7ff3040045b0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 -- 172.21.15.7:0/4267006425 shutdown_connections 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb7a060 0x562d1cb6c850 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb74ef0 0x562d1cb85ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 --2- 172.21.15.7:0/4267006425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74500 0x562d1cb6c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 -- 172.21.15.7:0/4267006425 >> 172.21.15.7:0/4267006425 conn(0x562d1cb6d860 msgr2=0x562d1cb6dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 -- 172.21.15.7:0/4267006425 shutdown_connections 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 -- 172.21.15.7:0/4267006425 wait complete. 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.086+0000 7ff31d803740 1 -- start start 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 Processor -- start 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 0x562d1cb833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74ef0 0x562d1cb83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb7a060 0x562d1cb84920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141a67a0 con 0x562d1cb74500 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141a7c10 con 0x562d1cb74ef0 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141a6e70 con 0x562d1cb7a060 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff31d803740 10 client.0 _ll_register_callbacks cb 0x7ff3040115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 0x562d1cb833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 0x562d1cb833d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44866/0 (socket says 172.21.15.7:44866) 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 -- 172.21.15.7:0/1128630029 learned_addr learned my addr 172.21.15.7:0/1128630029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff313fff640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74ef0 0x562d1cb83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3190d8640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb7a060 0x562d1cb84920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 -- 172.21.15.7:0/1128630029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb7a060 msgr2=0x562d1cb84920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb7a060 0x562d1cb84920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 -- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74ef0 msgr2=0x562d1cb83910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74ef0 0x562d1cb83910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 -- 172.21.15.7:0/1128630029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1cb84f10 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff3188d7640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 0x562d1cb833d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02e010 tx=0x7ff30c02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7477303aa845fe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.087+0000 7ff2f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff2f3fff640 1 -- 172.21.15.7:0/1128630029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c039040 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff2f3fff640 1 -- 172.21.15.7:0/1128630029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff30c0327f0 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff304004c50 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff2f3fff640 1 -- 172.21.15.7:0/1128630029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff30c032e60 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 10 client.16752 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d1cbf0b90 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff2f3fff640 1 -- 172.21.15.7:0/1128630029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff304004c50 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff2f3fff640 1 -- 172.21.15.7:0/1128630029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff30c004610 con 0x562d1cb74500 2024-03-19T00:43:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 10 client.16752 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 20 client.16752 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cbf1400 0x562d1cbf3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 4 client.16752 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d1cae39f0 con 0x562d1cbf1400 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.088+0000 7ff313fff640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cbf1400 0x562d1cbf3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff2f3fff640 10 client.16752 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff313fff640 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cbf1400 0x562d1cbf3820 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248a9639faefc38c server_cookie=b33d29a89c020940 in_seq=0 out_seq=0 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff2f3fff640 1 -- 172.21.15.7:0/1128630029 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d1cae39f0 con 0x562d1cbf1400 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff2f3fff640 10 client.16752 handle_command_reply: tid=0 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 client.16752 shutdown 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 msgr2=0x562d1cb833d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 0x562d1cb833d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02e010 tx=0x7ff30c02c8a0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 shutdown_connections 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d1cb7a060 0x562d1cb84920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d1cb74ef0 0x562d1cb83910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562d1cbf1400 0x562d1cbf3820 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 --2- 172.21.15.7:0/1128630029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d1cb74500 0x562d1cb833d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff2f3fff640 0 client.16752 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 >> 172.21.15.7:0/1128630029 conn(0x562d1cb6d860 msgr2=0x562d1cb70ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 shutdown_connections 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 1 -- 172.21.15.7:0/1128630029 wait complete. 2024-03-19T00:43:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.089+0000 7ff31d803740 20 client.16752 trim_cache size 0 max 16384 2024-03-19T00:43:53.099 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:43:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2fc0a2880 msgr2=0x7ff2fc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2fc0a2880 0x7ff2fc0a4d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x562d1cb75750 tx=0x7ff304005f90 comp rx=0 tx=0).stop 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 msgr2=0x7ff314196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 0x7ff314196eb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff30800d950 tx=0x7ff30800de10 comp rx=0 tx=0).stop 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 shutdown_connections 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2fc0a2880 0x7ff2fc0a4d40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3141022e0 0x7ff31419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101920 0x7ff314196eb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 --2- 172.21.15.7:0/3652945822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff314100730 0x7ff314196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 >> 172.21.15.7:0/3652945822 conn(0x7ff3140fbc80 msgr2=0x7ff314104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 shutdown_connections 2024-03-19T00:43:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:53.090+0000 7ff31ab3b640 1 -- 172.21.15.7:0/3652945822 wait complete. 2024-03-19T00:43:53.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:52 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2049493055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:43:53.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:52 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2049493055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:43:53.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:52 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2049493055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:43:54.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:43:54.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:43:54.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:43:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 -- 172.21.15.7:0/3278594490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed141022f0 msgr2=0x7fed14108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3278594490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed141022f0 0x7fed14108b80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009ca0 tx=0x7fed0c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 -- 172.21.15.7:0/3278594490 shutdown_connections 2024-03-19T00:43:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3278594490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed141022f0 0x7fed14108b80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3278594490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed14101930 0x7fed14101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3278594490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed14100740 0x7fed14100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 -- 172.21.15.7:0/3278594490 >> 172.21.15.7:0/3278594490 conn(0x7fed140fbcb0 msgr2=0x7fed140fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 -- 172.21.15.7:0/3278594490 shutdown_connections 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 -- 172.21.15.7:0/3278594490 wait complete. 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 Processor -- start 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.219+0000 7fed18ce8640 1 -- start start 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed18ce8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed14100740 0x7fed14069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed18ce8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 0x7fed1419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed18ce8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed141022f0 0x7fed1419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed18ce8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410c270 con 0x7fed14101930 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed18ce8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410c0f0 con 0x7fed141022f0 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed18ce8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410c3f0 con 0x7fed14100740 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 0x7fed1419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed1259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed14100740 0x7fed14069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 0x7fed1419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44884/0 (socket says 172.21.15.7:44884) 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 -- 172.21.15.7:0/3519410065 learned_addr learned my addr 172.21.15.7:0/3519410065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed12d9d640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed141022f0 0x7fed1419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 -- 172.21.15.7:0/3519410065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed14100740 msgr2=0x7fed14069660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed14100740 0x7fed14069660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 -- 172.21.15.7:0/3519410065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed141022f0 msgr2=0x7fed1419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed141022f0 0x7fed1419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 -- 172.21.15.7:0/3519410065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1419f6c0 con 0x7fed14101930 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0bfff640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 0x7fed1419c790 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00e9f0 tx=0x7fecfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.220+0000 7fed0b7fe640 1 -- 172.21.15.7:0/3519410065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00cd80 con 0x7fed14101930 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.221+0000 7fed0b7fe640 1 -- 172.21.15.7:0/3519410065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecfc0054d0 con 0x7fed14101930 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.221+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1419f950 con 0x7fed14101930 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.221+0000 7fed0b7fe640 1 -- 172.21.15.7:0/3519410065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc010490 con 0x7fed14101930 2024-03-19T00:43:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.221+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1419fe50 con 0x7fed14101930 2024-03-19T00:43:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.223+0000 7fed0b7fe640 1 -- 172.21.15.7:0/3519410065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fecfc005020 con 0x7fed14101930 2024-03-19T00:43:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.223+0000 7fed1b9ae740 1 Processor -- start 2024-03-19T00:43:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.223+0000 7fed1b9ae740 1 -- start start 2024-03-19T00:43:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de218d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218dd90 0x557de2192890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de2195390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557de20f2160 con 0x557de2192f00 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557de209ae10 con 0x557de218dd90 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557de20ee200 con 0x557de218d3a0 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed0b7fe640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed000a2800 0x7fed000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de218d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de2195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de218d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:36466/0 (socket says 172.21.15.7:36466) 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 -- 172.21.15.7:0/2787173877 learned_addr learned my addr 172.21.15.7:0/2787173877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed1259c640 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218dd90 0x557de2192890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed000a2800 0x7fed000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 -- 172.21.15.7:0/2787173877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218dd90 msgr2=0x557de2192890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218dd90 0x557de2192890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 -- 172.21.15.7:0/2787173877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 msgr2=0x557de2195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed0b7fe640 1 -- 172.21.15.7:0/3519410065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fecfc0cf1a0 con 0x7fed14101930 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de2195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.224+0000 7fed12d9d640 1 -- 172.21.15.7:0/2787173877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de1ece7e0 con 0x557de218d3a0 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed12d9d640 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de218d7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed0c032380 tx=0x7fed0c032ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874585382d46807c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed12d9d640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed000a2800 0x7fed000a4cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed0c047000 tx=0x7fed0c048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed08ff9640 1 -- 172.21.15.7:0/2787173877 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c050070 con 0x557de218d3a0 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed08ff9640 1 -- 172.21.15.7:0/2787173877 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed0c032810 con 0x557de218d3a0 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed08ff9640 1 -- 172.21.15.7:0/2787173877 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c039040 con 0x557de218d3a0 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2787173877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 msgr2=0x557de218d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de218d7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed0c032380 tx=0x7fed0c032ef0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2787173877 shutdown_connections 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de2195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218dd90 0x557de2192890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2787173877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de218d7c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.225+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2787173877 >> 172.21.15.7:0/2787173877 conn(0x557de2188910 msgr2=0x557de2188d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2787173877 shutdown_connections 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2787173877 wait complete. 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 -- start start 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 Processor -- start 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218d3a0 0x557de2197810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de2197d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de219ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557de2195e50 con 0x557de2192f00 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557de20ae1a0 con 0x557de218d3a0 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557de2195fd0 con 0x557de218dd90 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1b9ae740 10 client.0 _ll_register_callbacks cb 0x557de1e05c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed12d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de219ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de2197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218d3a0 0x557de2197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de2197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:36474/0 (socket says 172.21.15.7:36474) 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.226+0000 7fed0bfff640 1 -- 172.21.15.7:0/1788755670 learned_addr learned my addr 172.21.15.7:0/1788755670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed12d9d640 1 -- 172.21.15.7:0/1788755670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 msgr2=0x557de2197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed12d9d640 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de2197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed12d9d640 1 -- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218d3a0 msgr2=0x557de2197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed12d9d640 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218d3a0 0x557de2197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed12d9d640 1 -- 172.21.15.7:0/1788755670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de1f2ef70 con 0x557de2192f00 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed1259c640 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218d3a0 0x557de2197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed12d9d640 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de219ed90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fed0c032420 tx=0x7fed0c02c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b64528b413fa9c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fecf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fecf9ffb640 1 -- 172.21.15.7:0/1788755670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c057020 con 0x557de2192f00 2024-03-19T00:43:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fecf9ffb640 1 -- 172.21.15.7:0/1788755670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fed0c004250 con 0x557de2192f00 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed0c032810 con 0x557de2192f00 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fecf9ffb640 1 -- 172.21.15.7:0/1788755670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed0c062050 con 0x557de2192f00 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed1b9ae740 10 client.16764 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557de1dc6170 con 0x557de2192f00 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.227+0000 7fecf9ffb640 1 -- 172.21.15.7:0/1788755670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed0c032810 con 0x557de2192f00 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fecf9ffb640 1 -- 172.21.15.7:0/1788755670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fed0c03f070 con 0x557de2192f00 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1b9ae740 10 client.16764 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1b9ae740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1b9ae740 20 client.16764 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de21a32d0 0x557de21a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1b9ae740 4 client.16764 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de20ff6a0 con 0x557de21a32d0 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1259c640 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de21a32d0 0x557de21a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fed1259c640 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de21a32d0 0x557de21a5710 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4259ffc03be47828 server_cookie=4896cbf425f62196 in_seq=0 out_seq=0 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.228+0000 7fecf9ffb640 10 client.16764 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fecf9ffb640 1 -- 172.21.15.7:0/1788755670 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557de20ff6a0 con 0x557de21a32d0 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fecf9ffb640 10 client.16764 handle_command_reply: tid=0 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 client.16764 shutdown 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 msgr2=0x557de219ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de219ed90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fed0c032420 tx=0x7fed0c02c7f0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 shutdown_connections 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de21a32d0 0x557de21a5710 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de2192f00 0x557de219ed90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fecf9ffb640 0 client.16764 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de2197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/1788755670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de218d3a0 0x557de2197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 >> 172.21.15.7:0/1788755670 conn(0x557de2188910 msgr2=0x557de2187b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 shutdown_connections 2024-03-19T00:43:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 1 -- 172.21.15.7:0/1788755670 wait complete. 2024-03-19T00:43:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.229+0000 7fed1b9ae740 20 client.16764 trim_cache size 0 max 16384 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 Processor -- start 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 -- start start 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 0x557de218c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de219ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de21a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed141aca40 con 0x557de218d3a0 2024-03-19T00:43:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed141aa310 con 0x557de2192f00 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed141a8160 con 0x557de218dd90 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 0x557de218c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 0x557de218c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44910/0 (socket says 172.21.15.7:44910) 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed12d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de21a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 -- 172.21.15.7:0/3252686963 learned_addr learned my addr 172.21.15.7:0/3252686963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed0bfff640 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de219ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 -- 172.21.15.7:0/3252686963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 msgr2=0x557de219ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de219ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 -- 172.21.15.7:0/3252686963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 msgr2=0x557de21a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de21a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 -- 172.21.15.7:0/3252686963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de218e5f0 con 0x557de218d3a0 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fed1259c640 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 0x557de218c280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x557de218e190 tx=0x7fed04005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2e411beeb3fdfb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.247+0000 7fecfbfff640 1 -- 172.21.15.7:0/3252686963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04022170 con 0x557de218d3a0 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fecfbfff640 1 -- 172.21.15.7:0/3252686963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed040227c0 con 0x557de218d3a0 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fecfbfff640 1 -- 172.21.15.7:0/3252686963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04010430 con 0x557de218d3a0 2024-03-19T00:43:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 -- 172.21.15.7:0/3252686963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 msgr2=0x557de218c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 0x557de218c280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x557de218e190 tx=0x7fed04005f90 comp rx=0 tx=0).stop 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 -- 172.21.15.7:0/3252686963 shutdown_connections 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de21a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218dd90 0x557de219ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/3252686963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218d3a0 0x557de218c280 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 -- 172.21.15.7:0/3252686963 >> 172.21.15.7:0/3252686963 conn(0x557de2188910 msgr2=0x557de218bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 -- 172.21.15.7:0/3252686963 shutdown_connections 2024-03-19T00:43:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 -- 172.21.15.7:0/3252686963 wait complete. 2024-03-19T00:43:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 -- start start 2024-03-19T00:43:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.248+0000 7fed1b9ae740 1 Processor -- start 2024-03-19T00:43:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de219e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 0x557de219cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de219d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed141a87d0 con 0x557de218dd90 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed141a9c40 con 0x557de2192f00 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed141a8ea0 con 0x557de218d3a0 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 10 client.0 _ll_register_callbacks cb 0x7fed0c04e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 0x557de219cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 0x557de219cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44916/0 (socket says 172.21.15.7:44916) 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 -- 172.21.15.7:0/2980625426 learned_addr learned my addr 172.21.15.7:0/2980625426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed12d9d640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de219d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1259c640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de219e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 -- 172.21.15.7:0/2980625426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 msgr2=0x557de219e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de219e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 -- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 msgr2=0x557de219d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de219d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 -- 172.21.15.7:0/2980625426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de219da40 con 0x557de218dd90 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed0bfff640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 0x557de219cfc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fecfc09b480 tx=0x7fecfc09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e5f2698984ca9b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fecf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fecf9ffb640 1 -- 172.21.15.7:0/2980625426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0b3020 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fecf9ffb640 1 -- 172.21.15.7:0/2980625426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fecfc0046c0 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.249+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed040227c0 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fecf9ffb640 1 -- 172.21.15.7:0/2980625426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecfc0a5030 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 10 client.16770 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557de220a3c0 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fecf9ffb640 1 -- 172.21.15.7:0/2980625426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed040227c0 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fecf9ffb640 1 -- 172.21.15.7:0/2980625426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fecfc097030 con 0x557de218dd90 2024-03-19T00:43:54.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 10 client.16770 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 20 client.16770 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de220adc0 0x557de220d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 4 client.16770 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557de20f99f0 con 0x557de220adc0 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.250+0000 7fed1259c640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de220adc0 0x557de220d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.251+0000 7fecf9ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.251+0000 7fed1259c640 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de220adc0 0x557de220d1e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284231bf0297cace server_cookie=dc8f6e1526ce4572 in_seq=0 out_seq=0 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fecf9ffb640 1 -- 172.21.15.7:0/2980625426 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557de20f99f0 con 0x557de220adc0 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fecf9ffb640 10 client.16770 handle_command_reply: tid=0 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 client.16770 shutdown 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 msgr2=0x557de219cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 0x557de219cfc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fecfc09b480 tx=0x7fecfc09f040 comp rx=0 tx=0).stop 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 shutdown_connections 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de2192f00 0x557de219d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de220adc0 0x557de220d1e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de218dd90 0x557de219cfc0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 --2- 172.21.15.7:0/2980625426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de218d3a0 0x557de219e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 >> 172.21.15.7:0/2980625426 conn(0x557de2188910 msgr2=0x557de218c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fecf9ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 shutdown_connections 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 1 -- 172.21.15.7:0/2980625426 wait complete. 2024-03-19T00:43:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.252+0000 7fed1b9ae740 20 client.16770 trim_cache size 0 max 16384 2024-03-19T00:43:54.266 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:43:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:43:54.267 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f618a999-ce9e-474c-b854-0c5865303ef3", 2024-03-19T00:43:54.267 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:43:54.267 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:43:54.267 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:53 smithi204 ceph-mon[66572]: pgmap v529: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed000a2800 msgr2=0x7fed000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed000a2800 0x7fed000a4cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed0c047000 tx=0x7fed0c048000 comp rx=0 tx=0).stop 2024-03-19T00:43:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 msgr2=0x7fed1419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 0x7fed1419c790 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00e9f0 tx=0x7fecfc00edf0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 shutdown_connections 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fed000a2800 0x7fed000a4cc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed141022f0 0x7fed1419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14101930 0x7fed1419c790 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 --2- 172.21.15.7:0/3519410065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed14100740 0x7fed14069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 >> 172.21.15.7:0/3519410065 conn(0x7fed140fbcb0 msgr2=0x7fed14104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 shutdown_connections 2024-03-19T00:43:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.253+0000 7fed18ce8640 1 -- 172.21.15.7:0/3519410065 wait complete. 2024-03-19T00:43:54.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:53 smithi007 ceph-mon[62854]: pgmap v529: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:54.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:43:54.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:43:54.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:53 smithi044 ceph-mon[66366]: pgmap v529: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:43:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 -- 172.21.15.7:0/921289357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 msgr2=0x7f86fc101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 --2- 172.21.15.7:0/921289357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc101070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec02bc30 comp rx=0 tx=0).stop 2024-03-19T00:43:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 -- 172.21.15.7:0/921289357 shutdown_connections 2024-03-19T00:43:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 --2- 172.21.15.7:0/921289357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc102320 0x7f86fc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 --2- 172.21.15.7:0/921289357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86fc101960 0x7f86fc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 --2- 172.21.15.7:0/921289357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc101070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 -- 172.21.15.7:0/921289357 >> 172.21.15.7:0/921289357 conn(0x7f86fc0fb440 msgr2=0x7f86fc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 -- 172.21.15.7:0/921289357 shutdown_connections 2024-03-19T00:43:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.375+0000 7f8700fa9640 1 -- 172.21.15.7:0/921289357 wait complete. 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 Processor -- start 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 -- start start 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 0x7f86fc19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86fc102320 0x7f86fc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10c240 con 0x7f86fc101960 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10c0c0 con 0x7f86fc100c50 2024-03-19T00:43:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f8700fa9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10c3c0 con 0x7f86fc102320 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f86f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 0x7f86fc19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f86f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 0x7f86fc19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44938/0 (socket says 172.21.15.7:44938) 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f86f9d9b640 1 -- 172.21.15.7:0/4122372645 learned_addr learned my addr 172.21.15.7:0/4122372645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.376+0000 7f86fad9d640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86fc102320 0x7f86fc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86f9d9b640 1 -- 172.21.15.7:0/4122372645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86fc102320 msgr2=0x7f86fc19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86fc102320 0x7f86fc19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86f9d9b640 1 -- 172.21.15.7:0/4122372645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 msgr2=0x7f86fc100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86f9d9b640 1 -- 172.21.15.7:0/4122372645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc19f750 con 0x7f86fc101960 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86fa59c640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 0x7f86fc19c820 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f86f000e9f0 tx=0x7f86f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86eb7fe640 1 -- 172.21.15.7:0/4122372645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000ccc0 con 0x7f86fc101960 2024-03-19T00:43:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc19f9e0 con 0x7f86fc101960 2024-03-19T00:43:54.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86eb7fe640 1 -- 172.21.15.7:0/4122372645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f00108e0 con 0x7f86fc101960 2024-03-19T00:43:54.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f86eb7fe640 1 -- 172.21.15.7:0/4122372645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00187c0 con 0x7f86fc101960 2024-03-19T00:43:54.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.377+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc19ff30 con 0x7f86fc101960 2024-03-19T00:43:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.379+0000 7f86eb7fe640 1 -- 172.21.15.7:0/4122372645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f86f0010a90 con 0x7f86fc101960 2024-03-19T00:43:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.379+0000 7f8703b8f740 1 Processor -- start 2024-03-19T00:43:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 -- start start 2024-03-19T00:43:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 0x5652460dff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460e05b0 0x5652460e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56524603f500 con 0x5652460e05b0 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565245fe7be0 con 0x5652460db440 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f8703b8f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56524603b5a0 con 0x5652460daa50 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86eb7fe640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86e40a2800 0x7f86e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460e05b0 0x5652460e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 0x5652460dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:36518/0 (socket says 172.21.15.7:36518) 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 0x5652460dff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46248/0 (socket says 172.21.15.7:46248) 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 -- 172.21.15.7:0/1521216526 learned_addr learned my addr 172.21.15.7:0/1521216526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86e40a2800 0x7f86e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86eb7fe640 1 -- 172.21.15.7:0/4122372645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f86f00cf230 con 0x7f86fc101960 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 -- 172.21.15.7:0/1521216526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 msgr2=0x5652460dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 0x5652460dff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 -- 172.21.15.7:0/1521216526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460e05b0 msgr2=0x5652460e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460e05b0 0x5652460e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fad9d640 1 -- 172.21.15.7:0/1521216526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565246009000 con 0x5652460daa50 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86fa59c640 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 0x5652460dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.380+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460e05b0 0x5652460e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f86fad9d640 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460dae70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005f40 tx=0x7f86f400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a230e88669227c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f86fad9d640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86e40a2800 0x7f86e40a4cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f86f400fc90 tx=0x7f86f400e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f86e8ff9640 1 -- 172.21.15.7:0/1521216526 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4023070 con 0x5652460daa50 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f86e8ff9640 1 -- 172.21.15.7:0/1521216526 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f4011930 con 0x5652460daa50 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f86e8ff9640 1 -- 172.21.15.7:0/1521216526 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f401fbc0 con 0x5652460daa50 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f8703b8f740 1 -- 172.21.15.7:0/1521216526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 msgr2=0x5652460dae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.381+0000 7f8703b8f740 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460dae70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005f40 tx=0x7f86f400f790 comp rx=0 tx=0).stop 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- 172.21.15.7:0/1521216526 shutdown_connections 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460e05b0 0x5652460e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460db440 0x5652460dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 --2- 172.21.15.7:0/1521216526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460dae70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- 172.21.15.7:0/1521216526 >> 172.21.15.7:0/1521216526 conn(0x5652460d5fc0 msgr2=0x5652460d63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- 172.21.15.7:0/1521216526 shutdown_connections 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- 172.21.15.7:0/1521216526 wait complete. 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- start start 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 Processor -- start 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5652460e3500 con 0x5652460daa50 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565245ffb160 con 0x5652460e05b0 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5652460e3680 con 0x5652460db440 2024-03-19T00:43:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f8703b8f740 10 client.0 _ll_register_callbacks cb 0x565246037a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44956/0 (socket says 172.21.15.7:44956) 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 -- 172.21.15.7:0/2125629085 learned_addr learned my addr 172.21.15.7:0/2125629085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fad9d640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 -- 172.21.15.7:0/2125629085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 msgr2=0x5652460e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 -- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 msgr2=0x5652460ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.382+0000 7f86fa59c640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86fa59c640 1 -- 172.21.15.7:0/2125629085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565245e1d7d0 con 0x5652460daa50 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86fa59c640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e4f20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7485d37629e641e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86e1ffb640 1 -- 172.21.15.7:0/2125629085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec046070 con 0x5652460daa50 2024-03-19T00:43:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86e1ffb640 1 -- 172.21.15.7:0/2125629085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f86ec0046c0 con 0x5652460daa50 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4011930 con 0x5652460daa50 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86e1ffb640 1 -- 172.21.15.7:0/2125629085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86ec005d50 con 0x5652460daa50 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 10 client.16782 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565245d533f0 con 0x5652460daa50 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86e1ffb640 1 -- 172.21.15.7:0/2125629085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f4011930 con 0x5652460daa50 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f86e1ffb640 1 -- 172.21.15.7:0/2125629085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f86ec0038c0 con 0x5652460daa50 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 10 client.16782 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 20 client.16782 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652460f0cb0 0x5652460f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 4 client.16782 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.383+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524604ca40 con 0x5652460f0cb0 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.384+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652460f0cb0 0x5652460f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.384+0000 7f86e1ffb640 10 client.16782 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.384+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652460f0cb0 0x5652460f3100 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9025f7cb18c8e115 server_cookie=df27b3ead28eb852 in_seq=0 out_seq=0 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f86e1ffb640 1 -- 172.21.15.7:0/2125629085 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56524604ca40 con 0x5652460f0cb0 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f86e1ffb640 10 client.16782 handle_command_reply: tid=0 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 client.16782 shutdown 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 msgr2=0x5652460e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e4f20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec005250 comp rx=0 tx=0).stop 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 shutdown_connections 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652460f0cb0 0x5652460f3100 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2125629085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e4f20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f86e1ffb640 0 client.16782 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 >> 172.21.15.7:0/2125629085 conn(0x5652460d5fc0 msgr2=0x5652460d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 shutdown_connections 2024-03-19T00:43:54.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 1 -- 172.21.15.7:0/2125629085 wait complete. 2024-03-19T00:43:54.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.385+0000 7f8703b8f740 20 client.16782 trim_cache size 0 max 16384 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 Processor -- start 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 -- start start 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460e4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 0x5652460ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1acc10 con 0x5652460db440 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1aa4e0 con 0x5652460e05b0 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f8703b8f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a8330 con 0x5652460daa50 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f86f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 0x5652460ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f86f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 0x5652460ec3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44972/0 (socket says 172.21.15.7:44972) 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f86f9d9b640 1 -- 172.21.15.7:0/2372812110 learned_addr learned my addr 172.21.15.7:0/2372812110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f86fa59c640 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.402+0000 7f86fad9d640 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86f9d9b640 1 -- 172.21.15.7:0/2372812110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 msgr2=0x5652460e4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460e4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86f9d9b640 1 -- 172.21.15.7:0/2372812110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 msgr2=0x5652460e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86f9d9b640 1 -- 172.21.15.7:0/2372812110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565245d130f0 con 0x5652460db440 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86fa59c640 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 0x5652460ec3f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f86f00981e0 tx=0x7f86f006d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71678ab6c4273a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86e3fff640 1 -- 172.21.15.7:0/2372812110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0099b10 con 0x5652460db440 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86e3fff640 1 -- 172.21.15.7:0/2372812110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f0084aa0 con 0x5652460db440 2024-03-19T00:43:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f86e3fff640 1 -- 172.21.15.7:0/2372812110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00a3b20 con 0x5652460db440 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 -- 172.21.15.7:0/2372812110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 msgr2=0x5652460ec3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 0x5652460ec3f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f86f00981e0 tx=0x7f86f006d490 comp rx=0 tx=0).stop 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 -- 172.21.15.7:0/2372812110 shutdown_connections 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460db440 0x5652460ec3f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 --2- 172.21.15.7:0/2372812110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460daa50 0x5652460e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 -- 172.21.15.7:0/2372812110 >> 172.21.15.7:0/2372812110 conn(0x5652460d5fc0 msgr2=0x5652460d9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 -- 172.21.15.7:0/2372812110 shutdown_connections 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.403+0000 7f8703b8f740 1 -- 172.21.15.7:0/2372812110 wait complete. 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 -- start start 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 Processor -- start 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a89a0 con 0x5652460daa50 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a9e10 con 0x5652460e05b0 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a9070 con 0x5652460db440 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f8703b8f740 10 client.0 _ll_register_callbacks cb 0x7f86ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44986/0 (socket says 172.21.15.7:44986) 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 -- 172.21.15.7:0/3378352739 learned_addr learned my addr 172.21.15.7:0/3378352739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fad9d640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 -- 172.21.15.7:0/3378352739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 msgr2=0x5652460e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 -- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 msgr2=0x5652460eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:43:54.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 -- 172.21.15.7:0/3378352739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652460eb450 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86fa59c640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e9910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f86ec003ce0 tx=0x7f86ec009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16624333c0cb61a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.404+0000 7f86e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f86e1ffb640 1 -- 172.21.15.7:0/3378352739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec046070 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f86e1ffb640 1 -- 172.21.15.7:0/3378352739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f86ec0406d0 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0084aa0 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f86e1ffb640 1 -- 172.21.15.7:0/3378352739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86ec02cc90 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 10 client.16788 fetch_fsmap learned FSMap version 27 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565246157000 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f86e1ffb640 1 -- 172.21.15.7:0/3378352739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f0084aa0 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f86e1ffb640 1 -- 172.21.15.7:0/3378352739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f86ec0046c0 con 0x5652460daa50 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 10 client.16788 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:43:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 20 client.16788 populate_metadata read hostname 'smithi007' 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652461578c0 0x565246159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 4 client.16788 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565246046d90 con 0x5652461578c0 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.405+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652461578c0 0x565246159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f86e1ffb640 10 client.16788 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f86f9d9b640 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652461578c0 0x565246159ce0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2952a0ab3a4ef07 server_cookie=9ebac57b750db68f in_seq=0 out_seq=0 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f86e1ffb640 1 -- 172.21.15.7:0/3378352739 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565246046d90 con 0x5652461578c0 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f86e1ffb640 10 client.16788 handle_command_reply: tid=0 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 client.16788 shutdown 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 msgr2=0x5652460e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e9910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f86ec003ce0 tx=0x7f86ec009620 comp rx=0 tx=0).stop 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 shutdown_connections 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652460e05b0 0x5652460eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652460db440 0x5652460e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5652461578c0 0x565246159ce0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 --2- 172.21.15.7:0/3378352739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652460daa50 0x5652460e9910 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 >> 172.21.15.7:0/3378352739 conn(0x5652460d5fc0 msgr2=0x5652460d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f86e1ffb640 0 client.16788 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 shutdown_connections 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 1 -- 172.21.15.7:0/3378352739 wait complete. 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.406+0000 7f8703b8f740 20 client.16788 trim_cache size 0 max 16384 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:43:54.414 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: "f618a999-ce9e-474c-b854-0c5865303ef3": { 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "f618a999-ce9e-474c-b854-0c5865303ef3", 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:43:54.415 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86e40a2800 msgr2=0x7f86e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86e40a2800 0x7f86e40a4cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f86f400fc90 tx=0x7f86f400e000 comp rx=0 tx=0).stop 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 msgr2=0x7f86fc19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 0x7f86fc19c820 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f86f000e9f0 tx=0x7f86f000edf0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 shutdown_connections 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f86e40a2800 0x7f86e40a4cc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86fc102320 0x7f86fc19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86fc101960 0x7f86fc19c820 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 --2- 172.21.15.7:0/4122372645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86fc100c50 0x7f86fc100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:43:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 >> 172.21.15.7:0/4122372645 conn(0x7f86fc0fb440 msgr2=0x7f86fc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:43:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 shutdown_connections 2024-03-19T00:43:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:43:54.407+0000 7f8700fa9640 1 -- 172.21.15.7:0/4122372645 wait complete. 2024-03-19T00:43:54.418 INFO:tasks.cephfs.filesystem:scrub status for tag:f618a999-ce9e-474c-b854-0c5865303ef3 - {'path': '/', 'tag': 'f618a999-ce9e-474c-b854-0c5865303ef3', 'options': 'recursive,force'} 2024-03-19T00:43:55.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:54 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:43:55.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:54 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:55.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:54 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:43:55.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:54 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:54 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:43:55.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:54 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:55 smithi204 ceph-mon[66572]: pgmap v530: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:55 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:43:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:55 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:55 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:43:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:55 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:55 smithi044 ceph-mon[66366]: pgmap v530: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:55 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:43:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:55 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:55 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:43:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:55 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:55 smithi007 ceph-mon[62854]: pgmap v530: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:43:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:55 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:43:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:55 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:55 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:43:56.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:55 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:43:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:57 smithi204 ceph-mon[66572]: pgmap v531: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:57 smithi044 ceph-mon[66366]: pgmap v531: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:43:58.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:57 smithi007 ceph-mon[62854]: pgmap v531: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:43:59 smithi204 ceph-mon[66572]: pgmap v532: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:43:59 smithi044 ceph-mon[66366]: pgmap v532: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:00.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:43:59 smithi007 ceph-mon[62854]: pgmap v532: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:01 smithi204 ceph-mon[66572]: pgmap v533: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:44:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:01 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:44:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:01 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:01 smithi044 ceph-mon[66366]: pgmap v533: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:01 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:01 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:44:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:01 smithi007 ceph-mon[62854]: pgmap v533: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:44:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:01 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:44:02.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:01 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:44:04.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:03 smithi204 ceph-mon[66572]: pgmap v534: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:44:04.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:03 smithi044 ceph-mon[66366]: pgmap v534: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:44:04.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:03 smithi007 ceph-mon[62854]: pgmap v534: 129 pgs: 129 active+clean; 1.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-19T00:44:06.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:05 smithi204 ceph-mon[66572]: pgmap v535: 129 pgs: 129 active+clean; 1.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:44:06.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:05 smithi044 ceph-mon[66366]: pgmap v535: 129 pgs: 129 active+clean; 1.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:44:06.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:05 smithi007 ceph-mon[62854]: pgmap v535: 129 pgs: 129 active+clean; 1.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:44:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:08 smithi204 ceph-mon[66572]: pgmap v536: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-03-19T00:44:08.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:08 smithi044 ceph-mon[66366]: pgmap v536: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-03-19T00:44:08.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:08 smithi007 ceph-mon[62854]: pgmap v536: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-03-19T00:44:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:10 smithi204 ceph-mon[66572]: pgmap v537: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:44:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:10 smithi044 ceph-mon[66366]: pgmap v537: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:44:10.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:10 smithi007 ceph-mon[62854]: pgmap v537: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:44:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:12 smithi204 ceph-mon[66572]: pgmap v538: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:12.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:12 smithi044 ceph-mon[66366]: pgmap v538: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:12.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:12 smithi007 ceph-mon[62854]: pgmap v538: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:14 smithi204 ceph-mon[66572]: pgmap v539: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:14 smithi044 ceph-mon[66366]: pgmap v539: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:14.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:14 smithi007 ceph-mon[62854]: pgmap v539: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:16 smithi204 ceph-mon[66572]: pgmap v540: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:16 smithi044 ceph-mon[66366]: pgmap v540: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:16.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:16 smithi007 ceph-mon[62854]: pgmap v540: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:18 smithi204 ceph-mon[66572]: pgmap v541: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:18 smithi044 ceph-mon[66366]: pgmap v541: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:18.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:18 smithi007 ceph-mon[62854]: pgmap v541: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:20.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:20 smithi007 ceph-mon[62854]: pgmap v542: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:44:20.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:20 smithi044 ceph-mon[66366]: pgmap v542: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:44:20.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:20 smithi204 ceph-mon[66572]: pgmap v542: 129 pgs: 129 active+clean; 1.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:22 smithi044 ceph-mon[66366]: pgmap v543: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:44:22.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:22 smithi007 ceph-mon[62854]: pgmap v543: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:44:22.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:22 smithi204 ceph-mon[66572]: pgmap v543: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:44:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:24 smithi044 ceph-mon[66366]: pgmap v544: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:24.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:44:24.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:44:24.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:24 smithi007 ceph-mon[62854]: pgmap v544: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:24.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:24 smithi204 ceph-mon[66572]: pgmap v544: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 -- 172.21.15.7:0/2493140299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc1022f0 msgr2=0x7fafbc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 --2- 172.21.15.7:0/2493140299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc1022f0 0x7fafbc108b80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fafac009ca0 tx=0x7fafac02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 -- 172.21.15.7:0/2493140299 shutdown_connections 2024-03-19T00:44:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 --2- 172.21.15.7:0/2493140299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc1022f0 0x7fafbc108b80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 --2- 172.21.15.7:0/2493140299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc101930 0x7fafbc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 --2- 172.21.15.7:0/2493140299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc100740 0x7fafbc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 -- 172.21.15.7:0/2493140299 >> 172.21.15.7:0/2493140299 conn(0x7fafbc0fbcb0 msgr2=0x7fafbc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 -- 172.21.15.7:0/2493140299 shutdown_connections 2024-03-19T00:44:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.536+0000 7fafc373b640 1 -- 172.21.15.7:0/2493140299 wait complete. 2024-03-19T00:44:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 Processor -- start 2024-03-19T00:44:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 -- start start 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 0x7fafbc069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc101930 0x7fafbc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc1022f0 0x7fafbc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10c270 con 0x7fafbc100740 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10c0f0 con 0x7fafbc1022f0 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc373b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10c3f0 con 0x7fafbc101930 2024-03-19T00:44:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 0x7fafbc069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 0x7fafbc069660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52654/0 (socket says 172.21.15.7:52654) 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 -- 172.21.15.7:0/1270575748 learned_addr learned my addr 172.21.15.7:0/1270575748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc101930 0x7fafbc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc1022f0 0x7fafbc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 -- 172.21.15.7:0/1270575748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc101930 msgr2=0x7fafbc19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc101930 0x7fafbc19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 -- 172.21.15.7:0/1270575748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc1022f0 msgr2=0x7fafbc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc1022f0 0x7fafbc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 -- 172.21.15.7:0/1270575748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc19f690 con 0x7fafbc100740 2024-03-19T00:44:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc101930 0x7fafbc19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc1022f0 0x7fafbc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.537+0000 7fafc14d7640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 0x7fafbc069660 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9c0 tx=0x7fafb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.538+0000 7fafba7fc640 1 -- 172.21.15.7:0/1270575748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb400cd80 con 0x7fafbc100740 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.538+0000 7fafba7fc640 1 -- 172.21.15.7:0/1270575748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb40035f0 con 0x7fafbc100740 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.538+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc19f920 con 0x7fafbc100740 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.538+0000 7fafba7fc640 1 -- 172.21.15.7:0/1270575748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4010690 con 0x7fafbc100740 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.538+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc19fe70 con 0x7fafbc100740 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.540+0000 7fafba7fc640 1 -- 172.21.15.7:0/1270575748 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fafb40037a0 con 0x7fafbc100740 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.540+0000 7fafc6401740 1 Processor -- start 2024-03-19T00:44:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 -- start start 2024-03-19T00:44:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449be7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449becf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449befa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafba7fc640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafa40a2880 0x7fafa40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565449b4c500 con 0x565449be8440 2024-03-19T00:44:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565449af4be0 con 0x565449bed5b0 2024-03-19T00:44:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc6401740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565449b485a0 con 0x565449be7a50 2024-03-19T00:44:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449be7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449becf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449be7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:51562/0 (socket says 172.21.15.7:51562) 2024-03-19T00:44:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 -- 172.21.15.7:0/1859063740 learned_addr learned my addr 172.21.15.7:0/1859063740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449befa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafa40a2880 0x7fafa40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafba7fc640 1 -- 172.21.15.7:0/1270575748 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fafb40cf260 con 0x7fafbc100740 2024-03-19T00:44:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 -- 172.21.15.7:0/1859063740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 msgr2=0x565449befa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449befa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 -- 172.21.15.7:0/1859063740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 msgr2=0x565449becf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449becf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.541+0000 7fafc1cd8640 1 -- 172.21.15.7:0/1859063740 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565449b16000 con 0x565449be7a50 2024-03-19T00:44:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449be7e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafac033000 tx=0x7fafac003b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ed9b3a63219bc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafa40a2880 0x7fafa40a4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafac032380 tx=0x7fafac042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafa37fe640 1 -- 172.21.15.7:0/1859063740 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac051070 con 0x565449be7a50 2024-03-19T00:44:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafa37fe640 1 -- 172.21.15.7:0/1859063740 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafac032950 con 0x565449be7a50 2024-03-19T00:44:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafa37fe640 1 -- 172.21.15.7:0/1859063740 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac039040 con 0x565449be7a50 2024-03-19T00:44:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 -- 172.21.15.7:0/1859063740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 msgr2=0x565449be7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449be7e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafac033000 tx=0x7fafac003b60 comp rx=0 tx=0).stop 2024-03-19T00:44:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 -- 172.21.15.7:0/1859063740 shutdown_connections 2024-03-19T00:44:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449befa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449becf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 --2- 172.21.15.7:0/1859063740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449be7e70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.542+0000 7fafc6401740 1 -- 172.21.15.7:0/1859063740 >> 172.21.15.7:0/1859063740 conn(0x565449be2fc0 msgr2=0x565449be33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 -- 172.21.15.7:0/1859063740 shutdown_connections 2024-03-19T00:44:24.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 -- 172.21.15.7:0/1859063740 wait complete. 2024-03-19T00:44:24.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 -- start start 2024-03-19T00:44:24.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 Processor -- start 2024-03-19T00:44:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449be8440 0x565449bf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449bed5b0 0x565449bf9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565449bf0500 con 0x565449bed5b0 2024-03-19T00:44:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565449b08160 con 0x565449be8440 2024-03-19T00:44:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x565449bf0680 con 0x565449be7a50 2024-03-19T00:44:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc6401740 10 client.0 _ll_register_callbacks cb 0x565449b44a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc1cd8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449bed5b0 0x565449bf9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:51564/0 (socket says 172.21.15.7:51564) 2024-03-19T00:44:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc14d7640 1 -- 172.21.15.7:0/1573231145 learned_addr learned my addr 172.21.15.7:0/1573231145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.543+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449be8440 0x565449bf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc14d7640 1 -- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449be8440 msgr2=0x565449bf23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc14d7640 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449be8440 0x565449bf23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc14d7640 1 -- 172.21.15.7:0/1573231145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449bed5b0 msgr2=0x565449bf9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc14d7640 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449bed5b0 0x565449bf9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc14d7640 1 -- 172.21.15.7:0/1573231145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544992a7d0 con 0x565449be7a50 2024-03-19T00:44:24.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc14d7640 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf1eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafb40cca50 tx=0x7fafb409b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b168a0efd2a44d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:44:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafa0ff9640 1 -- 172.21.15.7:0/1573231145 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4066060 con 0x565449be7a50 2024-03-19T00:44:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafa0ff9640 1 -- 172.21.15.7:0/1573231145 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fafb409cdd0 con 0x565449be7a50 2024-03-19T00:44:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565449b59a40 con 0x565449be7a50 2024-03-19T00:44:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafa0ff9640 1 -- 172.21.15.7:0/1573231145 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fafb409bfa0 con 0x565449be7a50 2024-03-19T00:44:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc6401740 10 client.26557 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654498603f0 con 0x565449be7a50 2024-03-19T00:44:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.544+0000 7fafa0ff9640 1 -- 172.21.15.7:0/1573231145 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x565449b59a40 con 0x565449be7a50 2024-03-19T00:44:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafa0ff9640 1 -- 172.21.15.7:0/1573231145 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fafb40a3bb0 con 0x565449be7a50 2024-03-19T00:44:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc6401740 10 client.26557 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc6401740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc6401740 20 client.26557 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc6401740 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449bfddc0 0x565449c00260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc6401740 4 client.26557 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565449b53d90 con 0x565449bfddc0 2024-03-19T00:44:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449bfddc0 0x565449c00260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafa0ff9640 10 client.26557 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.545+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449bfddc0 0x565449c00260 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e944d255a5f166 server_cookie=8ebb9621400ea67d in_seq=0 out_seq=0 2024-03-19T00:44:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.546+0000 7fafa0ff9640 1 -- 172.21.15.7:0/1573231145 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565449b53d90 con 0x565449bfddc0 2024-03-19T00:44:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.546+0000 7fafa0ff9640 10 client.26557 handle_command_reply: tid=0 2024-03-19T00:44:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.546+0000 7fafc6401740 1 client.26557 shutdown 2024-03-19T00:44:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 msgr2=0x565449bf1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf1eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafb40cca50 tx=0x7fafb409b4a0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 shutdown_connections 2024-03-19T00:44:24.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449bfddc0 0x565449c00260 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449bed5b0 0x565449bf9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449be8440 0x565449bf23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 --2- 172.21.15.7:0/1573231145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf1eb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafa0ff9640 0 client.26557 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:24.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 >> 172.21.15.7:0/1573231145 conn(0x565449be2fc0 msgr2=0x565449be21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 shutdown_connections 2024-03-19T00:44:24.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 1 -- 172.21.15.7:0/1573231145 wait complete. 2024-03-19T00:44:24.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.547+0000 7fafc6401740 20 client.26557 trim_cache size 0 max 16384 2024-03-19T00:44:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 Processor -- start 2024-03-19T00:44:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 -- start start 2024-03-19T00:44:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449c04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449c05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aca90 con 0x565449be8440 2024-03-19T00:44:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aa360 con 0x565449bed5b0 2024-03-19T00:44:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.564+0000 7fafc6401740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a81b0 con 0x565449be7a50 2024-03-19T00:44:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc0cd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449c05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449c04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc0cd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52678/0 (socket says 172.21.15.7:52678) 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449c05060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56920/0 (socket says 172.21.15.7:56920) 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc0cd6640 1 -- 172.21.15.7:0/3016222694 learned_addr learned my addr 172.21.15.7:0/3016222694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 -- 172.21.15.7:0/3016222694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 msgr2=0x565449c04b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449c04b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 -- 172.21.15.7:0/3016222694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 msgr2=0x565449bf93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 -- 172.21.15.7:0/3016222694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654498200f0 con 0x565449bed5b0 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449c05060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fafac006010 tx=0x7fafac02e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368caf204248fc26 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafa2ffd640 1 -- 172.21.15.7:0/3016222694 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac039050 con 0x565449bed5b0 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafa2ffd640 1 -- 172.21.15.7:0/3016222694 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafac05a070 con 0x565449bed5b0 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.565+0000 7fafa2ffd640 1 -- 172.21.15.7:0/3016222694 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac051040 con 0x565449bed5b0 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 -- 172.21.15.7:0/3016222694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 msgr2=0x565449c05060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449c05060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fafac006010 tx=0x7fafac02e040 comp rx=0 tx=0).stop 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 -- 172.21.15.7:0/3016222694 shutdown_connections 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449c05060 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 --2- 172.21.15.7:0/3016222694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449c04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 -- 172.21.15.7:0/3016222694 >> 172.21.15.7:0/3016222694 conn(0x565449be2fc0 msgr2=0x565449bf1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 -- 172.21.15.7:0/3016222694 shutdown_connections 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 -- 172.21.15.7:0/3016222694 wait complete. 2024-03-19T00:44:24.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 -- start start 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.566+0000 7fafc6401740 1 Processor -- start 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449bf7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a8820 con 0x565449be8440 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a9c90 con 0x565449bed5b0 2024-03-19T00:44:24.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a8ef0 con 0x565449be7a50 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc6401740 10 client.0 _ll_register_callbacks cb 0x7fafb4003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc0cd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:51584/0 (socket says 172.21.15.7:51584) 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 -- 172.21.15.7:0/3670186535 learned_addr learned my addr 172.21.15.7:0/3670186535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc1cd8640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449bf7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 -- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 msgr2=0x565449bf7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449bf7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 -- 172.21.15.7:0/3670186535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 msgr2=0x565449bf94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 -- 172.21.15.7:0/3670186535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565449bf7e80 con 0x565449be7a50 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafc14d7640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf8f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009fd0 tx=0x7fafb40862c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b413a4621b9469 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.567+0000 7fafa0ff9640 1 -- 172.21.15.7:0/3670186535 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40a4040 con 0x565449be7a50 2024-03-19T00:44:24.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafa0ff9640 1 -- 172.21.15.7:0/3670186535 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fafb4066060 con 0x565449be7a50 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafac05a070 con 0x565449be7a50 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafa0ff9640 1 -- 172.21.15.7:0/3670186535 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fafb409b8a0 con 0x565449be7a50 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 10 client.26563 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565449c64d70 con 0x565449be7a50 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafa0ff9640 1 -- 172.21.15.7:0/3670186535 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafac05a070 con 0x565449be7a50 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafa0ff9640 1 -- 172.21.15.7:0/3670186535 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fafb40975f0 con 0x565449be7a50 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 10 client.26563 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 10 client.26563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 20 client.26563 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449c65330 0x565449c67740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 4 client.26563 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565449ad1540 con 0x565449c65330 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.568+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449c65330 0x565449c67740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafa0ff9640 10 client.26563 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc0cd6640 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449c65330 0x565449c67740 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc0c841b434bfd6 server_cookie=8e9dee390a90654f in_seq=0 out_seq=0 2024-03-19T00:44:24.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafa0ff9640 1 -- 172.21.15.7:0/3670186535 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565449ad1540 con 0x565449c65330 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafa0ff9640 10 client.26563 handle_command_reply: tid=0 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 client.26563 shutdown 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 msgr2=0x565449bf8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf8f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009fd0 tx=0x7fafb40862c0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 shutdown_connections 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565449bed5b0 0x565449bf7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x565449c65330 0x565449c67740 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565449be8440 0x565449bf94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 --2- 172.21.15.7:0/3670186535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565449be7a50 0x565449bf8f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafa0ff9640 0 client.26563 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 >> 172.21.15.7:0/3670186535 conn(0x565449be2fc0 msgr2=0x565449be1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 shutdown_connections 2024-03-19T00:44:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 1 -- 172.21.15.7:0/3670186535 wait complete. 2024-03-19T00:44:24.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc6401740 20 client.26563 trim_cache size 0 max 16384 2024-03-19T00:44:24.598 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:44:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:44:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:44:24.598 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafa40a2880 msgr2=0x7fafa40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc373b640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafa40a2880 0x7fafa40a4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafac032380 tx=0x7fafac042000 comp rx=0 tx=0).stop 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.569+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 msgr2=0x7fafbc069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 0x7fafbc069660 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9c0 tx=0x7fafb400ee80 comp rx=0 tx=0).stop 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 shutdown_connections 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fafa40a2880 0x7fafa40a4d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafbc1022f0 0x7fafbc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc101930 0x7fafbc19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 --2- 172.21.15.7:0/1270575748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafbc100740 0x7fafbc069660 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 >> 172.21.15.7:0/1270575748 conn(0x7fafbc0fbcb0 msgr2=0x7fafbc104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 shutdown_connections 2024-03-19T00:44:24.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.570+0000 7fafc373b640 1 -- 172.21.15.7:0/1270575748 wait complete. 2024-03-19T00:44:24.602 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:44:24.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:44:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 -- 172.21.15.7:0/1174305736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e78100b10 msgr2=0x7f3e78100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1174305736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e78100b10 0x7f3e78100f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009900 tx=0x7f3e6c02bcd0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 -- 172.21.15.7:0/1174305736 shutdown_connections 2024-03-19T00:44:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1174305736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e781014b0 0x7f3e78101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1174305736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e78100b10 0x7f3e78100f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1174305736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e781001b0 0x7f3e781005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 -- 172.21.15.7:0/1174305736 >> 172.21.15.7:0/1174305736 conn(0x7f3e780fbd10 msgr2=0x7f3e780fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 -- 172.21.15.7:0/1174305736 shutdown_connections 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.744+0000 7f3e7de19640 1 -- 172.21.15.7:0/1174305736 wait complete. 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 Processor -- start 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 -- start start 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e781001b0 0x7f3e78199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 0x7f3e7819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e781014b0 0x7f3e781a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810f5d0 con 0x7f3e78100b10 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810f450 con 0x7f3e781014b0 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e7de19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810f750 con 0x7f3e781001b0 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e76ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 0x7f3e7819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e76ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 0x7f3e7819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52704/0 (socket says 172.21.15.7:52704) 2024-03-19T00:44:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e76ffd640 1 -- 172.21.15.7:0/1062027269 learned_addr learned my addr 172.21.15.7:0/1062027269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e777fe640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e781001b0 0x7f3e78199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e77fff640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e781014b0 0x7f3e781a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e76ffd640 1 -- 172.21.15.7:0/1062027269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e781001b0 msgr2=0x7f3e78199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.745+0000 7f3e76ffd640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e781001b0 0x7f3e78199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e76ffd640 1 -- 172.21.15.7:0/1062027269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e781014b0 msgr2=0x7f3e781a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e76ffd640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e781014b0 0x7f3e781a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e76ffd640 1 -- 172.21.15.7:0/1062027269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e781a18c0 con 0x7f3e78100b10 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e76ffd640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 0x7f3e7819a1c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c043020 con 0x7f3e78100b10 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e781a1b50 con 0x7f3e78100b10 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e6c02dd20 con 0x7f3e78100b10 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781a20f0 con 0x7f3e78100b10 2024-03-19T00:44:24.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.746+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c003ae0 con 0x7f3e78100b10 2024-03-19T00:44:24.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.747+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7810b0e0 con 0x7f3e78100b10 2024-03-19T00:44:24.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.748+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f3e6c046020 con 0x7f3e78100b10 2024-03-19T00:44:24.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.749+0000 7f3e74ff9640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3e600a2800 0x7f3e600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.749+0000 7f3e777fe640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3e600a2800 0x7f3e600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.749+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3e6c0f0e40 con 0x7f3e78100b10 2024-03-19T00:44:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.749+0000 7f3e777fe640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3e600a2800 0x7f3e600a4cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3e7819ab50 tx=0x7f3e68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:24.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.753+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e6c04d020 con 0x7f3e78100b10 2024-03-19T00:44:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.975+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e78062420 con 0x7f3e78100b10 2024-03-19T00:44:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.976+0000 7f3e74ff9640 1 -- 172.21.15.7:0/1062027269 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f3e6c0bdb80 con 0x7f3e78100b10 2024-03-19T00:44:24.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:44:24.977 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:44:24.977 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3e600a2800 msgr2=0x7f3e600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3e600a2800 0x7f3e600a4cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3e7819ab50 tx=0x7f3e68005f90 comp rx=0 tx=0).stop 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 msgr2=0x7f3e7819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 0x7f3e7819a1c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c005e30 comp rx=0 tx=0).stop 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 shutdown_connections 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3e600a2800 0x7f3e600a4cc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e781014b0 0x7f3e781a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e78100b10 0x7f3e7819a1c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 --2- 172.21.15.7:0/1062027269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e781001b0 0x7f3e78199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 >> 172.21.15.7:0/1062027269 conn(0x7f3e780fbd10 msgr2=0x7f3e780fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 shutdown_connections 2024-03-19T00:44:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:24.978+0000 7f3e7de19640 1 -- 172.21.15.7:0/1062027269 wait complete. 2024-03-19T00:44:24.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 -- 172.21.15.7:0/1770245328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628102360 msgr2=0x7fe628104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 --2- 172.21.15.7:0/1770245328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628102360 0x7fe628104750 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009900 tx=0x7fe61c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 -- 172.21.15.7:0/1770245328 shutdown_connections 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 --2- 172.21.15.7:0/1770245328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6281075c0 0x7fe6281099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 --2- 172.21.15.7:0/1770245328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe628104c90 0x7fe628107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 --2- 172.21.15.7:0/1770245328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628102360 0x7fe628104750 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 -- 172.21.15.7:0/1770245328 >> 172.21.15.7:0/1770245328 conn(0x7fe6280fbc90 msgr2=0x7fe6280fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 -- 172.21.15.7:0/1770245328 shutdown_connections 2024-03-19T00:44:25.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.094+0000 7fe62eaea640 1 -- 172.21.15.7:0/1770245328 wait complete. 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 Processor -- start 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 -- start start 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe628102360 0x7fe628101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 0x7fe628100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6281075c0 0x7fe628100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810d380 con 0x7fe628104c90 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810d200 con 0x7fe6281075c0 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62eaea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810d500 con 0x7fe628102360 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 0x7fe628100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 0x7fe628100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52724/0 (socket says 172.21.15.7:52724) 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 -- 172.21.15.7:0/3494480693 learned_addr learned my addr 172.21.15.7:0/3494480693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62c886640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe628102360 0x7fe628101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe62d087640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6281075c0 0x7fe628100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 -- 172.21.15.7:0/3494480693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe628102360 msgr2=0x7fe628101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe628102360 0x7fe628101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 -- 172.21.15.7:0/3494480693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6281075c0 msgr2=0x7fe628100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6281075c0 0x7fe628100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 -- 172.21.15.7:0/3494480693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6281010e0 con 0x7fe628104c90 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe627fff640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 0x7fe628100310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe61400e9c0 tx=0x7fe61400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.095+0000 7fe625ffb640 1 -- 172.21.15.7:0/3494480693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61400cd80 con 0x7fe628104c90 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.096+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6281a1bb0 con 0x7fe628104c90 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.096+0000 7fe625ffb640 1 -- 172.21.15.7:0/3494480693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6140054d0 con 0x7fe628104c90 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.096+0000 7fe625ffb640 1 -- 172.21.15.7:0/3494480693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614010430 con 0x7fe628104c90 2024-03-19T00:44:25.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.096+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6281a2100 con 0x7fe628104c90 2024-03-19T00:44:25.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe625ffb640 1 -- 172.21.15.7:0/3494480693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe614004fd0 con 0x7fe628104c90 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 Processor -- start 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe625ffb640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 -- start start 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233e2f0 0x5594d233e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d23437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d2343e50 0x5594d23462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe62d087640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594d22a24b0 con 0x5594d233ece0 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5594d224ad70 con 0x5594d233e2f0 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe625ffb640 1 -- 172.21.15.7:0/3494480693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe6140cf200 con 0x7fe628104c90 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe6317b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594d229e550 con 0x5594d2343e50 2024-03-19T00:44:25.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe627fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d23437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.098+0000 7fe62c886640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233e2f0 0x5594d233e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d23437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52726/0 (socket says 172.21.15.7:52726) 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 -- 172.21.15.7:0/3396778601 learned_addr learned my addr 172.21.15.7:0/3396778601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe62c886640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233e2f0 0x5594d233e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56964/0 (socket says 172.21.15.7:56964) 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe62d087640 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d2343e50 0x5594d23462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 -- 172.21.15.7:0/3396778601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d2343e50 msgr2=0x5594d23462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d2343e50 0x5594d23462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 -- 172.21.15.7:0/3396778601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233e2f0 msgr2=0x5594d233e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233e2f0 0x5594d233e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 -- 172.21.15.7:0/3396778601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d226d210 con 0x5594d233ece0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe62d087640 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d2343e50 0x5594d23462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe62d087640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe620005fd0 tx=0x7fe620011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe627fff640 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d23437e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe61409b7d0 tx=0x7fe614097b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6514cb90dc2f7932 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe612ffd640 1 -- 172.21.15.7:0/3396778601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140cce60 con 0x5594d233ece0 2024-03-19T00:44:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe612ffd640 1 -- 172.21.15.7:0/3396778601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe61409bf40 con 0x5594d233ece0 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.099+0000 7fe612ffd640 1 -- 172.21.15.7:0/3396778601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61409baa0 con 0x5594d233ece0 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 -- 172.21.15.7:0/3396778601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 msgr2=0x5594d23437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d23437e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe61409b7d0 tx=0x7fe614097b60 comp rx=0 tx=0).stop 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 -- 172.21.15.7:0/3396778601 shutdown_connections 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d2343e50 0x5594d23462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d23437e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3396778601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233e2f0 0x5594d233e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 -- 172.21.15.7:0/3396778601 >> 172.21.15.7:0/3396778601 conn(0x5594d2339860 msgr2=0x5594d2339c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 -- 172.21.15.7:0/3396778601 shutdown_connections 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 -- 172.21.15.7:0/3396778601 wait complete. 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 -- start start 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.100+0000 7fe6317b4740 1 Processor -- start 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d23487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233ece0 0x5594d2348cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 0x5594d234fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594d2346e00 con 0x5594d2343e50 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5594d225e310 con 0x5594d233ece0 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594d2346f80 con 0x5594d233e2f0 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 10 client.0 _ll_register_callbacks cb 0x5594d22522d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 0x5594d234fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 0x5594d234fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52732/0 (socket says 172.21.15.7:52732) 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 -- 172.21.15.7:0/2629538057 learned_addr learned my addr 172.21.15.7:0/2629538057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62c886640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d23487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe627fff640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233ece0 0x5594d2348cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 -- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 msgr2=0x5594d23487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d23487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 -- 172.21.15.7:0/2629538057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233ece0 msgr2=0x5594d2348cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233ece0 0x5594d2348cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 -- 172.21.15.7:0/2629538057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d1fb63f0 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe62d087640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 0x5594d234fd30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe620005f40 tx=0x7fe620004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf49a7aafc19c9d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe607fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe607fff640 1 -- 172.21.15.7:0/2629538057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620024070 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe607fff640 1 -- 172.21.15.7:0/2629538057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe620029460 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61409bf40 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe607fff640 1 -- 172.21.15.7:0/2629538057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe62001a040 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 10 client.16824 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.101+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594d20807d0 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe607fff640 1 -- 172.21.15.7:0/2629538057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe61409bf40 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe607fff640 1 -- 172.21.15.7:0/2629538057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe6200044b0 con 0x5594d2343e50 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe6317b4740 10 client.16824 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe6317b4740 10 client.16824 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe6317b4740 20 client.16824 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe6317b4740 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23546b0 0x5594d2356b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe6317b4740 4 client.16824 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594d23003e0 con 0x5594d23546b0 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe62c886640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23546b0 0x5594d2356b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe607fff640 10 client.16824 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.102+0000 7fe62c886640 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23546b0 0x5594d2356b10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=456e5b8b8ac026a4 server_cookie=61a6dda99dab7be8 in_seq=0 out_seq=0 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe607fff640 1 -- 172.21.15.7:0/2629538057 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5594d23003e0 con 0x5594d23546b0 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe607fff640 10 client.16824 handle_command_reply: tid=0 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 client.16824 shutdown 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 msgr2=0x5594d234fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 0x5594d234fd30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe620005f40 tx=0x7fe620004ab0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 shutdown_connections 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23546b0 0x5594d2356b10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d2343e50 0x5594d234fd30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe607fff640 0 client.16824 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d233ece0 0x5594d2348cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 --2- 172.21.15.7:0/2629538057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d23487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 >> 172.21.15.7:0/2629538057 conn(0x5594d2339860 msgr2=0x5594d2338ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.103+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 shutdown_connections 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.104+0000 7fe6317b4740 1 -- 172.21.15.7:0/2629538057 wait complete. 2024-03-19T00:44:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.104+0000 7fe6317b4740 20 client.16824 trim_cache size 0 max 16384 2024-03-19T00:44:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 Processor -- start 2024-03-19T00:44:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 -- start start 2024-03-19T00:44:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 0x5594d23401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233ece0 0x5594d2340720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281aee00 con 0x5594d233e2f0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ac6d0 con 0x5594d2343e50 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe6317b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281aa520 con 0x5594d233ece0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 0x5594d23401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 0x5594d23401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52736/0 (socket says 172.21.15.7:52736) 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 -- 172.21.15.7:0/3257240239 learned_addr learned my addr 172.21.15.7:0/3257240239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe627fff640 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233ece0 0x5594d2340720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62d087640 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 -- 172.21.15.7:0/3257240239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233ece0 msgr2=0x5594d2340720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233ece0 0x5594d2340720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 -- 172.21.15.7:0/3257240239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 msgr2=0x5594d234fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 -- 172.21.15.7:0/3257240239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d230f880 con 0x5594d233e2f0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.117+0000 7fe62c886640 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 0x5594d23401e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009900 tx=0x7fe61c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b2e164b90f5353 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6127fc640 1 -- 172.21.15.7:0/3257240239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c04f020 con 0x5594d233e2f0 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6127fc640 1 -- 172.21.15.7:0/3257240239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe61c02cba0 con 0x5594d233e2f0 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6127fc640 1 -- 172.21.15.7:0/3257240239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c049070 con 0x5594d233e2f0 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 -- 172.21.15.7:0/3257240239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 msgr2=0x5594d23401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 0x5594d23401e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009900 tx=0x7fe61c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 -- 172.21.15.7:0/3257240239 shutdown_connections 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233ece0 0x5594d2340720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 --2- 172.21.15.7:0/3257240239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233e2f0 0x5594d23401e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 -- 172.21.15.7:0/3257240239 >> 172.21.15.7:0/3257240239 conn(0x5594d2339860 msgr2=0x5594d23487b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 -- 172.21.15.7:0/3257240239 shutdown_connections 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.118+0000 7fe6317b4740 1 -- 172.21.15.7:0/3257240239 wait complete. 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 -- start start 2024-03-19T00:44:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 Processor -- start 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d234d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d234d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281aab90 con 0x5594d233ece0 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ac000 con 0x5594d2343e50 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ab260 con 0x5594d233e2f0 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe6317b4740 10 client.0 _ll_register_callbacks cb 0x7fe620009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d234d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d234d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52742/0 (socket says 172.21.15.7:52742) 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 -- 172.21.15.7:0/1551750425 learned_addr learned my addr 172.21.15.7:0/1551750425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe62d087640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe62c886640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d234d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 -- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 msgr2=0x5594d234d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d234d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 -- 172.21.15.7:0/1551750425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 msgr2=0x5594d234e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe627fff640 1 -- 172.21.15.7:0/1551750425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d234ed90 con 0x5594d233ece0 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.119+0000 7fe62d087640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe627fff640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d234d790 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe61409b540 tx=0x7fe6140a4050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a552e08c32df433d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe607fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe607fff640 1 -- 172.21.15.7:0/1551750425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140cce60 con 0x5594d233ece0 2024-03-19T00:44:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe607fff640 1 -- 172.21.15.7:0/1551750425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe6140a3b50 con 0x5594d233ece0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61c02cba0 con 0x5594d233ece0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe607fff640 1 -- 172.21.15.7:0/1551750425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe61409bf40 con 0x5594d233ece0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 10 client.16830 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594d2350090 con 0x5594d233ece0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe607fff640 1 -- 172.21.15.7:0/1551750425 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe61c02cba0 con 0x5594d233ece0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe607fff640 1 -- 172.21.15.7:0/1551750425 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe614097590 con 0x5594d233ece0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 10 client.16830 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 10 client.16830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 20 client.16830 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23bace0 0x5594d23bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 4 client.16830 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594d22af9f0 con 0x5594d23bace0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.120+0000 7fe62c886640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23bace0 0x5594d23bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe607fff640 10 client.16830 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:25.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe62c886640 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23bace0 0x5594d23bd0d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62b3dad010f75d9d server_cookie=631ebd7961d95de4 in_seq=0 out_seq=0 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe607fff640 1 -- 172.21.15.7:0/1551750425 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594d22af9f0 con 0x5594d23bace0 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe607fff640 10 client.16830 handle_command_reply: tid=0 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 client.16830 shutdown 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 msgr2=0x5594d234d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d234d790 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe61409b540 tx=0x7fe6140a4050 comp rx=0 tx=0).stop 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 shutdown_connections 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5594d23bace0 0x5594d23bd0d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594d2343e50 0x5594d234e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594d233ece0 0x5594d234d790 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe607fff640 0 client.16830 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 --2- 172.21.15.7:0/1551750425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594d233e2f0 0x5594d234d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 >> 172.21.15.7:0/1551750425 conn(0x5594d2339860 msgr2=0x5594d23401e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 shutdown_connections 2024-03-19T00:44:25.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 1 -- 172.21.15.7:0/1551750425 wait complete. 2024-03-19T00:44:25.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.121+0000 7fe6317b4740 20 client.16830 trim_cache size 0 max 16384 2024-03-19T00:44:25.131 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 msgr2=0x7fe6180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe620005fd0 tx=0x7fe620011bc0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 msgr2=0x7fe628100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 0x7fe628100310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe61400e9c0 tx=0x7fe61400ee80 comp rx=0 tx=0).stop 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 shutdown_connections 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe6180a2800 0x7fe6180a4cc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6281075c0 0x7fe628100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe628104c90 0x7fe628100310 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 --2- 172.21.15.7:0/3494480693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe628102360 0x7fe628101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 >> 172.21.15.7:0/3494480693 conn(0x7fe6280fbc90 msgr2=0x7fe628102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 shutdown_connections 2024-03-19T00:44:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.122+0000 7fe62eaea640 1 -- 172.21.15.7:0/3494480693 wait complete. 2024-03-19T00:44:25.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:251/600: open fd = 3 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:252/600: open fd = 3 2024-03-19T00:44:25.213 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:253/600: open fd = 3 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:254/600: open fd = 3 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:255/600: open fd = 3 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:256/600: open fd = 3 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:257/600: open fd = 3 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:258/600: open fd = 3 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:259/600: open fd = 3 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:260/600: open fd = 3 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:261/600: open fd = 3 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:262/600: open fd = 3 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.216 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:263/600: open fd = 3 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:264/600: open fd = 3 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:265/600: open fd = 3 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:266/600: open fd = 3 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:267/600: open fd = 3 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:268/600: open fd = 3 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:269/600: open fd = 3 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.218 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:270/600: open fd = 3 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:271/600: open fd = 3 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:272/600: open fd = 3 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:273/600: open fd = 3 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:274/600: open fd = 3 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:275/600: open fd = 3 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:276/600: open fd = 3 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:277/600: open fd = 3 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:278/600: open fd = 3 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:279/600: open fd = 3 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:280/600: open fd = 3 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:281/600: open fd = 3 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:282/600: open fd = 3 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:283/600: open fd = 3 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:284/600: open fd = 3 2024-03-19T00:44:25.222 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:285/600: open fd = 3 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:286/600: open fd = 3 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:287/600: open fd = 3 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:288/600: open fd = 3 2024-03-19T00:44:25.223 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:289/600: open fd = 3 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:290/600: open fd = 3 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:291/600: open fd = 3 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:292/600: open fd = 3 2024-03-19T00:44:25.224 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:293/600: open fd = 3 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:294/600: open fd = 3 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:295/600: open fd = 3 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:296/600: open fd = 3 2024-03-19T00:44:25.225 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:297/600: open fd = 3 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:298/600: open fd = 3 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:299/600: open fd = 3 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:300/600: open fd = 3 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:301/600: open fd = 3 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:302/600: open fd = 3 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:303/600: open fd = 3 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:304/600: open fd = 3 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:305/600: open fd = 3 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:306/600: open fd = 3 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:307/600: open fd = 3 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:308/600: open fd = 3 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:309/600: open fd = 3 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:310/600: open fd = 3 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:311/600: open fd = 3 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.229 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.230 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:44:25.230 INFO:tasks.workunit.client.0.smithi007.stdout:312/600: open fd = 3 2024-03-19T00:44:25.230 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.230 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:44:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2615078969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb541022f0 msgr2=0x7feb54108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2615078969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb541022f0 0x7feb54108b80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7feb44009ca0 tx=0x7feb4402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2615078969 shutdown_connections 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2615078969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb541022f0 0x7feb54108b80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2615078969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb54101930 0x7feb54101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2615078969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb54100740 0x7feb54100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2615078969 >> 172.21.15.7:0/2615078969 conn(0x7feb540fbc90 msgr2=0x7feb540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2615078969 shutdown_connections 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.237+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2615078969 wait complete. 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 Processor -- start 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 -- start start 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb54100740 0x7feb541968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 0x7feb54196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb541022f0 0x7feb5419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb5410c250 con 0x7feb54101930 2024-03-19T00:44:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb5410c0d0 con 0x7feb541022f0 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5c0b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb5410c3d0 con 0x7feb54100740 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 0x7feb54196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 0x7feb54196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52766/0 (socket says 172.21.15.7:52766) 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5964d640 1 -- 172.21.15.7:0/2575741396 learned_addr learned my addr 172.21.15.7:0/2575741396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb59e4e640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb54100740 0x7feb541968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.238+0000 7feb5a64f640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb541022f0 0x7feb5419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5964d640 1 -- 172.21.15.7:0/2575741396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb54100740 msgr2=0x7feb541968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5964d640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb54100740 0x7feb541968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5964d640 1 -- 172.21.15.7:0/2575741396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb541022f0 msgr2=0x7feb5419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5964d640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb541022f0 0x7feb5419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5964d640 1 -- 172.21.15.7:0/2575741396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5419e510 con 0x7feb54101930 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5964d640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 0x7feb54196e30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb4800ea60 tx=0x7feb4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb52ffd640 1 -- 172.21.15.7:0/2575741396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4800cde0 con 0x7feb54101930 2024-03-19T00:44:25.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5419e800 con 0x7feb54101930 2024-03-19T00:44:25.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb52ffd640 1 -- 172.21.15.7:0/2575741396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb480035f0 con 0x7feb54101930 2024-03-19T00:44:25.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb52ffd640 1 -- 172.21.15.7:0/2575741396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48010690 con 0x7feb54101930 2024-03-19T00:44:25.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.239+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5419ed50 con 0x7feb54101930 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb52ffd640 1 -- 172.21.15.7:0/2575741396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7feb480037a0 con 0x7feb54101930 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 Processor -- start 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 -- start start 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 0x560b5c5d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b5c5364b0 con 0x560b5c5d7e50 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560b5c4ded70 con 0x560b5c5d22f0 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560b5c532550 con 0x560b5c5d2ce0 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5a64f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 0x560b5c5d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5a64f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 0x560b5c5d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57028/0 (socket says 172.21.15.7:57028) 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb5a64f640 1 -- 172.21.15.7:0/1278447172 learned_addr learned my addr 172.21.15.7:0/1278447172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb59e4e640 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.242+0000 7feb52ffd640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7feb3c0a2800 0x7feb3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 -- 172.21.15.7:0/1278447172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 msgr2=0x560b5c5d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 -- 172.21.15.7:0/1278447172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 msgr2=0x560b5c5da2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5da2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 -- 172.21.15.7:0/1278447172 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5c501210 con 0x560b5c5d22f0 2024-03-19T00:44:25.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb59e4e640 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb52ffd640 1 -- 172.21.15.7:0/2575741396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7feb480cf050 con 0x7feb54101930 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7feb3c0a2800 0x7feb3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 0x560b5c5d2710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb44032380 tx=0x7feb44009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a62a78d7c7987d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb3bfff640 1 -- 172.21.15.7:0/1278447172 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44039040 con 0x560b5c5d22f0 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb5a64f640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7feb3c0a2800 0x7feb3c0a4cc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feb44048000 tx=0x7feb44045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb3bfff640 1 -- 172.21.15.7:0/1278447172 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb44003cf0 con 0x560b5c5d22f0 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.243+0000 7feb3bfff640 1 -- 172.21.15.7:0/1278447172 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4404a640 con 0x560b5c5d22f0 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 -- 172.21.15.7:0/1278447172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 msgr2=0x560b5c5d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 0x560b5c5d2710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb44032380 tx=0x7feb44009890 comp rx=0 tx=0).stop 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 -- 172.21.15.7:0/1278447172 shutdown_connections 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5da2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/1278447172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d22f0 0x560b5c5d2710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 -- 172.21.15.7:0/1278447172 >> 172.21.15.7:0/1278447172 conn(0x560b5c5cd860 msgr2=0x560b5c5cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 -- 172.21.15.7:0/1278447172 shutdown_connections 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 -- 172.21.15.7:0/1278447172 wait complete. 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 -- start start 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.244+0000 7feb5ed7e740 1 Processor -- start 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d7e50 0x560b5c5e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b5c5dae00 con 0x560b5c5d22f0 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560b5c4f2310 con 0x560b5c5d2ce0 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560b5c5daf80 con 0x560b5c5d7e50 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5ed7e740 10 client.0 _ll_register_callbacks cb 0x560b5c4e62d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5dc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52788/0 (socket says 172.21.15.7:52788) 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 -- 172.21.15.7:0/3599543296 learned_addr learned my addr 172.21.15.7:0/3599543296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5964d640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5a64f640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d7e50 0x560b5c5e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 -- 172.21.15.7:0/3599543296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d7e50 msgr2=0x560b5c5e3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d7e50 0x560b5c5e3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 -- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 msgr2=0x560b5c5dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5dcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 -- 172.21.15.7:0/3599543296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5c24a3f0 con 0x560b5c5d22f0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb5964d640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb59e4e640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5dc880 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0126e0 tx=0x7feb4c012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cff7e4ebf7d5fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.245+0000 7feb397fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb397fa640 1 -- 172.21.15.7:0/3599543296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c004030 con 0x560b5c5d22f0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb397fa640 1 -- 172.21.15.7:0/3599543296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7feb4c0041d0 con 0x560b5c5d22f0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb44003cf0 con 0x560b5c5d22f0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb397fa640 1 -- 172.21.15.7:0/3599543296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb4c025050 con 0x560b5c5d22f0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 10 client.16842 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b5c3147d0 con 0x560b5c5d22f0 2024-03-19T00:44:25.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb397fa640 1 -- 172.21.15.7:0/3599543296 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb44003cf0 con 0x560b5c5d22f0 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb397fa640 1 -- 172.21.15.7:0/3599543296 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7feb4c003630 con 0x560b5c5d22f0 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 10 client.16842 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 10 client.16842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 20 client.16842 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c5e8720 0x560b5c5eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 4 client.16842 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5c5943e0 con 0x560b5c5e8720 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.246+0000 7feb5964d640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c5e8720 0x560b5c5eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.247+0000 7feb397fa640 10 client.16842 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.247+0000 7feb5964d640 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c5e8720 0x560b5c5eaba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4b52b5d5192dc9d server_cookie=7c26be2236806511 in_seq=0 out_seq=0 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.247+0000 7feb397fa640 1 -- 172.21.15.7:0/3599543296 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560b5c5943e0 con 0x560b5c5e8720 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.247+0000 7feb397fa640 10 client.16842 handle_command_reply: tid=0 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.247+0000 7feb5ed7e740 1 client.16842 shutdown 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 msgr2=0x560b5c5dc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5dc880 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0126e0 tx=0x7feb4c012ae0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 shutdown_connections 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c5e8720 0x560b5c5eaba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d7e50 0x560b5c5e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3599543296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5dc880 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb397fa640 0 client.16842 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 >> 172.21.15.7:0/3599543296 conn(0x560b5c5cd860 msgr2=0x560b5c5ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 shutdown_connections 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3599543296 wait complete. 2024-03-19T00:44:25.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.248+0000 7feb5ed7e740 20 client.16842 trim_cache size 0 max 16384 2024-03-19T00:44:25.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 Processor -- start 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 -- start start 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d22f0 0x560b5c5ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5eff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb541aba60 con 0x560b5c5d7e50 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb541a9330 con 0x560b5c5d2ce0 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb541a7180 con 0x560b5c5d22f0 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5a64f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5a64f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52804/0 (socket says 172.21.15.7:52804) 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5a64f640 1 -- 172.21.15.7:0/337396300 learned_addr learned my addr 172.21.15.7:0/337396300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb5964d640 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.261+0000 7feb59e4e640 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d22f0 0x560b5c5ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb5a64f640 1 -- 172.21.15.7:0/337396300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d22f0 msgr2=0x560b5c5ef4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb5a64f640 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d22f0 0x560b5c5ef4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb5a64f640 1 -- 172.21.15.7:0/337396300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 msgr2=0x560b5c5ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb5a64f640 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb5a64f640 1 -- 172.21.15.7:0/337396300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5c5a3880 con 0x560b5c5d7e50 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb5a64f640 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5eff20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feb44032420 tx=0x7feb44003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a04e05fe43f68b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb3b7fe640 1 -- 172.21.15.7:0/337396300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44059020 con 0x560b5c5d7e50 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb3b7fe640 1 -- 172.21.15.7:0/337396300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb4404adf0 con 0x560b5c5d7e50 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.262+0000 7feb3b7fe640 1 -- 172.21.15.7:0/337396300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44039040 con 0x560b5c5d7e50 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- 172.21.15.7:0/337396300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 msgr2=0x560b5c5eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5eff20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feb44032420 tx=0x7feb44003bc0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- 172.21.15.7:0/337396300 shutdown_connections 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d7e50 0x560b5c5eff20 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d2ce0 0x560b5c5ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/337396300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d22f0 0x560b5c5ef4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- 172.21.15.7:0/337396300 >> 172.21.15.7:0/337396300 conn(0x560b5c5cd860 msgr2=0x560b5c5d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- 172.21.15.7:0/337396300 shutdown_connections 2024-03-19T00:44:25.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- 172.21.15.7:0/337396300 wait complete. 2024-03-19T00:44:25.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- start start 2024-03-19T00:44:25.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 Processor -- start 2024-03-19T00:44:25.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d7e50 0x560b5c64ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb541a77f0 con 0x560b5c5d22f0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb541a8c60 con 0x560b5c5d7e50 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb541a7ec0 con 0x560b5c5d2ce0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5ed7e740 10 client.0 _ll_register_callbacks cb 0x7feb4c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb59e4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb59e4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5e3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52812/0 (socket says 172.21.15.7:52812) 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb59e4e640 1 -- 172.21.15.7:0/3432866430 learned_addr learned my addr 172.21.15.7:0/3432866430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.263+0000 7feb5964d640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5a64f640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d7e50 0x560b5c64ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb59e4e640 1 -- 172.21.15.7:0/3432866430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 msgr2=0x560b5c5e3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb59e4e640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5e3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb59e4e640 1 -- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d7e50 msgr2=0x560b5c64ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb59e4e640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d7e50 0x560b5c64ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb59e4e640 1 -- 172.21.15.7:0/3432866430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5c64f340 con 0x560b5c5d22f0 2024-03-19T00:44:25.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb59e4e640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5e3120 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009e00 tx=0x7feb4c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8164dcc0045a12b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb397fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb397fa640 1 -- 172.21.15.7:0/3432866430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c022040 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb397fa640 1 -- 172.21.15.7:0/3432866430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7feb4c0050c0 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb4404adf0 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb397fa640 1 -- 172.21.15.7:0/3432866430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb4c038050 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5ed7e740 10 client.16848 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b5c6506b0 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb397fa640 1 -- 172.21.15.7:0/3432866430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb4404adf0 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb397fa640 1 -- 172.21.15.7:0/3432866430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7feb4c00fb00 con 0x560b5c5d22f0 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5ed7e740 10 client.16848 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5ed7e740 10 client.16848 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.264+0000 7feb5ed7e740 20 client.16848 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c651080 0x560b5c6534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 4 client.16848 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:44:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b5c5439f0 con 0x560b5c651080 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5964d640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c651080 0x560b5c6534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5964d640 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c651080 0x560b5c6534a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f3d2dcf9621724b server_cookie=58b92a2ee61da031 in_seq=0 out_seq=0 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb397fa640 10 client.16848 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb397fa640 1 -- 172.21.15.7:0/3432866430 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b5c5439f0 con 0x560b5c651080 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb397fa640 10 client.16848 handle_command_reply: tid=0 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 client.16848 shutdown 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 msgr2=0x560b5c5e3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5e3120 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009e00 tx=0x7feb4c003ef0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 shutdown_connections 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x560b5c651080 0x560b5c6534a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560b5c5d7e50 0x560b5c64ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560b5c5d2ce0 0x560b5c5e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 --2- 172.21.15.7:0/3432866430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b5c5d22f0 0x560b5c5e3120 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb397fa640 0 client.16848 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 >> 172.21.15.7:0/3432866430 conn(0x560b5c5cd860 msgr2=0x560b5c5cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 shutdown_connections 2024-03-19T00:44:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.265+0000 7feb5ed7e740 1 -- 172.21.15.7:0/3432866430 wait complete. 2024-03-19T00:44:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5ed7e740 20 client.16848 trim_cache size 0 max 16384 2024-03-19T00:44:25.273 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7feb3c0a2800 msgr2=0x7feb3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7feb3c0a2800 0x7feb3c0a4cc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feb44048000 tx=0x7feb44045000 comp rx=0 tx=0).stop 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 msgr2=0x7feb54196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 0x7feb54196e30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb4800ea60 tx=0x7feb4800ef20 comp rx=0 tx=0).stop 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 shutdown_connections 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7feb3c0a2800 0x7feb3c0a4cc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb541022f0 0x7feb5419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54101930 0x7feb54196e30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 --2- 172.21.15.7:0/2575741396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb54100740 0x7feb541968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 >> 172.21.15.7:0/2575741396 conn(0x7feb540fbc90 msgr2=0x7feb54104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 shutdown_connections 2024-03-19T00:44:25.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.266+0000 7feb5c0b2640 1 -- 172.21.15.7:0/2575741396 wait complete. 2024-03-19T00:44:25.276 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:25 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1062027269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:44:25.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:44:25.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:25 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1062027269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:44:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 -- 172.21.15.7:0/2824046355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d181022f0 msgr2=0x7f9d18108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 --2- 172.21.15.7:0/2824046355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d181022f0 0x7f9d18108b80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009ca0 tx=0x7f9d0802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 -- 172.21.15.7:0/2824046355 shutdown_connections 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 --2- 172.21.15.7:0/2824046355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d181022f0 0x7f9d18108b80 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 --2- 172.21.15.7:0/2824046355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d18101930 0x7f9d18101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 --2- 172.21.15.7:0/2824046355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d18100740 0x7f9d18100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.381+0000 7f9d1d663640 1 -- 172.21.15.7:0/2824046355 >> 172.21.15.7:0/2824046355 conn(0x7f9d180fbc90 msgr2=0x7f9d180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 -- 172.21.15.7:0/2824046355 shutdown_connections 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 -- 172.21.15.7:0/2824046355 wait complete. 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 Processor -- start 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 -- start start 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 0x7f9d181979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d18101930 0x7f9d18197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d181022f0 0x7f9d1819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1810d390 con 0x7f9d18100740 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1810d210 con 0x7f9d181022f0 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.382+0000 7f9d1d663640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1810d510 con 0x7f9d18101930 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 0x7f9d181979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 0x7f9d181979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52830/0 (socket says 172.21.15.7:52830) 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 -- 172.21.15.7:0/3499119209 learned_addr learned my addr 172.21.15.7:0/3499119209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d167fc640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d18101930 0x7f9d18197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d177fe640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d181022f0 0x7f9d1819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 -- 172.21.15.7:0/3499119209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d18101930 msgr2=0x7f9d18197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d18101930 0x7f9d18197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 -- 172.21.15.7:0/3499119209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d181022f0 msgr2=0x7f9d1819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d181022f0 0x7f9d1819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 -- 172.21.15.7:0/3499119209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1819f5d0 con 0x7f9d18100740 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 0x7f9d181979b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000e9c0 tx=0x7f9d1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d0ffff640 1 -- 172.21.15.7:0/3499119209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000cde0 con 0x7f9d18100740 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d0ffff640 1 -- 172.21.15.7:0/3499119209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d100108e0 con 0x7f9d18100740 2024-03-19T00:44:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1819f8c0 con 0x7f9d18100740 2024-03-19T00:44:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d0ffff640 1 -- 172.21.15.7:0/3499119209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10018990 con 0x7f9d18100740 2024-03-19T00:44:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.383+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1819fe10 con 0x7f9d18100740 2024-03-19T00:44:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.385+0000 7f9d0ffff640 1 -- 172.21.15.7:0/3499119209 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f9d1001f020 con 0x7f9d18100740 2024-03-19T00:44:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 Processor -- start 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 -- start start 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557232436700 0x557232436b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x55723243bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x55723243e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5572323964b0 con 0x55723243c260 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55723233ed70 con 0x5572324370f0 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d2024a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557232392550 con 0x557232436700 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d16ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x55723243bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x55723243e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d0ffff640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d040a2800 0x7f9d040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557232436700 0x557232436b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d16ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x55723243bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57082/0 (socket says 172.21.15.7:57082) 2024-03-19T00:44:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557232436700 0x557232436b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:51734/0 (socket says 172.21.15.7:51734) 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.386+0000 7f9d16ffd640 1 -- 172.21.15.7:0/2497338436 learned_addr learned my addr 172.21.15.7:0/2497338436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d177fe640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d040a2800 0x7f9d040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d167fc640 1 -- 172.21.15.7:0/2497338436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557232436700 msgr2=0x557232436b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d167fc640 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557232436700 0x557232436b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d0ffff640 1 -- 172.21.15.7:0/3499119209 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d100cefd0 con 0x7f9d18100740 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d167fc640 1 -- 172.21.15.7:0/2497338436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 msgr2=0x55723243bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d167fc640 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x55723243bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d167fc640 1 -- 172.21.15.7:0/2497338436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557232361210 con 0x55723243c260 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d167fc640 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x55723243e6f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009bd0 tx=0x7f9d00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0a83077c1a28dd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d0d7fa640 1 -- 172.21.15.7:0/2497338436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00004230 con 0x55723243c260 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d177fe640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d040a2800 0x7f9d040a4cc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d08006010 tx=0x7f9d0802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d0d7fa640 1 -- 172.21.15.7:0/2497338436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d00009e40 con 0x55723243c260 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.387+0000 7f9d0d7fa640 1 -- 172.21.15.7:0/2497338436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00004c60 con 0x55723243c260 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- 172.21.15.7:0/2497338436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 msgr2=0x55723243e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x55723243e6f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009bd0 tx=0x7f9d00011810 comp rx=0 tx=0).stop 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- 172.21.15.7:0/2497338436 shutdown_connections 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x55723243e6f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x55723243bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2497338436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557232436700 0x557232436b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- 172.21.15.7:0/2497338436 >> 172.21.15.7:0/2497338436 conn(0x55723242d860 msgr2=0x55723242dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- 172.21.15.7:0/2497338436 shutdown_connections 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- 172.21.15.7:0/2497338436 wait complete. 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- start start 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 Processor -- start 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x557232440b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572324370f0 0x557232441070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x5572324480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55723243f210 con 0x55723243c260 2024-03-19T00:44:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557232352310 con 0x557232436700 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55723243f390 con 0x5572324370f0 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.388+0000 7f9d2024a740 10 client.0 _ll_register_callbacks cb 0x5572323462d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x5572324480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x5572324480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52850/0 (socket says 172.21.15.7:52850) 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 -- 172.21.15.7:0/2785295750 learned_addr learned my addr 172.21.15.7:0/2785295750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d167fc640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572324370f0 0x557232441070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x557232440b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 -- 172.21.15.7:0/2785295750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572324370f0 msgr2=0x557232441070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572324370f0 0x557232441070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 -- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 msgr2=0x557232440b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x557232440b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 -- 172.21.15.7:0/2785295750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572320aa3f0 con 0x55723243c260 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d177fe640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x5572324480b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009ca0 tx=0x7f9d080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40762a8f19f0c30e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9cfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/2785295750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08053060 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/2785295750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d08004450 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572323f43e0 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/2785295750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d0805a050 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d2024a740 10 client.16860 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572321747d0 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.389+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/2785295750 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5572323f43e0 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/2785295750 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9d0803f070 con 0x55723243c260 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d2024a740 10 client.16860 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d2024a740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d2024a740 20 client.16860 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55723244ca30 0x55723244eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d2024a740 4 client.16860 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572323a39f0 con 0x55723244ca30 2024-03-19T00:44:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55723244ca30 0x55723244eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9cfe7fc640 10 client.16860 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.390+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55723244ca30 0x55723244eec0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e009f3b3da990454 server_cookie=28f2305644bea828 in_seq=0 out_seq=0 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/2785295750 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572323a39f0 con 0x55723244ca30 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9cfe7fc640 10 client.16860 handle_command_reply: tid=0 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 client.16860 shutdown 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 msgr2=0x5572324480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x5572324480b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009ca0 tx=0x7f9d080095c0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 shutdown_connections 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55723244ca30 0x55723244eec0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55723243c260 0x5572324480b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9cfe7fc640 0 client.16860 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572324370f0 0x557232441070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 --2- 172.21.15.7:0/2785295750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x557232440b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 >> 172.21.15.7:0/2785295750 conn(0x55723242d860 msgr2=0x55723242cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 shutdown_connections 2024-03-19T00:44:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 1 -- 172.21.15.7:0/2785295750 wait complete. 2024-03-19T00:44:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.391+0000 7f9d2024a740 20 client.16860 trim_cache size 0 max 16384 2024-03-19T00:44:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.404+0000 7f9d2024a740 1 Processor -- start 2024-03-19T00:44:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.404+0000 7f9d2024a740 1 -- start start 2024-03-19T00:44:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 0x55723242c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x557232448070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x55723242c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d2024a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181aa950 con 0x557232436700 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d2024a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181a8220 con 0x5572324370f0 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d2024a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181a6070 con 0x55723243c260 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 0x55723242c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 0x55723242c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52860/0 (socket says 172.21.15.7:52860) 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 -- 172.21.15.7:0/521658824 learned_addr learned my addr 172.21.15.7:0/521658824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d177fe640 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x55723242c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d167fc640 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x557232448070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 -- 172.21.15.7:0/521658824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 msgr2=0x55723242c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x55723242c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 -- 172.21.15.7:0/521658824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 msgr2=0x557232448070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x557232448070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 -- 172.21.15.7:0/521658824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557232403880 con 0x557232436700 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 0x55723242c310 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9d10018460 tx=0x7f9d100ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b61f3a4ea26ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d0cff9640 1 -- 172.21.15.7:0/521658824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100b4070 con 0x557232436700 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d0cff9640 1 -- 172.21.15.7:0/521658824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d1009bcb0 con 0x557232436700 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.405+0000 7f9d0cff9640 1 -- 172.21.15.7:0/521658824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10010a90 con 0x557232436700 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 -- 172.21.15.7:0/521658824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 msgr2=0x55723242c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 0x55723242c310 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9d10018460 tx=0x7f9d100ce040 comp rx=0 tx=0).stop 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 -- 172.21.15.7:0/521658824 shutdown_connections 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x55723242c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5572324370f0 0x557232448070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 --2- 172.21.15.7:0/521658824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557232436700 0x55723242c310 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 -- 172.21.15.7:0/521658824 >> 172.21.15.7:0/521658824 conn(0x55723242d860 msgr2=0x55723242dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 -- 172.21.15.7:0/521658824 shutdown_connections 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 -- 172.21.15.7:0/521658824 wait complete. 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 -- start start 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.406+0000 7f9d2024a740 1 Processor -- start 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x5572324460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 0x557232446620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x557232447630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181a66e0 con 0x5572324370f0 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181a7b50 con 0x557232436700 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181a6db0 con 0x55723243c260 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 10 client.0 _ll_register_callbacks cb 0x7f9d00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 0x557232446620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 0x557232446620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52876/0 (socket says 172.21.15.7:52876) 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 -- 172.21.15.7:0/538524912 learned_addr learned my addr 172.21.15.7:0/538524912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d177fe640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x557232447630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x5572324460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 -- 172.21.15.7:0/538524912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 msgr2=0x557232447630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x557232447630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 -- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 msgr2=0x5572324460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x5572324460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 -- 172.21.15.7:0/538524912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557232447c20 con 0x5572324370f0 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d167fc640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 0x557232446620 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55723242e7d0 tx=0x7f9d00009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa80f7af482309d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9cfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/538524912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0001e040 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/538524912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d00005220 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.407+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d1009bcb0 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/538524912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d00034050 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 10 client.16866 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572324b3670 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/538524912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d1009bcb0 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/538524912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9d00018070 con 0x5572324370f0 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 10 client.16866 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 20 client.16866 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572324b3ea0 0x5572324b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 4 client.16866 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55723239dd40 con 0x5572324b3ea0 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572324b3ea0 0x5572324b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9cfe7fc640 10 client.16866 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.408+0000 7f9d16ffd640 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572324b3ea0 0x5572324b62c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac67dd517c496c18 server_cookie=3ae6d38aa4fa06e6 in_seq=0 out_seq=0 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9cfe7fc640 1 -- 172.21.15.7:0/538524912 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55723239dd40 con 0x5572324b3ea0 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9cfe7fc640 10 client.16866 handle_command_reply: tid=0 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 client.16866 shutdown 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 msgr2=0x557232446620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 0x557232446620 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55723242e7d0 tx=0x7f9d00009410 comp rx=0 tx=0).stop 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 shutdown_connections 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723243c260 0x557232447630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5572324b3ea0 0x5572324b62c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5572324370f0 0x557232446620 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 --2- 172.21.15.7:0/538524912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557232436700 0x5572324460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9cfe7fc640 0 client.16866 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 >> 172.21.15.7:0/538524912 conn(0x55723242d860 msgr2=0x557232430ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 shutdown_connections 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 1 -- 172.21.15.7:0/538524912 wait complete. 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d2024a740 20 client.16866 trim_cache size 0 max 16384 2024-03-19T00:44:25.419 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d040a2800 msgr2=0x7f9d040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d040a2800 0x7f9d040a4cc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d08006010 tx=0x7f9d0802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 msgr2=0x7f9d181979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 0x7f9d181979b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000e9c0 tx=0x7f9d1000ee80 comp rx=0 tx=0).stop 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 shutdown_connections 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9d040a2800 0x7f9d040a4cc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d181022f0 0x7f9d1819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d18101930 0x7f9d18197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 --2- 172.21.15.7:0/3499119209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d18100740 0x7f9d181979b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.409+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 >> 172.21.15.7:0/3499119209 conn(0x7f9d180fbc90 msgr2=0x7f9d18104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.410+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 shutdown_connections 2024-03-19T00:44:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:25.410+0000 7f9d1d663640 1 -- 172.21.15.7:0/3499119209 wait complete. 2024-03-19T00:44:25.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:25 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1062027269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:44:26.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:26 smithi044 ceph-mon[66366]: pgmap v545: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:26.423 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:44:26.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:44:26.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:44:26.452 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:26 smithi007 ceph-mon[62854]: pgmap v545: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:26.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:26 smithi204 ceph-mon[66572]: pgmap v545: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:26.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/1456972658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e4100a10 msgr2=0x7ff2e4100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/1456972658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e4100a10 0x7ff2e4100e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009900 tx=0x7ff2d402bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/1456972658 shutdown_connections 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/1456972658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2e41013b0 0x7ff2e4101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/1456972658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e4100a10 0x7ff2e4100e70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/1456972658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e41000b0 0x7ff2e41004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/1456972658 >> 172.21.15.7:0/1456972658 conn(0x7ff2e40fbcb0 msgr2=0x7ff2e40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/1456972658 shutdown_connections 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.542+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/1456972658 wait complete. 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 Processor -- start 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 -- start start 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2e41000b0 0x7ff2e4199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e4100a10 0x7ff2e419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 0x7ff2e41a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410f560 con 0x7ff2e41013b0 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410f3e0 con 0x7ff2e4100a10 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2eb8dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410f6e0 con 0x7ff2e41000b0 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2e9e79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 0x7ff2e41a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2e9e79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 0x7ff2e41a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52892/0 (socket says 172.21.15.7:52892) 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2e9e79640 1 -- 172.21.15.7:0/3982668995 learned_addr learned my addr 172.21.15.7:0/3982668995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2e9678640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2e41000b0 0x7ff2e4199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.543+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e4100a10 0x7ff2e419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2e9e79640 1 -- 172.21.15.7:0/3982668995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2e41000b0 msgr2=0x7ff2e4199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2e41000b0 0x7ff2e4199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2e9e79640 1 -- 172.21.15.7:0/3982668995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e4100a10 msgr2=0x7ff2e419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e4100a10 0x7ff2e419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2e9e79640 1 -- 172.21.15.7:0/3982668995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e41a18d0 con 0x7ff2e41013b0 2024-03-19T00:44:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 0x7ff2e41a1230 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00eb10 tx=0x7ff2dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2da7fc640 1 -- 172.21.15.7:0/3982668995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc004b20 con 0x7ff2e41013b0 2024-03-19T00:44:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2da7fc640 1 -- 172.21.15.7:0/3982668995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2dc004cc0 con 0x7ff2e41013b0 2024-03-19T00:44:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e41a1bc0 con 0x7ff2e41013b0 2024-03-19T00:44:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2da7fc640 1 -- 172.21.15.7:0/3982668995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc010690 con 0x7ff2e41013b0 2024-03-19T00:44:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.544+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a2130 con 0x7ff2e41013b0 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.546+0000 7ff2da7fc640 1 -- 172.21.15.7:0/3982668995 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7ff2dc005020 con 0x7ff2e41013b0 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 Processor -- start 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 -- start start 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de47573a0 0x557de47577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de475c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2da7fc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2d00a2880 0x7ff2d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de475f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557de46bc160 con 0x557de4757d90 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557de4664e10 con 0x557de475cf00 2024-03-19T00:44:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557de46b8200 con 0x557de47573a0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e9e79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de475f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e8e77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de475c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e9678640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2d00a2880 0x7ff2d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e9e79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de475f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57148/0 (socket says 172.21.15.7:57148) 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e9e79640 1 -- 172.21.15.7:0/1076865406 learned_addr learned my addr 172.21.15.7:0/1076865406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e8e77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de475c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52908/0 (socket says 172.21.15.7:52908) 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2da7fc640 1 -- 172.21.15.7:0/3982668995 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff2dc0cf350 con 0x7ff2e41013b0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.547+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de47573a0 0x557de47577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9e79640 1 -- 172.21.15.7:0/1076865406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de47573a0 msgr2=0x557de47577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de47573a0 0x557de47577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9e79640 1 -- 172.21.15.7:0/1076865406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 msgr2=0x557de475c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de475c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9e79640 1 -- 172.21.15.7:0/1076865406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de44987e0 con 0x557de475cf00 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9678640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2d00a2880 0x7ff2d00a4d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff2e419ab60 tx=0x7ff2e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de475f390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005850 tx=0x7ff2dc009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d44c4be068f6ec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2cf7fe640 1 -- 172.21.15.7:0/1076865406 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc09b800 con 0x557de475cf00 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2cf7fe640 1 -- 172.21.15.7:0/1076865406 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2dc009030 con 0x557de475cf00 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.548+0000 7ff2cf7fe640 1 -- 172.21.15.7:0/1076865406 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc099080 con 0x557de475cf00 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1076865406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 msgr2=0x557de475f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de475f390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005850 tx=0x7ff2dc009a70 comp rx=0 tx=0).stop 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1076865406 shutdown_connections 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de475f390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de475c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1076865406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de47573a0 0x557de47577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1076865406 >> 172.21.15.7:0/1076865406 conn(0x557de4752910 msgr2=0x557de4752d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1076865406 shutdown_connections 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1076865406 wait complete. 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 -- start start 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.549+0000 7ff2ee5a4740 1 Processor -- start 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de47617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4761d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de4768d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557de475feb0 con 0x557de47573a0 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557de46781a0 con 0x557de475cf00 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557de4760030 con 0x557de4757d90 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 10 client.0 _ll_register_callbacks cb 0x557de43cfc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de47617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de47617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52910/0 (socket says 172.21.15.7:52910) 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 -- 172.21.15.7:0/196884796 learned_addr learned my addr 172.21.15.7:0/196884796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de4768d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4761d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 -- 172.21.15.7:0/196884796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 msgr2=0x557de4761d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4761d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 -- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 msgr2=0x557de4768d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de4768d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 -- 172.21.15.7:0/196884796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de44f8f70 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de4768d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2e9678640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de47617f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000edb0 tx=0x7ff2e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7401761447da5eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ccff9640 1 -- 172.21.15.7:0/196884796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001ccb0 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ccff9640 1 -- 172.21.15.7:0/196884796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff2e0028480 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.550+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc009030 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ccff9640 1 -- 172.21.15.7:0/196884796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2e0031050 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 10 client.16878 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557de4390170 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ccff9640 1 -- 172.21.15.7:0/196884796 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2dc009030 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ccff9640 1 -- 172.21.15.7:0/196884796 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff2e001d670 con 0x557de47573a0 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 10 client.16878 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 20 client.16878 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de476d6e0 0x557de476fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 4 client.16878 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de46c96a0 con 0x557de476d6e0 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.551+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de476d6e0 0x557de476fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de476d6e0 0x557de476fb80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be68d3bb2ea8f734 server_cookie=a96105dc9c0e422b in_seq=0 out_seq=0 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2ccff9640 10 client.16878 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2ccff9640 1 -- 172.21.15.7:0/196884796 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557de46c96a0 con 0x557de476d6e0 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2ccff9640 10 client.16878 handle_command_reply: tid=0 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2ee5a4740 1 client.16878 shutdown 2024-03-19T00:44:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 msgr2=0x557de47617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.552+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de47617f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000edb0 tx=0x7ff2e0004730 comp rx=0 tx=0).stop 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 shutdown_connections 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de4768d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4761d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de476d6e0 0x557de476fb80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/196884796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de47617f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ccff9640 0 client.16878 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 >> 172.21.15.7:0/196884796 conn(0x557de4752910 msgr2=0x557de4751b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 shutdown_connections 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/196884796 wait complete. 2024-03-19T00:44:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.553+0000 7ff2ee5a4740 20 client.16878 trim_cache size 0 max 16384 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 Processor -- start 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 -- start start 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de4761b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4768d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de47512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41aee50 con 0x557de47573a0 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41ac720 con 0x557de475cf00 2024-03-19T00:44:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41aa570 con 0x557de4757d90 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de4761b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de4761b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52920/0 (socket says 172.21.15.7:52920) 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 -- 172.21.15.7:0/2492863811 learned_addr learned my addr 172.21.15.7:0/2492863811 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9e79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de47512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4768d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 -- 172.21.15.7:0/2492863811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 msgr2=0x557de4768d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4768d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 -- 172.21.15.7:0/2492863811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 msgr2=0x557de47512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de47512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 -- 172.21.15.7:0/2492863811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de4752260 con 0x557de47573a0 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de47512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.571+0000 7ff2e9678640 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de4761b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000efd0 tx=0x7ff2e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a4602016b7805e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ceffd640 1 -- 172.21.15.7:0/2492863811 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e003b070 con 0x557de47573a0 2024-03-19T00:44:26.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ceffd640 1 -- 172.21.15.7:0/2492863811 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e001d410 con 0x557de47573a0 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ceffd640 1 -- 172.21.15.7:0/2492863811 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0004400 con 0x557de47573a0 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/2492863811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 msgr2=0x557de4761b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de4761b10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000efd0 tx=0x7ff2e00118d0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/2492863811 shutdown_connections 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de475cf00 0x557de47512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de4757d90 0x557de4768d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/2492863811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de47573a0 0x557de4761b10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/2492863811 >> 172.21.15.7:0/2492863811 conn(0x557de4752910 msgr2=0x557de4761400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/2492863811 shutdown_connections 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.572+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/2492863811 wait complete. 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 -- start start 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 Processor -- start 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de47573a0 0x557de4766820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de4766d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de475cf00 0x557de4767d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41aabe0 con 0x557de4757d90 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41ac050 con 0x557de47573a0 2024-03-19T00:44:26.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41ab2b0 con 0x557de475cf00 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2ee5a4740 10 client.0 _ll_register_callbacks cb 0x7ff2e000e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de4766d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de4766d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52932/0 (socket says 172.21.15.7:52932) 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 -- 172.21.15.7:0/1165955198 learned_addr learned my addr 172.21.15.7:0/1165955198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e9678640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de47573a0 0x557de4766820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e9e79640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de475cf00 0x557de4767d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 -- 172.21.15.7:0/1165955198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de475cf00 msgr2=0x557de4767d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de475cf00 0x557de4767d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 -- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de47573a0 msgr2=0x557de4766820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de47573a0 0x557de4766820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.573+0000 7ff2e8e77640 1 -- 172.21.15.7:0/1165955198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de4768360 con 0x557de4757d90 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2e9678640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de47573a0 0x557de4766820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:26.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2e8e77640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de4766d60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff2d402c170 tx=0x7ff2d4004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96e432ee6433392 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ccff9640 1 -- 172.21.15.7:0/1165955198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4004480 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ccff9640 1 -- 172.21.15.7:0/1165955198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff2d403a040 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e001d410 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ccff9640 1 -- 172.21.15.7:0/1165955198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2d4059050 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 10 client.16884 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557de47d3f50 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ccff9640 1 -- 172.21.15.7:0/1165955198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e001d410 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ccff9640 1 -- 172.21.15.7:0/1165955198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff2d40052a0 con 0x557de4757d90 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 10 client.16884 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 20 client.16884 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de47d4980 0x557de47d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 4 client.16884 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.574+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557de46c39f0 con 0x557de47d4980 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.575+0000 7ff2e9678640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de47d4980 0x557de47d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.575+0000 7ff2e9678640 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de47d4980 0x557de47d6da0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b06a17eb4900fae server_cookie=faae4c922794ed1f in_seq=0 out_seq=0 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.575+0000 7ff2ccff9640 10 client.16884 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ccff9640 1 -- 172.21.15.7:0/1165955198 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557de46c39f0 con 0x557de47d4980 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ccff9640 10 client.16884 handle_command_reply: tid=0 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 client.16884 shutdown 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 msgr2=0x557de4766d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de4766d60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff2d402c170 tx=0x7ff2d4004350 comp rx=0 tx=0).stop 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 shutdown_connections 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557de475cf00 0x557de4767d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x557de47d4980 0x557de47d6da0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557de4757d90 0x557de4766d60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 --2- 172.21.15.7:0/1165955198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557de47573a0 0x557de4766820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 >> 172.21.15.7:0/1165955198 conn(0x557de4752910 msgr2=0x557de47512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ccff9640 0 client.16884 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 shutdown_connections 2024-03-19T00:44:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 1 -- 172.21.15.7:0/1165955198 wait complete. 2024-03-19T00:44:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.576+0000 7ff2ee5a4740 20 client.16884 trim_cache size 0 max 16384 2024-03-19T00:44:26.585 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:44:26.586 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:44:26.586 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "651fb1f3-7d03-4a2f-b8ac-f8f2f7251b3b", 2024-03-19T00:44:26.586 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:44:26.586 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2d00a2880 msgr2=0x7ff2d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2d00a2880 0x7ff2d00a4d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff2e419ab60 tx=0x7ff2e0005f90 comp rx=0 tx=0).stop 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 msgr2=0x7ff2e41a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 0x7ff2e41a1230 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00eb10 tx=0x7ff2dc00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 shutdown_connections 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff2d00a2880 0x7ff2d00a4d40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2e41013b0 0x7ff2e41a1230 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2e4100a10 0x7ff2e419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 --2- 172.21.15.7:0/3982668995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2e41000b0 0x7ff2e4199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 >> 172.21.15.7:0/3982668995 conn(0x7ff2e40fbcb0 msgr2=0x7ff2e40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 shutdown_connections 2024-03-19T00:44:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.577+0000 7ff2eb8dc640 1 -- 172.21.15.7:0/3982668995 wait complete. 2024-03-19T00:44:26.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:44:26.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 -- 172.21.15.7:0/2723147851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 msgr2=0x7fb754100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 --2- 172.21.15.7:0/2723147851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb754100da0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb744009900 tx=0x7fb74402bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 -- 172.21.15.7:0/2723147851 shutdown_connections 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 --2- 172.21.15.7:0/2723147851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7541012e0 0x7fb754108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 --2- 172.21.15.7:0/2723147851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb754100da0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 --2- 172.21.15.7:0/2723147851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb754101cf0 0x7fb754102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 -- 172.21.15.7:0/2723147851 >> 172.21.15.7:0/2723147851 conn(0x7fb7540fb430 msgr2=0x7fb7540fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 -- 172.21.15.7:0/2723147851 shutdown_connections 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 -- 172.21.15.7:0/2723147851 wait complete. 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 Processor -- start 2024-03-19T00:44:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.733+0000 7fb75b790640 1 -- start start 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb7541046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7541012e0 0x7fb754102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb754101cf0 0x7fb754103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410c260 con 0x7fb754100940 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410c0e0 con 0x7fb7541012e0 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410c3e0 con 0x7fb754101cf0 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb7541046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb7541046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52946/0 (socket says 172.21.15.7:52946) 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 -- 172.21.15.7:0/2822830847 learned_addr learned my addr 172.21.15.7:0/2822830847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb759d2d640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb754101cf0 0x7fb754103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb758d2b640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7541012e0 0x7fb754102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 -- 172.21.15.7:0/2822830847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb754101cf0 msgr2=0x7fb754103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb754101cf0 0x7fb754103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 -- 172.21.15.7:0/2822830847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7541012e0 msgr2=0x7fb754102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7541012e0 0x7fb754102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 -- 172.21.15.7:0/2822830847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb754103b00 con 0x7fb754100940 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75952c640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb7541046f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e9f0 tx=0x7fb75000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb74a7fc640 1 -- 172.21.15.7:0/2822830847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000cd80 con 0x7fb754100940 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7541a1bb0 con 0x7fb754100940 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb74a7fc640 1 -- 172.21.15.7:0/2822830847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7500108e0 con 0x7fb754100940 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb74a7fc640 1 -- 172.21.15.7:0/2822830847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750018920 con 0x7fb754100940 2024-03-19T00:44:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.734+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7541a20d0 con 0x7fb754100940 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb74a7fc640 1 -- 172.21.15.7:0/2822830847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb750010a90 con 0x7fb754100940 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb75e378740 1 Processor -- start 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb75e378740 1 -- start start 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb75e378740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb75e378740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7f440 0x556733d83f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb75e378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d86a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.737+0000 7fb74a7fc640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7400a2800 0x7fb7400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75e378740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556733ce3500 con 0x556733d7f440 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75e378740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556733c8bbe0 con 0x556733d7ea50 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75e378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556733cdf5a0 con 0x556733d845b0 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb758d2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d86a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7f440 0x556733d83f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb758d2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d86a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:51838/0 (socket says 172.21.15.7:51838) 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb758d2b640 1 -- 172.21.15.7:0/788146799 learned_addr learned my addr 172.21.15.7:0/788146799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb74a7fc640 1 -- 172.21.15.7:0/2822830847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7500cf0b0 con 0x7fb754100940 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb759d2d640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7400a2800 0x7fb7400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 -- 172.21.15.7:0/788146799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 msgr2=0x556733d86a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d86a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 -- 172.21.15.7:0/788146799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 msgr2=0x556733d7ee70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d7ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 -- 172.21.15.7:0/788146799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556733cad000 con 0x556733d7f440 2024-03-19T00:44:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb759d2d640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7400a2800 0x7fb7400a4cc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005fd0 tx=0x7fb74c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.738+0000 7fb75952c640 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7f440 0x556733d83f40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb75000c6a0 tx=0x7fb75009b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5101b79c4b75826e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb73f7fe640 1 -- 172.21.15.7:0/788146799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750010de0 con 0x556733d7f440 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb73f7fe640 1 -- 172.21.15.7:0/788146799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb75009b7a0 con 0x556733d7f440 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb73f7fe640 1 -- 172.21.15.7:0/788146799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750014c10 con 0x556733d7f440 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 -- 172.21.15.7:0/788146799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7f440 msgr2=0x556733d83f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7f440 0x556733d83f40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb75000c6a0 tx=0x7fb75009b670 comp rx=0 tx=0).stop 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 -- 172.21.15.7:0/788146799 shutdown_connections 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d86a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7f440 0x556733d83f40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 --2- 172.21.15.7:0/788146799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d7ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 -- 172.21.15.7:0/788146799 >> 172.21.15.7:0/788146799 conn(0x556733d79fc0 msgr2=0x556733d7a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 -- 172.21.15.7:0/788146799 shutdown_connections 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.739+0000 7fb75e378740 1 -- 172.21.15.7:0/788146799 wait complete. 2024-03-19T00:44:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 -- start start 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 Processor -- start 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d88e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d90400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556733d87500 con 0x556733d7ea50 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556733c9f160 con 0x556733d7f440 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556733d87680 con 0x556733d845b0 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d88e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75e378740 10 client.0 _ll_register_callbacks cb 0x556733cdba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d88e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52974/0 (socket says 172.21.15.7:52974) 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75952c640 1 -- 172.21.15.7:0/3090928783 learned_addr learned my addr 172.21.15.7:0/3090928783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb758d2b640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb759d2d640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d90400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75952c640 1 -- 172.21.15.7:0/3090928783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 msgr2=0x556733d90400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75952c640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d90400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.740+0000 7fb75952c640 1 -- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 msgr2=0x556733d893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75952c640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75952c640 1 -- 172.21.15.7:0/3090928783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556733ac17d0 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75952c640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d88e80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb75000cfd0 tx=0x7fb750010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3528238767c9557e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb73cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb73cff9640 1 -- 172.21.15.7:0/3090928783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500a3e90 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb73cff9640 1 -- 172.21.15.7:0/3090928783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb750014e20 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb75009b7a0 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb73cff9640 1 -- 172.21.15.7:0/3090928783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb750010430 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 10 client.16896 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567339f73f0 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb73cff9640 1 -- 172.21.15.7:0/3090928783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb75009b7a0 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb73cff9640 1 -- 172.21.15.7:0/3090928783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb7500976f0 con 0x556733d7ea50 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 10 client.16896 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 20 client.16896 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733d948f0 0x556733d96ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 4 client.16896 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.741+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556733cf0a40 con 0x556733d948f0 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.742+0000 7fb758d2b640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733d948f0 0x556733d96ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.742+0000 7fb73cff9640 10 client.16896 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.742+0000 7fb758d2b640 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733d948f0 0x556733d96ce0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecceb17238c1e43 server_cookie=97477dfb502483eb in_seq=0 out_seq=0 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb73cff9640 1 -- 172.21.15.7:0/3090928783 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556733cf0a40 con 0x556733d948f0 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb73cff9640 10 client.16896 handle_command_reply: tid=0 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 client.16896 shutdown 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 msgr2=0x556733d88e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d88e80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb75000cfd0 tx=0x7fb750010de0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 shutdown_connections 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d90400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733d948f0 0x556733d96ce0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 --2- 172.21.15.7:0/3090928783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d88e80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 >> 172.21.15.7:0/3090928783 conn(0x556733d79fc0 msgr2=0x556733d791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb73cff9640 0 client.16896 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 shutdown_connections 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 1 -- 172.21.15.7:0/3090928783 wait complete. 2024-03-19T00:44:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.743+0000 7fb75e378740 20 client.16896 trim_cache size 0 max 16384 2024-03-19T00:44:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.760+0000 7fb75e378740 1 Processor -- start 2024-03-19T00:44:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.760+0000 7fb75e378740 1 -- start start 2024-03-19T00:44:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.760+0000 7fb75e378740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.760+0000 7fb75e378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d7f440 0x556733d903c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.760+0000 7fb75e378740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 0x556733d78950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb75e378740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541aee30 con 0x556733d845b0 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb75e378740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541ac700 con 0x556733d7ea50 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb75e378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541aa550 con 0x556733d7f440 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 0x556733d78950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 0x556733d78950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52988/0 (socket says 172.21.15.7:52988) 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 -- 172.21.15.7:0/1345478205 learned_addr learned my addr 172.21.15.7:0/1345478205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb75952c640 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb758d2b640 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d7f440 0x556733d903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 -- 172.21.15.7:0/1345478205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d7f440 msgr2=0x556733d903c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d7f440 0x556733d903c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 -- 172.21.15.7:0/1345478205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 msgr2=0x556733d891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 -- 172.21.15.7:0/1345478205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567339b70f0 con 0x556733d845b0 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb75952c640 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:44:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb759d2d640 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 0x556733d78950 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009ad0 tx=0x7fb74c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa9a21314fb53de server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb73effd640 1 -- 172.21.15.7:0/1345478205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c003970 con 0x556733d845b0 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb73effd640 1 -- 172.21.15.7:0/1345478205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb74c004030 con 0x556733d845b0 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.761+0000 7fb73effd640 1 -- 172.21.15.7:0/1345478205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c01d6a0 con 0x556733d845b0 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 -- 172.21.15.7:0/1345478205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 msgr2=0x556733d78950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 0x556733d78950 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009ad0 tx=0x7fb74c014000 comp rx=0 tx=0).stop 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 -- 172.21.15.7:0/1345478205 shutdown_connections 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d845b0 0x556733d78950 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d7f440 0x556733d903c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 --2- 172.21.15.7:0/1345478205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7ea50 0x556733d891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 -- 172.21.15.7:0/1345478205 >> 172.21.15.7:0/1345478205 conn(0x556733d79fc0 msgr2=0x556733d88a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 -- 172.21.15.7:0/1345478205 shutdown_connections 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 -- 172.21.15.7:0/1345478205 wait complete. 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 -- start start 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.762+0000 7fb75e378740 1 Processor -- start 2024-03-19T00:44:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d8d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541aabc0 con 0x556733d7ea50 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541ac030 con 0x556733d7f440 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541ab290 con 0x556733d845b0 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 10 client.0 _ll_register_callbacks cb 0x7fb74c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d8d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d8d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52990/0 (socket says 172.21.15.7:52990) 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 -- 172.21.15.7:0/2232023101 learned_addr learned my addr 172.21.15.7:0/2232023101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb758d2b640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb759d2d640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 -- 172.21.15.7:0/2232023101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 msgr2=0x556733d8ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d8ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 -- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 msgr2=0x556733d8da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d8da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 -- 172.21.15.7:0/2232023101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556733d8f020 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb758d2b640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75952c640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d8d4e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7500afb20 tx=0x7fb750097400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8474a02277aa44e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb73cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb73cff9640 1 -- 172.21.15.7:0/2232023101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75009b7a0 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb73cff9640 1 -- 172.21.15.7:0/2232023101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7500afba0 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.763+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74c004030 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb73cff9640 1 -- 172.21.15.7:0/2232023101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7500afda0 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 10 client.16902 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556733d903a0 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb73cff9640 1 -- 172.21.15.7:0/2232023101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb74c004030 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb73cff9640 1 -- 172.21.15.7:0/2232023101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb7500974f0 con 0x556733d7ea50 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 10 client.16902 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 20 client.16902 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733dfac40 0x556733dfd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 4 client.16902 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556733cead90 con 0x556733dfac40 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb758d2b640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733dfac40 0x556733dfd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb73cff9640 10 client.16902 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.764+0000 7fb758d2b640 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733dfac40 0x556733dfd030 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903d6c5d20374961 server_cookie=17c701ad35629eb2 in_seq=0 out_seq=0 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb73cff9640 1 -- 172.21.15.7:0/2232023101 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556733cead90 con 0x556733dfac40 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb73cff9640 10 client.16902 handle_command_reply: tid=0 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 client.16902 shutdown 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 msgr2=0x556733d8d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d8d4e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7500afb20 tx=0x7fb750097400 comp rx=0 tx=0).stop 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 shutdown_connections 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556733d845b0 0x556733d8ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556733d7f440 0x556733d8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556733dfac40 0x556733dfd030 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 --2- 172.21.15.7:0/2232023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556733d7ea50 0x556733d8d4e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb73cff9640 0 client.16902 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 >> 172.21.15.7:0/2232023101 conn(0x556733d79fc0 msgr2=0x556733d7d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 shutdown_connections 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 1 -- 172.21.15.7:0/2232023101 wait complete. 2024-03-19T00:44:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75e378740 20 client.16902 trim_cache size 0 max 16384 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "651fb1f3-7d03-4a2f-b8ac-f8f2f7251b3b": { 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "651fb1f3-7d03-4a2f-b8ac-f8f2f7251b3b", 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:44:26.774 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:44:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7400a2800 msgr2=0x7fb7400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7400a2800 0x7fb7400a4cc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005fd0 tx=0x7fb74c009500 comp rx=0 tx=0).stop 2024-03-19T00:44:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 msgr2=0x7fb7541046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb7541046f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e9f0 tx=0x7fb75000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 shutdown_connections 2024-03-19T00:44:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7400a2800 0x7fb7400a4cc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb754101cf0 0x7fb754103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7541012e0 0x7fb754102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 --2- 172.21.15.7:0/2822830847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100940 0x7fb7541046f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.765+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 >> 172.21.15.7:0/2822830847 conn(0x7fb7540fb430 msgr2=0x7fb7540fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.766+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 shutdown_connections 2024-03-19T00:44:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:26.766+0000 7fb75b790640 1 -- 172.21.15.7:0/2822830847 wait complete. 2024-03-19T00:44:26.777 INFO:tasks.cephfs.filesystem:scrub status for tag:651fb1f3-7d03-4a2f-b8ac-f8f2f7251b3b - {'path': '/', 'tag': '651fb1f3-7d03-4a2f-b8ac-f8f2f7251b3b', 'options': 'recursive,force'} 2024-03-19T00:44:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: pgmap v546: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:44:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:44:28.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:44:28.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:28 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: pgmap v546: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:28 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: pgmap v546: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:28.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:28 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:30.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:30 smithi044 ceph-mon[66366]: pgmap v547: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:30.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:30 smithi007 ceph-mon[62854]: pgmap v547: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:30.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:30 smithi204 ceph-mon[66572]: pgmap v547: 129 pgs: 129 active+clean; 1.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:44:32.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:32 smithi044 ceph-mon[66366]: pgmap v548: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:32.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:32 smithi007 ceph-mon[62854]: pgmap v548: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:32.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:32 smithi204 ceph-mon[66572]: pgmap v548: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:44:34.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:34 smithi044 ceph-mon[66366]: pgmap v549: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:34.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:34 smithi007 ceph-mon[62854]: pgmap v549: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:34.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:34 smithi204 ceph-mon[66572]: pgmap v549: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:36.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:36 smithi044 ceph-mon[66366]: pgmap v550: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:44:36.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:36 smithi007 ceph-mon[62854]: pgmap v550: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:44:36.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:36 smithi204 ceph-mon[66572]: pgmap v550: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:44:37.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:37 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:44:37.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:37 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:44:37.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:37 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:44:37.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:37 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:44:37.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:37 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:44:37.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:37 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:44:38.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:38 smithi044 ceph-mon[66366]: pgmap v551: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:38.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:38 smithi007 ceph-mon[62854]: pgmap v551: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:38.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:38 smithi204 ceph-mon[66572]: pgmap v551: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:40 smithi044 ceph-mon[66366]: pgmap v552: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:40.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:40 smithi007 ceph-mon[62854]: pgmap v552: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:40.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:40 smithi204 ceph-mon[66572]: pgmap v552: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:44:42.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:42 smithi204 ceph-mon[66572]: pgmap v553: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:44:42.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:42 smithi044 ceph-mon[66366]: pgmap v553: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:44:42.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:42 smithi007 ceph-mon[62854]: pgmap v553: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:44:44.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:44 smithi044 ceph-mon[66366]: pgmap v554: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:44.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:44 smithi007 ceph-mon[62854]: pgmap v554: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:44.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:44 smithi204 ceph-mon[66572]: pgmap v554: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:46 smithi044 ceph-mon[66366]: pgmap v555: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:44:46.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:46 smithi007 ceph-mon[62854]: pgmap v555: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:44:46.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:46 smithi204 ceph-mon[66572]: pgmap v555: 129 pgs: 129 active+clean; 2.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:44:47.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:44:47.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:44:47.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:44:47.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:44:47.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:44:47.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:44:47.516 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:44:48.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:48 smithi044 ceph-mon[66366]: pgmap v556: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:48.408 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:48 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-03-19T00:44:48.408 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:48 smithi044 ceph-mon[66366]: Unable to set osd_memory_target on smithi007 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-19T00:44:48.408 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:48 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-03-19T00:44:48.408 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:48 smithi044 ceph-mon[66366]: Unable to set osd_memory_target on smithi044 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-19T00:44:48.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:48 smithi007 ceph-mon[62854]: pgmap v556: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:48.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:48 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-03-19T00:44:48.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:48 smithi007 ceph-mon[62854]: Unable to set osd_memory_target on smithi007 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-19T00:44:48.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:48 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-03-19T00:44:48.469 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:48 smithi007 ceph-mon[62854]: Unable to set osd_memory_target on smithi044 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-19T00:44:48.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:48 smithi204 ceph-mon[66572]: pgmap v556: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:48.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:48 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-03-19T00:44:48.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:48 smithi204 ceph-mon[66572]: Unable to set osd_memory_target on smithi007 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-19T00:44:48.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:48 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-03-19T00:44:48.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:48 smithi204 ceph-mon[66572]: Unable to set osd_memory_target on smithi044 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-19T00:44:49.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:49 smithi044 ceph-mon[66366]: pgmap v557: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:44:49.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:49 smithi007 ceph-mon[62854]: pgmap v557: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:44:49.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:49 smithi204 ceph-mon[66572]: pgmap v557: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:44:51.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:51 smithi007 ceph-mon[62854]: pgmap v558: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:44:51.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:51 smithi204 ceph-mon[66572]: pgmap v558: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:44:51.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:51 smithi044 ceph-mon[66366]: pgmap v558: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:44:53.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:53 smithi204 ceph-mon[66572]: pgmap v559: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:53.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:53 smithi044 ceph-mon[66366]: pgmap v559: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:53.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:53 smithi007 ceph-mon[62854]: pgmap v559: 129 pgs: 129 active+clean; 2.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:55.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:55 smithi204 ceph-mon[66572]: pgmap v560: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:44:55.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:55 smithi044 ceph-mon[66366]: pgmap v560: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:44:55.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:55 smithi007 ceph-mon[62854]: pgmap v560: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:44:56.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:44:56.778 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 -- 172.21.15.7:0/2813357871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5844100740 msgr2=0x7f5844100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 --2- 172.21.15.7:0/2813357871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5844100740 0x7f5844100b60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5834009900 tx=0x7f583402bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 -- 172.21.15.7:0/2813357871 shutdown_connections 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 --2- 172.21.15.7:0/2813357871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58441022f0 0x7f5844108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 --2- 172.21.15.7:0/2813357871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 0x7f5844101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 --2- 172.21.15.7:0/2813357871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5844100740 0x7f5844100b60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.894+0000 7f58496c0640 1 -- 172.21.15.7:0/2813357871 >> 172.21.15.7:0/2813357871 conn(0x7f58440fbcb0 msgr2=0x7f58440fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 -- 172.21.15.7:0/2813357871 shutdown_connections 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 -- 172.21.15.7:0/2813357871 wait complete. 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 Processor -- start 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 -- start start 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5844100740 0x7f5844197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 0x7f5844198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58441022f0 0x7f584419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f584410c240 con 0x7f58441022f0 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f584410c0c0 con 0x7f5844101930 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58496c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f584410c3c0 con 0x7f5844100740 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58441022f0 0x7f584419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58441022f0 0x7f584419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60390/0 (socket says 172.21.15.7:60390) 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f583bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 0x7f5844198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f58437fe640 1 -- 172.21.15.7:0/2547238237 learned_addr learned my addr 172.21.15.7:0/2547238237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.895+0000 7f5842ffd640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5844100740 0x7f5844197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f583bfff640 1 -- 172.21.15.7:0/2547238237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5844100740 msgr2=0x7f5844197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f583bfff640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5844100740 0x7f5844197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f583bfff640 1 -- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58441022f0 msgr2=0x7f584419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f583bfff640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58441022f0 0x7f584419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f583bfff640 1 -- 172.21.15.7:0/2547238237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584419f710 con 0x7f5844101930 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f583bfff640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 0x7f5844198030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f582c00e9f0 tx=0x7f582c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f5840ff9640 1 -- 172.21.15.7:0/2547238237 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c00cd80 con 0x7f5844101930 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584419fa00 con 0x7f5844101930 2024-03-19T00:44:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f5840ff9640 1 -- 172.21.15.7:0/2547238237 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c0035f0 con 0x7f5844101930 2024-03-19T00:44:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f5840ff9640 1 -- 172.21.15.7:0/2547238237 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c010690 con 0x7f5844101930 2024-03-19T00:44:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.896+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584419ffa0 con 0x7f5844101930 2024-03-19T00:44:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.898+0000 7f5840ff9640 1 -- 172.21.15.7:0/2547238237 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f582c0037a0 con 0x7f5844101930 2024-03-19T00:44:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 Processor -- start 2024-03-19T00:44:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 -- start start 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ade60 0x5581e04ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04b39c0 0x5581e04b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5581e040e500 con 0x5581e04ae850 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5581e03b6be0 con 0x5581e04b39c0 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f584c387740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5581e040a5a0 con 0x5581e04ade60 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60392/0 (socket says 172.21.15.7:60392) 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 -- 172.21.15.7:0/1362847569 learned_addr learned my addr 172.21.15.7:0/1362847569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f583bfff640 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04b39c0 0x5581e04b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f58437fe640 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ade60 0x5581e04ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5840ff9640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58300a2800 0x7f58300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 -- 172.21.15.7:0/1362847569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ade60 msgr2=0x5581e04ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ade60 0x5581e04ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 -- 172.21.15.7:0/1362847569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04b39c0 msgr2=0x5581e04b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.899+0000 7f5842ffd640 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04b39c0 0x5581e04b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f5842ffd640 1 -- 172.21.15.7:0/1362847569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e03d8000 con 0x5581e04ae850 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f58437fe640 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ade60 0x5581e04ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f58437fe640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58300a2800 0x7f58300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f5840ff9640 1 -- 172.21.15.7:0/2547238237 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f582c0cf0e0 con 0x7f5844101930 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f5842ffd640 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04b3350 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5834009900 tx=0x7f5834009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fa442ae33d2db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f58397fa640 1 -- 172.21.15.7:0/1362847569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834045020 con 0x5581e04ae850 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f58437fe640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58300a2800 0x7f58300a4cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f583c009fd0 tx=0x7f583c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f58397fa640 1 -- 172.21.15.7:0/1362847569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5834004040 con 0x5581e04ae850 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.900+0000 7f58397fa640 1 -- 172.21.15.7:0/1362847569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834005570 con 0x5581e04ae850 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 -- 172.21.15.7:0/1362847569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 msgr2=0x5581e04b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04b3350 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5834009900 tx=0x7f5834009560 comp rx=0 tx=0).stop 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 -- 172.21.15.7:0/1362847569 shutdown_connections 2024-03-19T00:44:56.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04b39c0 0x5581e04b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04b3350 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 --2- 172.21.15.7:0/1362847569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ade60 0x5581e04ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 -- 172.21.15.7:0/1362847569 >> 172.21.15.7:0/1362847569 conn(0x5581e04a4fc0 msgr2=0x5581e04a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 -- 172.21.15.7:0/1362847569 shutdown_connections 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 -- 172.21.15.7:0/1362847569 wait complete. 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 -- start start 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 Processor -- start 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.901+0000 7f584c387740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f584c387740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f584c387740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5581e04b6910 con 0x5581e04b39c0 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f584c387740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5581e03ca160 con 0x5581e04ade60 2024-03-19T00:44:56.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f584c387740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5581e04b6a90 con 0x5581e04ae850 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f584c387740 10 client.0 _ll_register_callbacks cb 0x5581e0406a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60400/0 (socket says 172.21.15.7:60400) 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 -- 172.21.15.7:0/4125121865 learned_addr learned my addr 172.21.15.7:0/4125121865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04b8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48412/0 (socket says 172.21.15.7:48412) 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f583bfff640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 -- 172.21.15.7:0/4125121865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 msgr2=0x5581e04b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 -- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 msgr2=0x5581e04b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 -- 172.21.15.7:0/4125121865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e01ec7d0 con 0x5581e04b39c0 2024-03-19T00:44:56.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f5842ffd640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f58437fe640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bf7d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f583c00faf0 tx=0x7f583c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2867698498e3fa8b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f582a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f582a7fc640 1 -- 172.21.15.7:0/4125121865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c010940 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f582a7fc640 1 -- 172.21.15.7:0/4125121865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f583c010ae0 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.902+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5834004040 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f582a7fc640 1 -- 172.21.15.7:0/4125121865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f583c010ce0 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 10 client.16914 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581e01223f0 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f582a7fc640 1 -- 172.21.15.7:0/4125121865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5834004040 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f582a7fc640 1 -- 172.21.15.7:0/4125121865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f583c025070 con 0x5581e04b39c0 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 10 client.16914 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 20 client.16914 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e04c4150 0x5581e04c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 4 client.16914 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581e041ba40 con 0x5581e04c4150 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.903+0000 7f5842ffd640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e04c4150 0x5581e04c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f5842ffd640 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e04c4150 0x5581e04c6610 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f430fe5e28576dc9 server_cookie=222ab45fbaa5a873 in_seq=0 out_seq=0 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f582a7fc640 10 client.16914 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f582a7fc640 1 -- 172.21.15.7:0/4125121865 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5581e041ba40 con 0x5581e04c4150 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f582a7fc640 10 client.16914 handle_command_reply: tid=0 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 client.16914 shutdown 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 msgr2=0x5581e04bf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bf7d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f583c00faf0 tx=0x7f583c004ab0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 shutdown_connections 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e04c4150 0x5581e04c6610 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bf7d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f582a7fc640 0 client.16914 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:56.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 --2- 172.21.15.7:0/4125121865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.904+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 >> 172.21.15.7:0/4125121865 conn(0x5581e04a4fc0 msgr2=0x5581e04a41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:56.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.905+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 shutdown_connections 2024-03-19T00:44:56.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.905+0000 7f584c387740 1 -- 172.21.15.7:0/4125121865 wait complete. 2024-03-19T00:44:56.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.905+0000 7f584c387740 20 client.16914 trim_cache size 0 max 16384 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 Processor -- start 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 -- start start 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04b39c0 0x5581e04caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58441aaa00 con 0x5581e04ae850 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58441a82d0 con 0x5581e04ade60 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f584c387740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58441a6120 con 0x5581e04b39c0 2024-03-19T00:44:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.922+0000 7f583bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04bf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60406/0 (socket says 172.21.15.7:60406) 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f58437fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04b39c0 0x5581e04caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48428/0 (socket says 172.21.15.7:48428) 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 -- 172.21.15.7:0/1305378238 learned_addr learned my addr 172.21.15.7:0/1305378238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 -- 172.21.15.7:0/1305378238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04b39c0 msgr2=0x5581e04caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04b39c0 0x5581e04caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 -- 172.21.15.7:0/1305378238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 msgr2=0x5581e04a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 -- 172.21.15.7:0/1305378238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e00e20f0 con 0x5581e04ae850 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f5842ffd640 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f583bfff640 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04bf790 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c6a0 tx=0x7f582c097840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f71e9f7c1be2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f5838ff9640 1 -- 172.21.15.7:0/1305378238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c09b020 con 0x5581e04ae850 2024-03-19T00:44:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f5838ff9640 1 -- 172.21.15.7:0/1305378238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c003b30 con 0x5581e04ae850 2024-03-19T00:44:56.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.923+0000 7f5838ff9640 1 -- 172.21.15.7:0/1305378238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c009310 con 0x5581e04ae850 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- 172.21.15.7:0/1305378238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 msgr2=0x5581e04bf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04bf790 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c6a0 tx=0x7f582c097840 comp rx=0 tx=0).stop 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- 172.21.15.7:0/1305378238 shutdown_connections 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04b39c0 0x5581e04caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04ae850 0x5581e04bf790 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- 172.21.15.7:0/1305378238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- 172.21.15.7:0/1305378238 >> 172.21.15.7:0/1305378238 conn(0x5581e04a4fc0 msgr2=0x5581e04a3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- 172.21.15.7:0/1305378238 shutdown_connections 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- 172.21.15.7:0/1305378238 wait complete. 2024-03-19T00:44:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- start start 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 Processor -- start 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04bdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58441a6790 con 0x5581e04b39c0 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58441a7c00 con 0x5581e04ade60 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.924+0000 7f584c387740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58441a6e60 con 0x5581e04ae850 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f584c387740 10 client.0 _ll_register_callbacks cb 0x7f583c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f5842ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48432/0 (socket says 172.21.15.7:48432) 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f5842ffd640 1 -- 172.21.15.7:0/3289730759 learned_addr learned my addr 172.21.15.7:0/3289730759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f583bfff640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04bdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 -- 172.21.15.7:0/3289730759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 msgr2=0x5581e04bdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04bdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 -- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 msgr2=0x5581e04bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 -- 172.21.15.7:0/3289730759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e04bf1f0 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f5842ffd640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f58437fe640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bec00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f583c00ffd0 tx=0x7f583c027960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9ade906ea9fff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f582a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f582a7fc640 1 -- 172.21.15.7:0/3289730759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c03a060 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f582a7fc640 1 -- 172.21.15.7:0/3289730759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f583c035440 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581e0415d90 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f582a7fc640 1 -- 172.21.15.7:0/3289730759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f583c035640 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f584c387740 10 client.16920 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.925+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581e052ad30 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f582a7fc640 1 -- 172.21.15.7:0/3289730759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5581e0415d90 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f582a7fc640 1 -- 172.21.15.7:0/3289730759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f583c025050 con 0x5581e04b39c0 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 10 client.16920 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 20 client.16920 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e052b700 0x5581e052db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 4 client.16920 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5581e0393540 con 0x5581e052b700 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f5842ffd640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e052b700 0x5581e052db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f582a7fc640 10 client.16920 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f5842ffd640 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e052b700 0x5581e052db20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bdbf09b79ed203 server_cookie=e2304a3f17f76d5c in_seq=0 out_seq=0 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f582a7fc640 1 -- 172.21.15.7:0/3289730759 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5581e0393540 con 0x5581e052b700 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f582a7fc640 10 client.16920 handle_command_reply: tid=0 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 1 client.16920 shutdown 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 msgr2=0x5581e04bec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.926+0000 7f584c387740 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bec00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f583c00ffd0 tx=0x7f583c027960 comp rx=0 tx=0).stop 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 shutdown_connections 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5581e052b700 0x5581e052db20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5581e04b39c0 0x5581e04bec00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5581e04ae850 0x5581e04bdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f582a7fc640 0 client.16920 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 --2- 172.21.15.7:0/3289730759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5581e04ade60 0x5581e04bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 >> 172.21.15.7:0/3289730759 conn(0x5581e04a4fc0 msgr2=0x5581e04a8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 shutdown_connections 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 1 -- 172.21.15.7:0/3289730759 wait complete. 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f584c387740 20 client.16920 trim_cache size 0 max 16384 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:44:56.934 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58300a2800 msgr2=0x7f58300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58300a2800 0x7f58300a4cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f583c009fd0 tx=0x7f583c004a40 comp rx=0 tx=0).stop 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 msgr2=0x7f5844198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 0x7f5844198030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f582c00e9f0 tx=0x7f582c00eeb0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 shutdown_connections 2024-03-19T00:44:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f58300a2800 0x7f58300a4cc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58441022f0 0x7f584419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5844101930 0x7f5844198030 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 --2- 172.21.15.7:0/2547238237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5844100740 0x7f5844197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 >> 172.21.15.7:0/2547238237 conn(0x7f58440fbcb0 msgr2=0x7f5844068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 shutdown_connections 2024-03-19T00:44:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:56.927+0000 7f58496c0640 1 -- 172.21.15.7:0/2547238237 wait complete. 2024-03-19T00:44:56.939 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:44:56.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:44:57.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 -- 172.21.15.7:0/3221886937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c100750 msgr2=0x7fa01c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 --2- 172.21.15.7:0/3221886937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c100750 0x7fa01c100b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa014009900 tx=0x7fa01402bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:57.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 -- 172.21.15.7:0/3221886937 shutdown_connections 2024-03-19T00:44:57.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 --2- 172.21.15.7:0/3221886937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 --2- 172.21.15.7:0/3221886937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c101940 0x7fa01c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 --2- 172.21.15.7:0/3221886937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c100750 0x7fa01c100b70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 -- 172.21.15.7:0/3221886937 >> 172.21.15.7:0/3221886937 conn(0x7fa01c0fbce0 msgr2=0x7fa01c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 -- 172.21.15.7:0/3221886937 shutdown_connections 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 -- 172.21.15.7:0/3221886937 wait complete. 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 Processor -- start 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.041+0000 7fa022d09640 1 -- start start 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c100750 0x7fa01c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c101940 0x7fa01c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10c270 con 0x7fa01c102300 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10c0f0 con 0x7fa01c101940 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10c3f0 con 0x7fa01c100750 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60432/0 (socket says 172.21.15.7:60432) 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 -- 172.21.15.7:0/1661666781 learned_addr learned my addr 172.21.15.7:0/1661666781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa020aa5640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c100750 0x7fa01c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa01bfff640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c101940 0x7fa01c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 -- 172.21.15.7:0/1661666781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c100750 msgr2=0x7fa01c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c100750 0x7fa01c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 -- 172.21.15.7:0/1661666781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c101940 msgr2=0x7fa01c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c101940 0x7fa01c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 -- 172.21.15.7:0/1661666781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c19f720 con 0x7fa01c102300 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa020aa5640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c100750 0x7fa01c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa0212a6640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c19f0b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e970 tx=0x7fa00400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00400ccc0 con 0x7fa01c102300 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01c19fa10 con 0x7fa01c102300 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa00400ce60 con 0x7fa01c102300 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.042+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004010430 con 0x7fa01c102300 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.043+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01c19ff60 con 0x7fa01c102300 2024-03-19T00:44:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.043+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01c1a01f0 con 0x7fa01c102300 2024-03-19T00:44:57.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.045+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa004004fb0 con 0x7fa01c102300 2024-03-19T00:44:57.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.046+0000 7fa019ffb640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0080a2800 0x7fa0080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.046+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa0040cf430 con 0x7fa01c102300 2024-03-19T00:44:57.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.046+0000 7fa020aa5640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0080a2800 0x7fa0080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.046+0000 7fa020aa5640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0080a2800 0x7fa0080a4cc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa014005ab0 tx=0x7fa014005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.050+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa004053740 con 0x7fa01c102300 2024-03-19T00:44:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.275+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa01c1a03c0 con 0x7fa01c102300 2024-03-19T00:44:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.276+0000 7fa019ffb640 1 -- 172.21.15.7:0/1661666781 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fa00409c110 con 0x7fa01c102300 2024-03-19T00:44:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:44:57.277 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:44:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:44:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0080a2800 msgr2=0x7fa0080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0080a2800 0x7fa0080a4cc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa014005ab0 tx=0x7fa014005fd0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 msgr2=0x7fa01c19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c19f0b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa00400e970 tx=0x7fa00400ed70 comp rx=0 tx=0).stop 2024-03-19T00:44:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 shutdown_connections 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0080a2800 0x7fa0080a4cc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa01c102300 0x7fa01c19f0b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa01c101940 0x7fa01c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 --2- 172.21.15.7:0/1661666781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa01c100750 0x7fa01c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 >> 172.21.15.7:0/1661666781 conn(0x7fa01c0fbce0 msgr2=0x7fa01c069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 shutdown_connections 2024-03-19T00:44:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.278+0000 7fa022d09640 1 -- 172.21.15.7:0/1661666781 wait complete. 2024-03-19T00:44:57.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:44:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 -- 172.21.15.7:0/2253503527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40102a10 msgr2=0x7f5e40102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2253503527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40102a10 0x7f5e40102e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009900 tx=0x7f5e3002bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 -- 172.21.15.7:0/2253503527 shutdown_connections 2024-03-19T00:44:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2253503527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e401033d0 0x7f5e40109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2253503527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40102a10 0x7f5e40102e90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2253503527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e40101820 0x7f5e40101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 -- 172.21.15.7:0/2253503527 >> 172.21.15.7:0/2253503527 conn(0x7f5e400fcd50 msgr2=0x7f5e400ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 -- 172.21.15.7:0/2253503527 shutdown_connections 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.393+0000 7f5e47f77640 1 -- 172.21.15.7:0/2253503527 wait complete. 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 Processor -- start 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 -- start start 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 0x7f5e40197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e40102a10 0x7f5e40197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e401033d0 0x7f5e4019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4010d350 con 0x7f5e40101820 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4010d1d0 con 0x7f5e40102a10 2024-03-19T00:44:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e47f77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4010d4d0 con 0x7f5e401033d0 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 0x7f5e40197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 0x7f5e40197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60442/0 (socket says 172.21.15.7:60442) 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 -- 172.21.15.7:0/2785804927 learned_addr learned my addr 172.21.15.7:0/2785804927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45512640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e40102a10 0x7f5e40197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e46514640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e401033d0 0x7f5e4019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 -- 172.21.15.7:0/2785804927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e401033d0 msgr2=0x7f5e4019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e401033d0 0x7f5e4019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 -- 172.21.15.7:0/2785804927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e40102a10 msgr2=0x7f5e40197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e40102a10 0x7f5e40197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e45d13640 1 -- 172.21.15.7:0/2785804927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4019f690 con 0x7f5e40101820 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.394+0000 7f5e46514640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e401033d0 0x7f5e4019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.395+0000 7f5e45d13640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 0x7f5e40197a70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00e9c0 tx=0x7f5e3c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.395+0000 7f5e36ffd640 1 -- 172.21.15.7:0/2785804927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00cd80 con 0x7f5e40101820 2024-03-19T00:44:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.395+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4019f980 con 0x7f5e40101820 2024-03-19T00:44:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.395+0000 7f5e36ffd640 1 -- 172.21.15.7:0/2785804927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e3c0035f0 con 0x7f5e40101820 2024-03-19T00:44:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.395+0000 7f5e36ffd640 1 -- 172.21.15.7:0/2785804927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c010690 con 0x7f5e40101820 2024-03-19T00:44:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.395+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4019fe50 con 0x7f5e40101820 2024-03-19T00:44:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.397+0000 7f5e36ffd640 1 -- 172.21.15.7:0/2785804927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5e3c0037a0 con 0x7f5e40101820 2024-03-19T00:44:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.397+0000 7f5e4a7fd740 1 Processor -- start 2024-03-19T00:44:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.397+0000 7f5e4a7fd740 1 -- start start 2024-03-19T00:44:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e2331710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e23392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578e22954b0 con 0x5578e2331ce0 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5578e223dd70 con 0x5578e23312f0 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578e2291550 con 0x5578e2336e50 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e36ffd640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e2c0a2800 0x7f5e2c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23367e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60446/0 (socket says 172.21.15.7:60446) 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 -- 172.21.15.7:0/2902042301 learned_addr learned my addr 172.21.15.7:0/2902042301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e46514640 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e2331710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45512640 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e23392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e46514640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e2c0a2800 0x7f5e2c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e36ffd640 1 -- 172.21.15.7:0/2785804927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e3c0cf3d0 con 0x7f5e40101820 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 -- 172.21.15.7:0/2902042301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 msgr2=0x5578e23392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e23392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 -- 172.21.15.7:0/2902042301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 msgr2=0x5578e2331710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e2331710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 -- 172.21.15.7:0/2902042301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e2260210 con 0x5578e2331ce0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45512640 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e23392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e45d13640 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23367e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0a3c20 tx=0x7f5e3c09b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4e7d7744e4c7ae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e46514640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e2c0a2800 0x7f5e2c0a4cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5578e2332540 tx=0x7f5e38009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.398+0000 7f5e2bfff640 1 -- 172.21.15.7:0/2902042301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c09c0b0 con 0x5578e2331ce0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e2bfff640 1 -- 172.21.15.7:0/2902042301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e3c09b9b0 con 0x5578e2331ce0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e2bfff640 1 -- 172.21.15.7:0/2902042301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0a3cc0 con 0x5578e2331ce0 2024-03-19T00:44:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/2902042301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 msgr2=0x5578e23367e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23367e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0a3c20 tx=0x7f5e3c09b5b0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/2902042301 shutdown_connections 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e23392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23367e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/2902042301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e2331710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/2902042301 >> 172.21.15.7:0/2902042301 conn(0x5578e232c860 msgr2=0x5578e232cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/2902042301 shutdown_connections 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.399+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/2902042301 wait complete. 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 -- start start 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 Processor -- start 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e233b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e233bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e2342d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578e2339da0 con 0x5578e2331ce0 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5578e2251310 con 0x5578e23312f0 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578e2339f20 con 0x5578e2336e50 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e4a7fd740 10 client.0 _ll_register_callbacks cb 0x5578e22452d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e233bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e233bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60452/0 (socket says 172.21.15.7:60452) 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 -- 172.21.15.7:0/627593697 learned_addr learned my addr 172.21.15.7:0/627593697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45d13640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e233b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e46514640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e2342d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 -- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 msgr2=0x5578e2342d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e2342d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 -- 172.21.15.7:0/627593697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 msgr2=0x5578e233b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e233b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e45512640 1 -- 172.21.15.7:0/627593697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e1fa93f0 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.400+0000 7f5e46514640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e2342d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e45512640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e233bce0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002c170 tx=0x7f5e30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a671690fa58e04e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e297fa640 1 -- 172.21.15.7:0/627593697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30005030 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e297fa640 1 -- 172.21.15.7:0/627593697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e300051d0 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3c09b9b0 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e297fa640 1 -- 172.21.15.7:0/627593697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e300053d0 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 10 client.16938 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578e20737d0 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e297fa640 1 -- 172.21.15.7:0/627593697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e3c09b9b0 con 0x5578e2331ce0 2024-03-19T00:44:57.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e297fa640 1 -- 172.21.15.7:0/627593697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5e30003710 con 0x5578e2331ce0 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 10 client.16938 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 10 client.16938 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 20 client.16938 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23475e0 0x5578e2349a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 4 client.16938 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.401+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578e22f33e0 con 0x5578e23475e0 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.402+0000 7f5e45d13640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23475e0 0x5578e2349a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.402+0000 7f5e45d13640 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23475e0 0x5578e2349a40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fabcc9cf15db57d0 server_cookie=fa8b61913f3b7bb1 in_seq=0 out_seq=0 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.402+0000 7f5e297fa640 10 client.16938 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e297fa640 1 -- 172.21.15.7:0/627593697 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5578e22f33e0 con 0x5578e23475e0 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e297fa640 10 client.16938 handle_command_reply: tid=0 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 client.16938 shutdown 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 msgr2=0x5578e233bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e233bce0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002c170 tx=0x7f5e30004270 comp rx=0 tx=0).stop 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 shutdown_connections 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23475e0 0x5578e2349a40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e2336e50 0x5578e2342d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e233bce0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e297fa640 0 client.16938 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/627593697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e23312f0 0x5578e233b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 >> 172.21.15.7:0/627593697 conn(0x5578e232c860 msgr2=0x5578e232bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 shutdown_connections 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/627593697 wait complete. 2024-03-19T00:44:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.403+0000 7f5e4a7fd740 20 client.16938 trim_cache size 0 max 16384 2024-03-19T00:44:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 Processor -- start 2024-03-19T00:44:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 -- start start 2024-03-19T00:44:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e23331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2342ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401acb20 con 0x5578e2331ce0 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401aa3f0 con 0x5578e2336e50 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401a8240 con 0x5578e23312f0 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60466/0 (socket says 172.21.15.7:60466) 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 -- 172.21.15.7:0/3975487127 learned_addr learned my addr 172.21.15.7:0/3975487127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45d13640 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e23331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e46514640 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2342ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 -- 172.21.15.7:0/3975487127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 msgr2=0x5578e23331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e23331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 -- 172.21.15.7:0/3975487127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 msgr2=0x5578e2342ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2342ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 -- 172.21.15.7:0/3975487127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e2332570 con 0x5578e2331ce0 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e45512640 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23336e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5e30000c00 tx=0x7f5e300058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da77752447a229b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e2b7fe640 1 -- 172.21.15.7:0/3975487127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30005570 con 0x5578e2331ce0 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e2b7fe640 1 -- 172.21.15.7:0/3975487127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e30005710 con 0x5578e2331ce0 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.417+0000 7f5e2b7fe640 1 -- 172.21.15.7:0/3975487127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30040b60 con 0x5578e2331ce0 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/3975487127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 msgr2=0x5578e23336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23336e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5e30000c00 tx=0x7f5e300058d0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/3975487127 shutdown_connections 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2342ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e23336e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/3975487127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e23331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/3975487127 >> 172.21.15.7:0/3975487127 conn(0x5578e232c860 msgr2=0x5578e233b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/3975487127 shutdown_connections 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/3975487127 wait complete. 2024-03-19T00:44:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 -- start start 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.418+0000 7f5e4a7fd740 1 Processor -- start 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e2340200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e2340740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2341700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401a88b0 con 0x5578e2331ce0 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401a9d20 con 0x5578e2336e50 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401a8f80 con 0x5578e23312f0 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e4a7fd740 10 client.0 _ll_register_callbacks cb 0x7f5e30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e2340740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e2340740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60482/0 (socket says 172.21.15.7:60482) 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 -- 172.21.15.7:0/311303846 learned_addr learned my addr 172.21.15.7:0/311303846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45d13640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e2340200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e46514640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2341700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 -- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 msgr2=0x5578e2340200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e2340200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 -- 172.21.15.7:0/311303846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 msgr2=0x5578e2341700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2341700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 -- 172.21.15.7:0/311303846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e2341cf0 con 0x5578e2331ce0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e45512640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e2340740 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5e30003b50 tx=0x7f5e30005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfd8a9b1f3802d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.419+0000 7f5e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e297fa640 1 -- 172.21.15.7:0/311303846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30005570 con 0x5578e2331ce0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e297fa640 1 -- 172.21.15.7:0/311303846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e30040af0 con 0x5578e2331ce0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578e22a29f0 con 0x5578e2331ce0 2024-03-19T00:44:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e297fa640 1 -- 172.21.15.7:0/311303846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5e30009520 con 0x5578e2331ce0 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 10 client.16944 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578e2342ff0 con 0x5578e2331ce0 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e297fa640 1 -- 172.21.15.7:0/311303846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5578e22a29f0 con 0x5578e2331ce0 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e297fa640 1 -- 172.21.15.7:0/311303846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5e30055aa0 con 0x5578e2331ce0 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 10 client.16944 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 10 client.16944 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 20 client.16944 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23ae070 0x5578e23b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 4 client.16944 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578e229cd40 con 0x5578e23ae070 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.420+0000 7f5e45d13640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23ae070 0x5578e23b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e297fa640 10 client.16944 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:57.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e45d13640 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23ae070 0x5578e23b0490 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7564d7158937ed60 server_cookie=65f83bc8005a50a7 in_seq=0 out_seq=0 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e297fa640 1 -- 172.21.15.7:0/311303846 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578e229cd40 con 0x5578e23ae070 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e297fa640 10 client.16944 handle_command_reply: tid=0 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 client.16944 shutdown 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 msgr2=0x5578e2340740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e2340740 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5e30003b50 tx=0x7f5e30005b70 comp rx=0 tx=0).stop 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 shutdown_connections 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5578e23ae070 0x5578e23b0490 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578e2336e50 0x5578e2341700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e2331ce0 0x5578e2340740 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 --2- 172.21.15.7:0/311303846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578e23312f0 0x5578e2340200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e297fa640 0 client.16944 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:44:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 >> 172.21.15.7:0/311303846 conn(0x5578e232c860 msgr2=0x5578e233b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 shutdown_connections 2024-03-19T00:44:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 1 -- 172.21.15.7:0/311303846 wait complete. 2024-03-19T00:44:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.421+0000 7f5e4a7fd740 20 client.16944 trim_cache size 0 max 16384 2024-03-19T00:44:57.430 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:44:57.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e2c0a2800 msgr2=0x7f5e2c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e2c0a2800 0x7f5e2c0a4cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5578e2332540 tx=0x7f5e38009500 comp rx=0 tx=0).stop 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 msgr2=0x7f5e40197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 0x7f5e40197a70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00e9c0 tx=0x7f5e3c00ee80 comp rx=0 tx=0).stop 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 shutdown_connections 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5e2c0a2800 0x7f5e2c0a4cc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e401033d0 0x7f5e4019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e40102a10 0x7f5e40197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 --2- 172.21.15.7:0/2785804927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e40101820 0x7f5e40197a70 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 >> 172.21.15.7:0/2785804927 conn(0x7f5e400fcd50 msgr2=0x7f5e40105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 shutdown_connections 2024-03-19T00:44:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.422+0000 7f5e47f77640 1 -- 172.21.15.7:0/2785804927 wait complete. 2024-03-19T00:44:57.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:44:57.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:57 smithi204 ceph-mon[66572]: pgmap v561: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 -- 172.21.15.7:0/976333297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40100720 msgr2=0x7f2a40100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 --2- 172.21.15.7:0/976333297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40100720 0x7f2a40100b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009860 tx=0x7f2a3402bb70 comp rx=0 tx=0).stop 2024-03-19T00:44:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 -- 172.21.15.7:0/976333297 shutdown_connections 2024-03-19T00:44:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 --2- 172.21.15.7:0/976333297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 0x7f2a40108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 --2- 172.21.15.7:0/976333297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40101910 0x7f2a40101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 --2- 172.21.15.7:0/976333297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40100720 0x7f2a40100b40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 -- 172.21.15.7:0/976333297 >> 172.21.15.7:0/976333297 conn(0x7f2a400fbc50 msgr2=0x7f2a400fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 -- 172.21.15.7:0/976333297 shutdown_connections 2024-03-19T00:44:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.537+0000 7f2a46e36640 1 -- 172.21.15.7:0/976333297 wait complete. 2024-03-19T00:44:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 Processor -- start 2024-03-19T00:44:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 -- start start 2024-03-19T00:44:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40100720 0x7f2a40197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 0x7f2a40197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 0x7f2a4019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010d370 con 0x7f2a401022d0 2024-03-19T00:44:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010d1f0 con 0x7f2a40101910 2024-03-19T00:44:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a46e36640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4010d4f0 con 0x7f2a40100720 2024-03-19T00:44:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a453d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 0x7f2a4019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40100720 0x7f2a40197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 0x7f2a40197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 0x7f2a40197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48532/0 (socket says 172.21.15.7:48532) 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40100720 0x7f2a40197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:59846/0 (socket says 172.21.15.7:59846) 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 -- 172.21.15.7:0/3493000018 learned_addr learned my addr 172.21.15.7:0/3493000018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 -- 172.21.15.7:0/3493000018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40100720 msgr2=0x7f2a40197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40100720 0x7f2a40197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 -- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 msgr2=0x7f2a4019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 0x7f2a4019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 -- 172.21.15.7:0/3493000018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4019f6c0 con 0x7f2a40101910 2024-03-19T00:44:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 0x7f2a4019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 0x7f2a40197fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e9f0 tx=0x7f2a2c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.538+0000 7f2a3dffb640 1 -- 172.21.15.7:0/3493000018 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00cd80 con 0x7f2a40101910 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.539+0000 7f2a3dffb640 1 -- 172.21.15.7:0/3493000018 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2c0054d0 con 0x7f2a40101910 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.539+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4019f9b0 con 0x7f2a40101910 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.539+0000 7f2a3dffb640 1 -- 172.21.15.7:0/3493000018 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c010490 con 0x7f2a40101910 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.539+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4019feb0 con 0x7f2a40101910 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a3dffb640 1 -- 172.21.15.7:0/3493000018 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f2a2c005020 con 0x7f2a40101910 2024-03-19T00:44:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 Processor -- start 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 -- start start 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a3dffb640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2a300a2800 0x7f2a300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 0x562aedad3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedad87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedadb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562aeda374b0 con 0x562aedad8e50 2024-03-19T00:44:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562aed9dfd70 con 0x562aedad32f0 2024-03-19T00:44:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a49a27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562aeda33550 con 0x562aedad3ce0 2024-03-19T00:44:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2a300a2800 0x7f2a300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a3dffb640 1 -- 172.21.15.7:0/3493000018 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2a2c0cf120 con 0x7f2a40101910 2024-03-19T00:44:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 0x562aedad3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 0x562aedad3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48546/0 (socket says 172.21.15.7:48546) 2024-03-19T00:44:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 -- 172.21.15.7:0/574411907 learned_addr learned my addr 172.21.15.7:0/574411907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.541+0000 7f2a453d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedadb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedad87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 -- 172.21.15.7:0/574411907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 msgr2=0x562aedad87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedad87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 -- 172.21.15.7:0/574411907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 msgr2=0x562aedadb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedadb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 -- 172.21.15.7:0/574411907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aeda02210 con 0x562aedad32f0 2024-03-19T00:44:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2a300a2800 0x7f2a300a4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005fd0 tx=0x7f2a3800e6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 0x562aedad3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009860 tx=0x7f2a34005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d74b63d9d0082eb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a2affd640 1 -- 172.21.15.7:0/574411907 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34039040 con 0x562aedad32f0 2024-03-19T00:44:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a2affd640 1 -- 172.21.15.7:0/574411907 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a340346c0 con 0x562aedad32f0 2024-03-19T00:44:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.542+0000 7f2a2affd640 1 -- 172.21.15.7:0/574411907 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3402c880 con 0x562aedad32f0 2024-03-19T00:44:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 -- 172.21.15.7:0/574411907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 msgr2=0x562aedad3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 0x562aedad3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009860 tx=0x7f2a34005b20 comp rx=0 tx=0).stop 2024-03-19T00:44:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 -- 172.21.15.7:0/574411907 shutdown_connections 2024-03-19T00:44:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedadb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedad87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 --2- 172.21.15.7:0/574411907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad32f0 0x562aedad3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 -- 172.21.15.7:0/574411907 >> 172.21.15.7:0/574411907 conn(0x562aedace860 msgr2=0x562aedacec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 -- 172.21.15.7:0/574411907 shutdown_connections 2024-03-19T00:44:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 -- 172.21.15.7:0/574411907 wait complete. 2024-03-19T00:44:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 -- start start 2024-03-19T00:44:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.543+0000 7f2a49a27740 1 Processor -- start 2024-03-19T00:44:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedadd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedaddc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562aedadbda0 con 0x562aedad8e50 2024-03-19T00:44:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562aed9f3310 con 0x562aedad3ce0 2024-03-19T00:44:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562aedadbf20 con 0x562aedad32f0 2024-03-19T00:44:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 10 client.0 _ll_register_callbacks cb 0x562aed9e72d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60528/0 (socket says 172.21.15.7:60528) 2024-03-19T00:44:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedadd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 -- 172.21.15.7:0/560387067 learned_addr learned my addr 172.21.15.7:0/560387067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedaddc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 -- 172.21.15.7:0/560387067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 msgr2=0x562aedadd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedadd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 -- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 msgr2=0x562aedaddc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedaddc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 -- 172.21.15.7:0/560387067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aed74b3f0 con 0x562aedad8e50 2024-03-19T00:44:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedadd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a453d3640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae4ca0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2a3801dd90 tx=0x7f2a380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166628a4405445ae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a1ffff640 1 -- 172.21.15.7:0/560387067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800dd30 con 0x562aedad8e50 2024-03-19T00:44:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a1ffff640 1 -- 172.21.15.7:0/560387067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2a38003970 con 0x562aedad8e50 2024-03-19T00:44:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.544+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a340346c0 con 0x562aedad8e50 2024-03-19T00:44:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a1ffff640 1 -- 172.21.15.7:0/560387067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a38032050 con 0x562aedad8e50 2024-03-19T00:44:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 10 client.16956 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562aed8157d0 con 0x562aedad8e50 2024-03-19T00:44:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a1ffff640 1 -- 172.21.15.7:0/560387067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a340346c0 con 0x562aedad8e50 2024-03-19T00:44:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a1ffff640 1 -- 172.21.15.7:0/560387067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a38026070 con 0x562aedad8e50 2024-03-19T00:44:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 10 client.16956 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 10 client.16956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 20 client.16956 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedae95e0 0x562aedaebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 4 client.16956 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aeda953e0 con 0x562aedae95e0 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.545+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedae95e0 0x562aedaebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.546+0000 7f2a1ffff640 10 client.16956 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.546+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedae95e0 0x562aedaebaa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbcf9941c572850a server_cookie=a95edfb281e7bc05 in_seq=0 out_seq=0 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.546+0000 7f2a1ffff640 1 -- 172.21.15.7:0/560387067 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562aeda953e0 con 0x562aedae95e0 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.546+0000 7f2a1ffff640 10 client.16956 handle_command_reply: tid=0 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.546+0000 7f2a49a27740 1 client.16956 shutdown 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 msgr2=0x562aedae4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae4ca0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2a3801dd90 tx=0x7f2a380095c0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 shutdown_connections 2024-03-19T00:44:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedae95e0 0x562aedaebaa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae4ca0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedaddc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 --2- 172.21.15.7:0/560387067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedadd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a1ffff640 0 client.16956 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 >> 172.21.15.7:0/560387067 conn(0x562aedace860 msgr2=0x562aedacdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 shutdown_connections 2024-03-19T00:44:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 1 -- 172.21.15.7:0/560387067 wait complete. 2024-03-19T00:44:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.547+0000 7f2a49a27740 20 client.16956 trim_cache size 0 max 16384 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 Processor -- start 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 -- start start 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 0x562aedacd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedae4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad8e50 0x562aedacd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401acb30 con 0x562aedad32f0 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401aa400 con 0x562aedad8e50 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a49a27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401a8250 con 0x562aedad3ce0 2024-03-19T00:44:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 0x562aedacd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 0x562aedacd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60544/0 (socket says 172.21.15.7:60544) 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a44bd2640 1 -- 172.21.15.7:0/3872458551 learned_addr learned my addr 172.21.15.7:0/3872458551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad8e50 0x562aedacd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a3ffff640 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedae4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a44bd2640 1 -- 172.21.15.7:0/3872458551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 msgr2=0x562aedae4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.560+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedae4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a44bd2640 1 -- 172.21.15.7:0/3872458551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad8e50 msgr2=0x562aedacd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad8e50 0x562aedacd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a44bd2640 1 -- 172.21.15.7:0/3872458551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aedace400 con 0x562aedad32f0 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 0x562aedacd310 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2a3402e010 tx=0x7f2a3402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81646967cc674f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a2a7fc640 1 -- 172.21.15.7:0/3872458551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34055070 con 0x562aedad32f0 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a2a7fc640 1 -- 172.21.15.7:0/3872458551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a34004040 con 0x562aedad32f0 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a2a7fc640 1 -- 172.21.15.7:0/3872458551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34039040 con 0x562aedad32f0 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 -- 172.21.15.7:0/3872458551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 msgr2=0x562aedacd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 0x562aedacd310 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2a3402e010 tx=0x7f2a3402e040 comp rx=0 tx=0).stop 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 -- 172.21.15.7:0/3872458551 shutdown_connections 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad8e50 0x562aedacd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad3ce0 0x562aedae4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3872458551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad32f0 0x562aedacd310 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 -- 172.21.15.7:0/3872458551 >> 172.21.15.7:0/3872458551 conn(0x562aedace860 msgr2=0x562aedac9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 -- 172.21.15.7:0/3872458551 shutdown_connections 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.561+0000 7f2a49a27740 1 -- 172.21.15.7:0/3872458551 wait complete. 2024-03-19T00:44:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 -- start start 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 Processor -- start 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedae45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedae4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401a88c0 con 0x562aedad8e50 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401a9d30 con 0x562aedad3ce0 2024-03-19T00:44:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401a8f90 con 0x562aedad32f0 2024-03-19T00:44:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a49a27740 10 client.0 _ll_register_callbacks cb 0x7f2a3800e440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a453d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a44bd2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedae45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a453d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60554/0 (socket says 172.21.15.7:60554) 2024-03-19T00:44:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedae4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.562+0000 7f2a453d3640 1 -- 172.21.15.7:0/3825069270 learned_addr learned my addr 172.21.15.7:0/3825069270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a453d3640 1 -- 172.21.15.7:0/3825069270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 msgr2=0x562aedae45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedae45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a453d3640 1 -- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 msgr2=0x562aedae4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedae4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a453d3640 1 -- 172.21.15.7:0/3825069270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aedaa4880 con 0x562aedad8e50 2024-03-19T00:44:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedae45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a453d3640 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae2be0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2a38004730 tx=0x7f2a3800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0091fad0ddc775c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a1ffff640 1 -- 172.21.15.7:0/3825069270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3803a070 con 0x562aedad8e50 2024-03-19T00:44:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a1ffff640 1 -- 172.21.15.7:0/3825069270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2a380363f0 con 0x562aedad8e50 2024-03-19T00:44:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a34004040 con 0x562aedad8e50 2024-03-19T00:44:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a1ffff640 1 -- 172.21.15.7:0/3825069270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a38046050 con 0x562aedad8e50 2024-03-19T00:44:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 10 client.16962 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562aedb50260 con 0x562aedad8e50 2024-03-19T00:44:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a1ffff640 1 -- 172.21.15.7:0/3825069270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a34004040 con 0x562aedad8e50 2024-03-19T00:44:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a1ffff640 1 -- 172.21.15.7:0/3825069270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a38026040 con 0x562aedad8e50 2024-03-19T00:44:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 10 client.16962 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 10 client.16962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:44:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 20 client.16962 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedb50c40 0x562aedb53060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 4 client.16962 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:44:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.563+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562aeda449f0 con 0x562aedb50c40 2024-03-19T00:44:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedb50c40 0x562aedb53060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a1ffff640 10 client.16962 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a44bd2640 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedb50c40 0x562aedb53060 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a26aec4f29d76fa0 server_cookie=eb29f1cf70e0d7d6 in_seq=0 out_seq=0 2024-03-19T00:44:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a1ffff640 1 -- 172.21.15.7:0/3825069270 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562aeda449f0 con 0x562aedb50c40 2024-03-19T00:44:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a1ffff640 10 client.16962 handle_command_reply: tid=0 2024-03-19T00:44:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 client.16962 shutdown 2024-03-19T00:44:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 msgr2=0x562aedae2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae2be0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2a38004730 tx=0x7f2a3800dd90 comp rx=0 tx=0).stop 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 shutdown_connections 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x562aedb50c40 0x562aedb53060 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562aedad8e50 0x562aedae2be0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a1ffff640 0 client.16962 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562aedad3ce0 0x562aedae4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 --2- 172.21.15.7:0/3825069270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562aedad32f0 0x562aedae45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 >> 172.21.15.7:0/3825069270 conn(0x562aedace860 msgr2=0x562aedad1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 shutdown_connections 2024-03-19T00:44:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 1 -- 172.21.15.7:0/3825069270 wait complete. 2024-03-19T00:44:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.564+0000 7f2a49a27740 20 client.16962 trim_cache size 0 max 16384 2024-03-19T00:44:57.593 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:44:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2a300a2800 msgr2=0x7f2a300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2a300a2800 0x7f2a300a4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005fd0 tx=0x7f2a3800e6c0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 msgr2=0x7f2a40197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 0x7f2a40197fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e9f0 tx=0x7f2a2c00edf0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 shutdown_connections 2024-03-19T00:44:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2a300a2800 0x7f2a300a4cc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a401022d0 0x7f2a4019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a40101910 0x7f2a40197fe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 --2- 172.21.15.7:0/3493000018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a40100720 0x7f2a40197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 >> 172.21.15.7:0/3493000018 conn(0x7f2a400fbc50 msgr2=0x7f2a40104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 shutdown_connections 2024-03-19T00:44:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.565+0000 7f2a46e36640 1 -- 172.21.15.7:0/3493000018 wait complete. 2024-03-19T00:44:57.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:57 smithi007 ceph-mon[62854]: pgmap v561: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:57.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:44:57.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:57 smithi044 ceph-mon[66366]: pgmap v561: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/2517121103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14100730 msgr2=0x7f1a14100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/2517121103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14100730 0x7f1a14100b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009860 tx=0x7f1a0c02bb70 comp rx=0 tx=0).stop 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/2517121103 shutdown_connections 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/2517121103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a14108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/2517121103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14101920 0x7f1a14101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/2517121103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14100730 0x7f1a14100b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/2517121103 >> 172.21.15.7:0/2517121103 conn(0x7f1a140fbc80 msgr2=0x7f1a140fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/2517121103 shutdown_connections 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/2517121103 wait complete. 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 Processor -- start 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.702+0000 7f1a1b2f1640 1 -- start start 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14100730 0x7f1a1406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14101920 0x7f1a1419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a1419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410c230 con 0x7f1a141022e0 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410c0b0 con 0x7f1a14100730 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410c3b0 con 0x7f1a14101920 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1988e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a1419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14100730 0x7f1a1406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1988e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a1419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60570/0 (socket says 172.21.15.7:60570) 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1988e640 1 -- 172.21.15.7:0/3226809980 learned_addr learned my addr 172.21.15.7:0/3226809980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1888c640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14101920 0x7f1a1419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 -- 172.21.15.7:0/3226809980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14101920 msgr2=0x7f1a1419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14101920 0x7f1a1419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 -- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 msgr2=0x7f1a1419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a1419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 -- 172.21.15.7:0/3226809980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1419f6a0 con 0x7f1a14100730 2024-03-19T00:44:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1988e640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a1419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1908d640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14100730 0x7f1a1406aa70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c037040 tx=0x7f1a0c0040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a0a7fc640 1 -- 172.21.15.7:0/3226809980 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c041020 con 0x7f1a14100730 2024-03-19T00:44:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a0a7fc640 1 -- 172.21.15.7:0/3226809980 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c0044d0 con 0x7f1a14100730 2024-03-19T00:44:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1419f930 con 0x7f1a14100730 2024-03-19T00:44:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.703+0000 7f1a0a7fc640 1 -- 172.21.15.7:0/3226809980 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c039040 con 0x7f1a14100730 2024-03-19T00:44:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.704+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1419fe20 con 0x7f1a14100730 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 Processor -- start 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a0a7fc640 1 -- 172.21.15.7:0/3226809980 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1a0c044020 con 0x7f1a14100730 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 -- start start 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e057710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e057ce0 0x55a62e05c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 0x55a62e05f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a62dfbb4b0 con 0x55a62e05ce50 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a62df63d70 con 0x55a62e057ce0 2024-03-19T00:44:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a62dfb7550 con 0x55a62e0572f0 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1888c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 0x55a62e05f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1888c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 0x55a62e05f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60578/0 (socket says 172.21.15.7:60578) 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1908d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e057ce0 0x55a62e05c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1888c640 1 -- 172.21.15.7:0/1349071397 learned_addr learned my addr 172.21.15.7:0/1349071397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.706+0000 7f1a1988e640 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e057710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a0a7fc640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a000a2800 0x7f1a000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1888c640 1 -- 172.21.15.7:0/1349071397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 msgr2=0x55a62e057710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1888c640 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e057710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1888c640 1 -- 172.21.15.7:0/1349071397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e057ce0 msgr2=0x55a62e05c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1888c640 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e057ce0 0x55a62e05c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1888c640 1 -- 172.21.15.7:0/1349071397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62df86210 con 0x55a62e05ce50 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1988e640 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e057710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1988e640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a000a2800 0x7f1a000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a0a7fc640 1 -- 172.21.15.7:0/3226809980 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1a0c0f0600 con 0x7f1a14100730 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f1a1888c640 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 0x55a62e05f2e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009c90 tx=0x7f1a04011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6577f961bc290f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f19ff7fe640 1 -- 172.21.15.7:0/1349071397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0401e070 con 0x55a62e05ce50 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f19ff7fe640 1 -- 172.21.15.7:0/1349071397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a04019ae0 con 0x55a62e05ce50 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.707+0000 7f19ff7fe640 1 -- 172.21.15.7:0/1349071397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400e940 con 0x55a62e05ce50 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/1349071397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 msgr2=0x55a62e05f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1988e640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a000a2800 0x7f1a000a4cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55a62e058540 tx=0x7f1a10010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 0x55a62e05f2e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009c90 tx=0x7f1a04011810 comp rx=0 tx=0).stop 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/1349071397 shutdown_connections 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e05ce50 0x55a62e05f2e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e057ce0 0x55a62e05c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/1349071397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e057710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/1349071397 >> 172.21.15.7:0/1349071397 conn(0x55a62e052860 msgr2=0x55a62e052c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/1349071397 shutdown_connections 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/1349071397 wait complete. 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- start start 2024-03-19T00:44:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 Processor -- start 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e0617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e061cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e068d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a62e05fe00 con 0x55a62e057ce0 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a62df77310 con 0x55a62e05ce50 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a62e05ff80 con 0x55a62e0572f0 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.708+0000 7f1a1dfb8740 10 client.0 _ll_register_callbacks cb 0x55a62df6b2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e061cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e061cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60588/0 (socket says 172.21.15.7:60588) 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 -- 172.21.15.7:0/703242088 learned_addr learned my addr 172.21.15.7:0/703242088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1988e640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e068d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1908d640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e0617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 -- 172.21.15.7:0/703242088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 msgr2=0x55a62e0617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e0617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 -- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 msgr2=0x55a62e068d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e068d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 -- 172.21.15.7:0/703242088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62dccf3f0 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1888c640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e061cf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005bc0 tx=0x7f1a04009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4e60bf564fd3a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f19fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f19fcff9640 1 -- 172.21.15.7:0/703242088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0401e050 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f19fcff9640 1 -- 172.21.15.7:0/703242088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1a04005590 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a04019ae0 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f19fcff9640 1 -- 172.21.15.7:0/703242088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a04033050 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1dfb8740 10 client.16974 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.709+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a62dd997d0 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f19fcff9640 1 -- 172.21.15.7:0/703242088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a04019ae0 con 0x55a62e057ce0 2024-03-19T00:44:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f19fcff9640 1 -- 172.21.15.7:0/703242088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1a0400e9b0 con 0x55a62e057ce0 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1dfb8740 10 client.16974 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1dfb8740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1dfb8740 20 client.16974 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e06d430 0x55a62e06f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1dfb8740 4 client.16974 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a62e0193e0 con 0x55a62e06d430 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1908d640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e06d430 0x55a62e06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f19fcff9640 10 client.16974 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.710+0000 7f1a1908d640 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e06d430 0x55a62e06f890 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea207bb44cb8fc6 server_cookie=9eb5443aa2525865 in_seq=0 out_seq=0 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f19fcff9640 1 -- 172.21.15.7:0/703242088 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a62e0193e0 con 0x55a62e06d430 2024-03-19T00:44:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f19fcff9640 10 client.16974 handle_command_reply: tid=0 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 client.16974 shutdown 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 msgr2=0x55a62e061cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e061cf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005bc0 tx=0x7f1a04009fa0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 shutdown_connections 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e06d430 0x55a62e06f890 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e061cf0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/703242088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e0572f0 0x55a62e0617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f19fcff9640 0 client.16974 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 >> 172.21.15.7:0/703242088 conn(0x55a62e052860 msgr2=0x55a62e051ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 shutdown_connections 2024-03-19T00:44:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/703242088 wait complete. 2024-03-19T00:44:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.711+0000 7f1a1dfb8740 20 client.16974 trim_cache size 0 max 16384 2024-03-19T00:44:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.724+0000 7f1a1dfb8740 1 Processor -- start 2024-03-19T00:44:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.724+0000 7f1a1dfb8740 1 -- start start 2024-03-19T00:44:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e0572f0 0x55a62e0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e059720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e05ce50 0x55a62e068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141acb80 con 0x55a62e057ce0 2024-03-19T00:44:57.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141aa450 con 0x55a62e0572f0 2024-03-19T00:44:57.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141a82a0 con 0x55a62e05ce50 2024-03-19T00:44:57.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e059720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e059720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60594/0 (socket says 172.21.15.7:60594) 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 -- 172.21.15.7:0/3712446449 learned_addr learned my addr 172.21.15.7:0/3712446449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1908d640 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e0572f0 0x55a62e0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1988e640 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e05ce50 0x55a62e068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 -- 172.21.15.7:0/3712446449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e05ce50 msgr2=0x55a62e068cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e05ce50 0x55a62e068cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 -- 172.21.15.7:0/3712446449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e0572f0 msgr2=0x55a62e0591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e0572f0 0x55a62e0591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 -- 172.21.15.7:0/3712446449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62e028880 con 0x55a62e057ce0 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1908d640 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e0572f0 0x55a62e0591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f1a1888c640 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e059720 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1a04000c00 tx=0x7f1a040059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaf2f469bd6c59d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f19feffd640 1 -- 172.21.15.7:0/3712446449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0401e050 con 0x55a62e057ce0 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f19feffd640 1 -- 172.21.15.7:0/3712446449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0402a070 con 0x55a62e057ce0 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.725+0000 7f19feffd640 1 -- 172.21.15.7:0/3712446449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400e420 con 0x55a62e057ce0 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/3712446449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 msgr2=0x55a62e059720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e059720 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1a04000c00 tx=0x7f1a040059b0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/3712446449 shutdown_connections 2024-03-19T00:44:57.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e05ce50 0x55a62e068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e057ce0 0x55a62e059720 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/3712446449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e0572f0 0x55a62e0591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/3712446449 >> 172.21.15.7:0/3712446449 conn(0x55a62e052860 msgr2=0x55a62e0617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/3712446449 shutdown_connections 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/3712446449 wait complete. 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 -- start start 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.726+0000 7f1a1dfb8740 1 Processor -- start 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 0x55a62e065e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e057ce0 0x55a62e0663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141a8910 con 0x55a62e0572f0 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141a9d80 con 0x55a62e05ce50 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141a8fe0 con 0x55a62e057ce0 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 10 client.0 _ll_register_callbacks cb 0x7f1a0401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:57.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 0x55a62e065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 0x55a62e065e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60602/0 (socket says 172.21.15.7:60602) 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 -- 172.21.15.7:0/2493632508 learned_addr learned my addr 172.21.15.7:0/2493632508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1988e640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1888c640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e057ce0 0x55a62e0663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 -- 172.21.15.7:0/2493632508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e057ce0 msgr2=0x55a62e0663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e057ce0 0x55a62e0663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 -- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 msgr2=0x55a62e0673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e0673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 -- 172.21.15.7:0/2493632508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62e0679b0 con 0x55a62e0572f0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1908d640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 0x55a62e065e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0bc940 tx=0x7f1a0c0f6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e7e75aa06fed9f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f19fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f19fcff9640 1 -- 172.21.15.7:0/2493632508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0bd340 con 0x55a62e0572f0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f19fcff9640 1 -- 172.21.15.7:0/2493632508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1a0c0baf40 con 0x55a62e0572f0 2024-03-19T00:44:57.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.727+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0402a070 con 0x55a62e0572f0 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f19fcff9640 1 -- 172.21.15.7:0/2493632508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a0c0bcc40 con 0x55a62e0572f0 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 10 client.16980 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a62e068d80 con 0x55a62e0572f0 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f19fcff9640 1 -- 172.21.15.7:0/2493632508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0402a070 con 0x55a62e0572f0 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f19fcff9640 1 -- 172.21.15.7:0/2493632508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1a0c03c030 con 0x55a62e0572f0 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 10 client.16980 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:57.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 20 client.16980 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e0d3be0 0x55a62e0d6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 4 client.16980 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a62dfc89f0 con 0x55a62e0d3be0 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.728+0000 7f1a1888c640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e0d3be0 0x55a62e0d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f19fcff9640 10 client.16980 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1888c640 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e0d3be0 0x55a62e0d6000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b76ca817a1b751 server_cookie=21a07a26d6e5ea2 in_seq=0 out_seq=0 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f19fcff9640 1 -- 172.21.15.7:0/2493632508 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a62dfc89f0 con 0x55a62e0d3be0 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f19fcff9640 10 client.16980 handle_command_reply: tid=0 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 client.16980 shutdown 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 msgr2=0x55a62e065e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 0x55a62e065e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0bc940 tx=0x7f1a0c0f6550 comp rx=0 tx=0).stop 2024-03-19T00:44:57.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 shutdown_connections 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a62e05ce50 0x55a62e0673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62e057ce0 0x55a62e0663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55a62e0d3be0 0x55a62e0d6000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 --2- 172.21.15.7:0/2493632508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a62e0572f0 0x55a62e065e70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f19fcff9640 0 client.16980 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 >> 172.21.15.7:0/2493632508 conn(0x55a62e052860 msgr2=0x55a62e061750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 shutdown_connections 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 1 -- 172.21.15.7:0/2493632508 wait complete. 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.729+0000 7f1a1dfb8740 20 client.16980 trim_cache size 0 max 16384 2024-03-19T00:44:57.740 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:44:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a000a2800 msgr2=0x7f1a000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a000a2800 0x7f1a000a4cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55a62e058540 tx=0x7f1a10010040 comp rx=0 tx=0).stop 2024-03-19T00:44:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14100730 msgr2=0x7f1a1406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14100730 0x7f1a1406aa70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c037040 tx=0x7f1a0c0040e0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 shutdown_connections 2024-03-19T00:44:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1a000a2800 0x7f1a000a4cc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a141022e0 0x7f1a1419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a14101920 0x7f1a1419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 --2- 172.21.15.7:0/3226809980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a14100730 0x7f1a1406aa70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 >> 172.21.15.7:0/3226809980 conn(0x7f1a140fbc80 msgr2=0x7f1a14104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 shutdown_connections 2024-03-19T00:44:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:57.730+0000 7f1a1b2f1640 1 -- 172.21.15.7:0/3226809980 wait complete. 2024-03-19T00:44:58.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1661666781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:44:58.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1661666781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:44:58.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1661666781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:44:58.744 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:44:58.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:44:58.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 -- 172.21.15.7:0/3806629038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 msgr2=0x7fd3d0101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 --2- 172.21.15.7:0/3806629038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d0101830 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009ca0 tx=0x7fd3c802c100 comp rx=0 tx=0).stop 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 -- 172.21.15.7:0/3806629038 shutdown_connections 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 --2- 172.21.15.7:0/3806629038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d0101830 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 --2- 172.21.15.7:0/3806629038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0100a10 0x7fd3d0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 --2- 172.21.15.7:0/3806629038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 0x7fd3d01004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 -- 172.21.15.7:0/3806629038 >> 172.21.15.7:0/3806629038 conn(0x7fd3d00fbcb0 msgr2=0x7fd3d00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 -- 172.21.15.7:0/3806629038 shutdown_connections 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.862+0000 7fd3d7273640 1 -- 172.21.15.7:0/3806629038 wait complete. 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 Processor -- start 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 -- start start 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 0x7fd3d0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0100a10 0x7fd3d019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d01a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010f560 con 0x7fd3d01013b0 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010f3e0 con 0x7fd3d01000b0 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d7273640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010f6e0 con 0x7fd3d0100a10 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d01a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d500f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 0x7fd3d0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d01a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60618/0 (socket says 172.21.15.7:60618) 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d500f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 0x7fd3d0199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48644/0 (socket says 172.21.15.7:48644) 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 -- 172.21.15.7:0/957180987 learned_addr learned my addr 172.21.15.7:0/957180987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d480e640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0100a10 0x7fd3d019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 -- 172.21.15.7:0/957180987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0100a10 msgr2=0x7fd3d019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0100a10 0x7fd3d019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 -- 172.21.15.7:0/957180987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 msgr2=0x7fd3d0199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.863+0000 7fd3d5810640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 0x7fd3d0199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3d5810640 1 -- 172.21.15.7:0/957180987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01a1860 con 0x7fd3d01013b0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3d5810640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d01a11c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd3c802c710 tx=0x7fd3c8035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3c67fc640 1 -- 172.21.15.7:0/957180987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8039040 con 0x7fd3d01013b0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3c67fc640 1 -- 172.21.15.7:0/957180987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c8044070 con 0x7fd3d01013b0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01a1af0 con 0x7fd3d01013b0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3c67fc640 1 -- 172.21.15.7:0/957180987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80043c0 con 0x7fd3d01013b0 2024-03-19T00:44:58.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.864+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a1fe0 con 0x7fd3d01013b0 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.866+0000 7fd3c67fc640 1 -- 172.21.15.7:0/957180987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fd3c8004560 con 0x7fd3d01013b0 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 Processor -- start 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 -- start start 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bdd90 0x5573e14c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5573e1422160 con 0x5573e14bd3a0 2024-03-19T00:44:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5573e13cae10 con 0x5573e14c2f00 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5573e141e200 con 0x5573e14bdd90 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3c67fc640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd3bc0a2800 0x7fd3bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d480e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d480e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60632/0 (socket says 172.21.15.7:60632) 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d480e640 1 -- 172.21.15.7:0/2237525774 learned_addr learned my addr 172.21.15.7:0/2237525774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d500f640 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bdd90 0x5573e14c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.867+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d5810640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd3bc0a2800 0x7fd3bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3c67fc640 1 -- 172.21.15.7:0/957180987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd3c80f1420 con 0x7fd3d01013b0 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d480e640 1 -- 172.21.15.7:0/2237525774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bdd90 msgr2=0x5573e14c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bdd90 0x5573e14c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d480e640 1 -- 172.21.15.7:0/2237525774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 msgr2=0x5573e14c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d480e640 1 -- 172.21.15.7:0/2237525774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573e11fe7e0 con 0x5573e14bd3a0 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d500f640 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bdd90 0x5573e14c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bd7c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00138a0 tx=0x7fd3c00097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f74e228e784bcc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3d5810640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd3bc0a2800 0x7fd3bc0a4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80bd9f0 tx=0x7fd3c808c830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3bb7fe640 1 -- 172.21.15.7:0/2237525774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0013e20 con 0x5573e14bd3a0 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3bb7fe640 1 -- 172.21.15.7:0/2237525774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c0013170 con 0x5573e14bd3a0 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.868+0000 7fd3bb7fe640 1 -- 172.21.15.7:0/2237525774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00109d0 con 0x5573e14bd3a0 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2237525774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 msgr2=0x5573e14bd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bd7c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00138a0 tx=0x7fd3c00097a0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2237525774 shutdown_connections 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bdd90 0x5573e14c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2237525774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bd7c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2237525774 >> 172.21.15.7:0/2237525774 conn(0x5573e14b8910 msgr2=0x5573e14b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2237525774 shutdown_connections 2024-03-19T00:44:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2237525774 wait complete. 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 -- start start 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.869+0000 7fd3d9e5e740 1 Processor -- start 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bd3a0 0x5573e14c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5573e14c5eb0 con 0x5573e14bdd90 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5573e13de1a0 con 0x5573e14bd3a0 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5573e14c6030 con 0x5573e14c2f00 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d9e5e740 10 client.0 _ll_register_callbacks cb 0x5573e1135c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d480e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bd3a0 0x5573e14c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60642/0 (socket says 172.21.15.7:60642) 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 -- 172.21.15.7:0/2348754082 learned_addr learned my addr 172.21.15.7:0/2348754082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 -- 172.21.15.7:0/2348754082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 msgr2=0x5573e14cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 -- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bd3a0 msgr2=0x5573e14c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bd3a0 0x5573e14c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 -- 172.21.15.7:0/2348754082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573e125ef70 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3d500f640 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14c7e00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc012700 tx=0x7fd3cc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c832a90bcde260 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.870+0000 7fd3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/2348754082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc004030 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/2348754082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd3cc0041d0 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c0013170 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/2348754082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc025050 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 10 client.16992 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573e10f6170 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/2348754082 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c0013170 con 0x5573e14bdd90 2024-03-19T00:44:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/2348754082 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd3cc003980 con 0x5573e14bdd90 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 10 client.16992 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 20 client.16992 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e14d3880 0x5573e14d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 4 client.16992 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573e142f6a0 con 0x5573e14d3880 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.871+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e14d3880 0x5573e14d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.872+0000 7fd3d480e640 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e14d3880 0x5573e14d5d00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4df9acc14dcdfdf server_cookie=ebf1b07e61318439 in_seq=0 out_seq=0 2024-03-19T00:44:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.872+0000 7fd3b8ff9640 10 client.16992 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/2348754082 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573e142f6a0 con 0x5573e14d3880 2024-03-19T00:44:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3b8ff9640 10 client.16992 handle_command_reply: tid=0 2024-03-19T00:44:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 client.16992 shutdown 2024-03-19T00:44:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 msgr2=0x5573e14c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14c7e00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc012700 tx=0x7fd3cc012bc0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 shutdown_connections 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e14d3880 0x5573e14d5d00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14c7e00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/2348754082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bd3a0 0x5573e14c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3b8ff9640 0 client.16992 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 >> 172.21.15.7:0/2348754082 conn(0x5573e14b8910 msgr2=0x5573e14b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 shutdown_connections 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/2348754082 wait complete. 2024-03-19T00:44:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.873+0000 7fd3d9e5e740 20 client.16992 trim_cache size 0 max 16384 2024-03-19T00:44:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 Processor -- start 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 -- start start 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bdd90 0x5573e14cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14da600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01aec20 con 0x5573e14bd3a0 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01ac4f0 con 0x5573e14bdd90 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01aa340 con 0x5573e14c2f00 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60656/0 (socket says 172.21.15.7:60656) 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 -- 172.21.15.7:0/709649444 learned_addr learned my addr 172.21.15.7:0/709649444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d500f640 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bdd90 0x5573e14cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 -- 172.21.15.7:0/709649444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 msgr2=0x5573e14da600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14da600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 -- 172.21.15.7:0/709649444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bdd90 msgr2=0x5573e14cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bdd90 0x5573e14cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 -- 172.21.15.7:0/709649444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573e14b4160 con 0x5573e14bd3a0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.891+0000 7fd3d480e640 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bc280 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009250 tx=0x7fd3c0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758d863466976980 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3baffd640 1 -- 172.21.15.7:0/709649444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0013170 con 0x5573e14bd3a0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3baffd640 1 -- 172.21.15.7:0/709649444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c002e350 con 0x5573e14bd3a0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3baffd640 1 -- 172.21.15.7:0/709649444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000fac0 con 0x5573e14bd3a0 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/709649444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 msgr2=0x5573e14bc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bc280 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009250 tx=0x7fd3c0009810 comp rx=0 tx=0).stop 2024-03-19T00:44:58.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/709649444 shutdown_connections 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14c2f00 0x5573e14da600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14bdd90 0x5573e14cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/709649444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bd3a0 0x5573e14bc280 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/709649444 >> 172.21.15.7:0/709649444 conn(0x5573e14b8910 msgr2=0x5573e14bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/709649444 shutdown_connections 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/709649444 wait complete. 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.892+0000 7fd3d9e5e740 1 -- start start 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 Processor -- start 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bd3a0 0x5573e14cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01aa9b0 con 0x5573e14bdd90 2024-03-19T00:44:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01abe20 con 0x5573e14c2f00 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01ab080 con 0x5573e14bd3a0 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d9e5e740 10 client.0 _ll_register_callbacks cb 0x7fd3cc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60664/0 (socket says 172.21.15.7:60664) 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 -- 172.21.15.7:0/1052071429 learned_addr learned my addr 172.21.15.7:0/1052071429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d480e640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d480e640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bd3a0 0x5573e14cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 -- 172.21.15.7:0/1052071429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bd3a0 msgr2=0x5573e14cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bd3a0 0x5573e14cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 -- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 msgr2=0x5573e14cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.893+0000 7fd3d500f640 1 -- 172.21.15.7:0/1052071429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573e1539c40 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d500f640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14cda40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0126d0 tx=0x7fd3cc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60065a4d0239893 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/1052071429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc02d020 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/1052071429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd3cc005070 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c002e350 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/1052071429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc038050 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 10 client.16998 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573e153ae00 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/1052071429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c002e350 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/1052071429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd3cc0344a0 con 0x5573e14bdd90 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 10 client.16998 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 20 client.16998 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e153b830 0x5573e153dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 4 client.16998 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573e14299f0 con 0x5573e153b830 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.894+0000 7fd3d480e640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e153b830 0x5573e153dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.895+0000 7fd3b8ff9640 10 client.16998 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.895+0000 7fd3d480e640 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e153b830 0x5573e153dc50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1087467f802df1f6 server_cookie=431b266d4a842df0 in_seq=0 out_seq=0 2024-03-19T00:44:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3b8ff9640 1 -- 172.21.15.7:0/1052071429 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573e14299f0 con 0x5573e153b830 2024-03-19T00:44:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3b8ff9640 10 client.16998 handle_command_reply: tid=0 2024-03-19T00:44:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 client.16998 shutdown 2024-03-19T00:44:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 msgr2=0x5573e14cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14cda40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0126d0 tx=0x7fd3cc003910 comp rx=0 tx=0).stop 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 shutdown_connections 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5573e14c2f00 0x5573e14cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5573e153b830 0x5573e153dc50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573e14bdd90 0x5573e14cda40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 --2- 172.21.15.7:0/1052071429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573e14bd3a0 0x5573e14cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 >> 172.21.15.7:0/1052071429 conn(0x5573e14b8910 msgr2=0x5573e14bc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3b8ff9640 0 client.16998 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 shutdown_connections 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 1 -- 172.21.15.7:0/1052071429 wait complete. 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.896+0000 7fd3d9e5e740 20 client.16998 trim_cache size 0 max 16384 2024-03-19T00:44:58.903 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:44:58.904 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:44:58.904 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "281117a7-8b82-4882-a4f3-bd9843e81e66", 2024-03-19T00:44:58.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:44:58.904 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:44:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd3bc0a2800 msgr2=0x7fd3bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd3bc0a2800 0x7fd3bc0a4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80bd9f0 tx=0x7fd3c808c830 comp rx=0 tx=0).stop 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 msgr2=0x7fd3d01a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d01a11c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd3c802c710 tx=0x7fd3c8035d60 comp rx=0 tx=0).stop 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 shutdown_connections 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd3bc0a2800 0x7fd3bc0a4cc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3d01013b0 0x7fd3d01a11c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3d0100a10 0x7fd3d019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 --2- 172.21.15.7:0/957180987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3d01000b0 0x7fd3d0199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 >> 172.21.15.7:0/957180987 conn(0x7fd3d00fbcb0 msgr2=0x7fd3d00fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 shutdown_connections 2024-03-19T00:44:58.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:58.897+0000 7fd3d7273640 1 -- 172.21.15.7:0/957180987 wait complete. 2024-03-19T00:44:58.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:44:58.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:44:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/3908848639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe74101830 msgr2=0x7fbe74101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/3908848639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe74101830 0x7fbe74101c50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009900 tx=0x7fbe7002bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/3908848639 shutdown_connections 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/3908848639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe741033e0 0x7fbe74109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/3908848639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe74102a20 0x7fbe74102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/3908848639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe74101830 0x7fbe74101c50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/3908848639 >> 172.21.15.7:0/3908848639 conn(0x7fbe740fcda0 msgr2=0x7fbe740ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/3908848639 shutdown_connections 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/3908848639 wait complete. 2024-03-19T00:44:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 Processor -- start 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.013+0000 7fbe7c5a9640 1 -- start start 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74101830 0x7fbe74197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe74102a20 0x7fbe74197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 0x7fbe7419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7410d360 con 0x7fbe741033e0 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7410d1e0 con 0x7fbe74102a20 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7410d4e0 con 0x7fbe74101830 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 0x7fbe7419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 0x7fbe7419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60688/0 (socket says 172.21.15.7:60688) 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7a345640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74101830 0x7fbe74197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 -- 172.21.15.7:0/1288940390 learned_addr learned my addr 172.21.15.7:0/1288940390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe79b44640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe74102a20 0x7fbe74197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 -- 172.21.15.7:0/1288940390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74101830 msgr2=0x7fbe74197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74101830 0x7fbe74197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 -- 172.21.15.7:0/1288940390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe74102a20 msgr2=0x7fbe74197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe74102a20 0x7fbe74197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 -- 172.21.15.7:0/1288940390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7419f6a0 con 0x7fbe741033e0 2024-03-19T00:44:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7ab46640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 0x7fbe7419f030 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00eb10 tx=0x7fbe6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe6b7fe640 1 -- 172.21.15.7:0/1288940390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c004b20 con 0x7fbe741033e0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe6b7fe640 1 -- 172.21.15.7:0/1288940390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe6c004cc0 con 0x7fbe741033e0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7419f990 con 0x7fbe741033e0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe6b7fe640 1 -- 172.21.15.7:0/1288940390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c010690 con 0x7fbe741033e0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.014+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7419fee0 con 0x7fbe741033e0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe6b7fe640 1 -- 172.21.15.7:0/1288940390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fbe6c005020 con 0x7fbe741033e0 2024-03-19T00:44:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 Processor -- start 2024-03-19T00:44:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 -- start start 2024-03-19T00:44:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b879e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b87ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b881a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56514b7de500 con 0x56514b87f5b0 2024-03-19T00:44:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56514b786be0 con 0x56514b87a440 2024-03-19T00:44:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.017+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56514b7da5a0 con 0x56514b879a50 2024-03-19T00:44:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe6b7fe640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe600a2800 0x7fbe600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b87ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe79b44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b881a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b87ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48696/0 (socket says 172.21.15.7:48696) 2024-03-19T00:44:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 -- 172.21.15.7:0/80198497 learned_addr learned my addr 172.21.15.7:0/80198497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe79b44640 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b879e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7ab46640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe600a2800 0x7fbe600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe6b7fe640 1 -- 172.21.15.7:0/1288940390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbe6c0cf480 con 0x7fbe741033e0 2024-03-19T00:44:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 -- 172.21.15.7:0/80198497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 msgr2=0x56514b879e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b879e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 -- 172.21.15.7:0/80198497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 msgr2=0x56514b881a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b881a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 -- 172.21.15.7:0/80198497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514b7a8000 con 0x56514b87a440 2024-03-19T00:44:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe79b44640 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b881a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:44:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7a345640 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b87ef40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009900 tx=0x7fbe70009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae2d9acc2f67e2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe7ab46640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe600a2800 0x7fbe600a4cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c097f20 tx=0x7fbe6c090fa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe68bfe640 1 -- 172.21.15.7:0/80198497 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70045020 con 0x56514b87a440 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe68bfe640 1 -- 172.21.15.7:0/80198497 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe70004040 con 0x56514b87a440 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.018+0000 7fbe68bfe640 1 -- 172.21.15.7:0/80198497 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70005490 con 0x56514b87a440 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 -- 172.21.15.7:0/80198497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 msgr2=0x56514b87ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b87ef40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009900 tx=0x7fbe70009560 comp rx=0 tx=0).stop 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 -- 172.21.15.7:0/80198497 shutdown_connections 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b881a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b87ef40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/80198497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b879e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 -- 172.21.15.7:0/80198497 >> 172.21.15.7:0/80198497 conn(0x56514b874fc0 msgr2=0x56514b8753c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 -- 172.21.15.7:0/80198497 shutdown_connections 2024-03-19T00:44:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.019+0000 7fbe7f18d740 1 -- 172.21.15.7:0/80198497 wait complete. 2024-03-19T00:44:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 -- start start 2024-03-19T00:44:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 Processor -- start 2024-03-19T00:44:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 0x56514b883ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b884420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87f5b0 0x56514b88b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56514b882500 con 0x56514b879a50 2024-03-19T00:44:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56514b79a160 con 0x56514b87a440 2024-03-19T00:44:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56514b882680 con 0x56514b87f5b0 2024-03-19T00:44:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7f18d740 10 client.0 _ll_register_callbacks cb 0x56514b7d6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 0x56514b883ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 0x56514b883ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60708/0 (socket says 172.21.15.7:60708) 2024-03-19T00:44:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe7a345640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b884420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 -- 172.21.15.7:0/879688952 learned_addr learned my addr 172.21.15.7:0/879688952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87f5b0 0x56514b88b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 -- 172.21.15.7:0/879688952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87f5b0 msgr2=0x56514b88b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87f5b0 0x56514b88b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 -- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 msgr2=0x56514b884420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b884420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 -- 172.21.15.7:0/879688952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514b5bc7d0 con 0x56514b879a50 2024-03-19T00:44:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe79b44640 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 0x56514b883ee0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbe6401eab0 tx=0x7fbe64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ed9bc03b65082c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.020+0000 7fbe5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe5dffb640 1 -- 172.21.15.7:0/879688952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64003e00 con 0x56514b879a50 2024-03-19T00:44:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe5dffb640 1 -- 172.21.15.7:0/879688952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbe6401e080 con 0x56514b879a50 2024-03-19T00:44:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe70004040 con 0x56514b879a50 2024-03-19T00:44:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe5dffb640 1 -- 172.21.15.7:0/879688952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe6402b050 con 0x56514b879a50 2024-03-19T00:44:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 10 client.17010 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514b4f23f0 con 0x56514b879a50 2024-03-19T00:44:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe5dffb640 1 -- 172.21.15.7:0/879688952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe70004040 con 0x56514b879a50 2024-03-19T00:44:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe5dffb640 1 -- 172.21.15.7:0/879688952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe64010430 con 0x56514b879a50 2024-03-19T00:44:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 10 client.17010 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 20 client.17010 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b88fc90 0x56514b8920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 4 client.17010 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514b7eba40 con 0x56514b88fc90 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.021+0000 7fbe7a345640 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b88fc90 0x56514b8920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7a345640 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b88fc90 0x56514b8920e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307960359f51e4fc server_cookie=ebe3d052c3783372 in_seq=0 out_seq=0 2024-03-19T00:44:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe5dffb640 10 client.17010 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe5dffb640 1 -- 172.21.15.7:0/879688952 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56514b7eba40 con 0x56514b88fc90 2024-03-19T00:44:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe5dffb640 10 client.17010 handle_command_reply: tid=0 2024-03-19T00:44:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 client.17010 shutdown 2024-03-19T00:44:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 msgr2=0x56514b883ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 0x56514b883ee0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbe6401eab0 tx=0x7fbe64004730 comp rx=0 tx=0).stop 2024-03-19T00:44:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 shutdown_connections 2024-03-19T00:44:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87f5b0 0x56514b88b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87a440 0x56514b884420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b88fc90 0x56514b8920e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/879688952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b879a50 0x56514b883ee0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe5dffb640 0 client.17010 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.022+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 >> 172.21.15.7:0/879688952 conn(0x56514b874fc0 msgr2=0x56514b8741e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.023+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 shutdown_connections 2024-03-19T00:44:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.023+0000 7fbe7f18d740 1 -- 172.21.15.7:0/879688952 wait complete. 2024-03-19T00:44:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.023+0000 7fbe7f18d740 20 client.17010 trim_cache size 0 max 16384 2024-03-19T00:44:59.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 Processor -- start 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 -- start start 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b8701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 0x56514b884160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87f5b0 0x56514b8969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741aa9c0 con 0x56514b87a440 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741a8290 con 0x56514b87f5b0 2024-03-19T00:44:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741a60e0 con 0x56514b879a50 2024-03-19T00:44:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7a345640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 0x56514b884160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7a345640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 0x56514b884160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60724/0 (socket says 172.21.15.7:60724) 2024-03-19T00:44:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe7a345640 1 -- 172.21.15.7:0/3068532103 learned_addr learned my addr 172.21.15.7:0/3068532103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.040+0000 7fbe79b44640 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87f5b0 0x56514b8969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe79b44640 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b8701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7a345640 1 -- 172.21.15.7:0/3068532103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 msgr2=0x56514b8701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7a345640 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b8701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7a345640 1 -- 172.21.15.7:0/3068532103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87f5b0 msgr2=0x56514b8969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7a345640 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87f5b0 0x56514b8969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7a345640 1 -- 172.21.15.7:0/3068532103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514b4b20f0 con 0x56514b87a440 2024-03-19T00:44:59.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe79b44640 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b8701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:44:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7a345640 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 0x56514b884160 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x56514b87aeb0 tx=0x7fbe7002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d782a22024e00e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe5ffff640 1 -- 172.21.15.7:0/3068532103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70045040 con 0x56514b87a440 2024-03-19T00:44:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe5ffff640 1 -- 172.21.15.7:0/3068532103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe7002cbe0 con 0x56514b87a440 2024-03-19T00:44:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe5ffff640 1 -- 172.21.15.7:0/3068532103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004be0 con 0x56514b87a440 2024-03-19T00:44:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7f18d740 1 -- 172.21.15.7:0/3068532103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 msgr2=0x56514b884160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.041+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 0x56514b884160 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x56514b87aeb0 tx=0x7fbe7002bc30 comp rx=0 tx=0).stop 2024-03-19T00:44:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- 172.21.15.7:0/3068532103 shutdown_connections 2024-03-19T00:44:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b87f5b0 0x56514b8969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87a440 0x56514b884160 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/3068532103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b879a50 0x56514b8701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- 172.21.15.7:0/3068532103 >> 172.21.15.7:0/3068532103 conn(0x56514b874fc0 msgr2=0x56514b878ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- 172.21.15.7:0/3068532103 shutdown_connections 2024-03-19T00:44:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- 172.21.15.7:0/3068532103 wait complete. 2024-03-19T00:44:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- start start 2024-03-19T00:44:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 Processor -- start 2024-03-19T00:44:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b879a50 0x56514b88ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87a440 0x56514b88b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b8891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741a6750 con 0x56514b87f5b0 2024-03-19T00:44:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741a7bc0 con 0x56514b879a50 2024-03-19T00:44:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741a6e20 con 0x56514b87a440 2024-03-19T00:44:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7f18d740 10 client.0 _ll_register_callbacks cb 0x7fbe64009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:44:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe79b44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b8891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe79b44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b8891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60726/0 (socket says 172.21.15.7:60726) 2024-03-19T00:44:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe79b44640 1 -- 172.21.15.7:0/2332815615 learned_addr learned my addr 172.21.15.7:0/2332815615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:44:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe7a345640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87a440 0x56514b88b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.042+0000 7fbe79b44640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b879a50 0x56514b88ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe79b44640 1 -- 172.21.15.7:0/2332815615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87a440 msgr2=0x56514b88b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe79b44640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87a440 0x56514b88b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe79b44640 1 -- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b879a50 msgr2=0x56514b88ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:44:59.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe79b44640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b879a50 0x56514b88ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe79b44640 1 -- 172.21.15.7:0/2332815615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514b889a10 con 0x56514b87f5b0 2024-03-19T00:44:59.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe79b44640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b8891e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbe64001eb0 tx=0x7fbe64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ce5f8f7492d07 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:44:59.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:44:59.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe5dffb640 1 -- 172.21.15.7:0/2332815615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6401e110 con 0x56514b87f5b0 2024-03-19T00:44:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe5dffb640 1 -- 172.21.15.7:0/2332815615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbe6401e2b0 con 0x56514b87f5b0 2024-03-19T00:44:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe7002cbe0 con 0x56514b87f5b0 2024-03-19T00:44:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe5dffb640 1 -- 172.21.15.7:0/2332815615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe6403f050 con 0x56514b87f5b0 2024-03-19T00:44:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 10 client.17016 fetch_fsmap learned FSMap version 27 2024-03-19T00:44:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514b8f6b90 con 0x56514b87f5b0 2024-03-19T00:44:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe5dffb640 1 -- 172.21.15.7:0/2332815615 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe7002cbe0 con 0x56514b87f5b0 2024-03-19T00:44:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe5dffb640 1 -- 172.21.15.7:0/2332815615 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe64027040 con 0x56514b87f5b0 2024-03-19T00:44:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 10 client.17016 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:44:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:44:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 20 client.17016 populate_metadata read hostname 'smithi007' 2024-03-19T00:44:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b8f7510 0x56514b8f9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:44:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 4 client.17016 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:44:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.043+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56514b7e5d90 con 0x56514b8f7510 2024-03-19T00:44:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7a345640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b8f7510 0x56514b8f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:44:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7a345640 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b8f7510 0x56514b8f9930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f82c1b644ed26b3 server_cookie=22235ac3f5aaed53 in_seq=0 out_seq=0 2024-03-19T00:44:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe5dffb640 10 client.17016 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe5dffb640 1 -- 172.21.15.7:0/2332815615 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56514b7e5d90 con 0x56514b8f7510 2024-03-19T00:44:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe5dffb640 10 client.17016 handle_command_reply: tid=0 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 client.17016 shutdown 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 msgr2=0x56514b8891e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b8891e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbe64001eb0 tx=0x7fbe64004730 comp rx=0 tx=0).stop 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 shutdown_connections 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56514b8f7510 0x56514b8f9930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56514b87f5b0 0x56514b8891e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe5dffb640 0 client.17016 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56514b87a440 0x56514b88b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 --2- 172.21.15.7:0/2332815615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56514b879a50 0x56514b88ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 >> 172.21.15.7:0/2332815615 conn(0x56514b874fc0 msgr2=0x56514b878ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 shutdown_connections 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 1 -- 172.21.15.7:0/2332815615 wait complete. 2024-03-19T00:44:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.044+0000 7fbe7f18d740 20 client.17016 trim_cache size 0 max 16384 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "281117a7-8b82-4882-a4f3-bd9843e81e66": { 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "281117a7-8b82-4882-a4f3-bd9843e81e66", 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:44:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:44:59.075 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe600a2800 msgr2=0x7fbe600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe600a2800 0x7fbe600a4cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c097f20 tx=0x7fbe6c090fa0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 msgr2=0x7fbe7419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 0x7fbe7419f030 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00eb10 tx=0x7fbe6c00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 shutdown_connections 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fbe600a2800 0x7fbe600a4cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe741033e0 0x7fbe7419f030 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe74102a20 0x7fbe74197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 --2- 172.21.15.7:0/1288940390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe74101830 0x7fbe74197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:44:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 >> 172.21.15.7:0/1288940390 conn(0x7fbe740fcda0 msgr2=0x7fbe74105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:44:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 shutdown_connections 2024-03-19T00:44:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:44:59.045+0000 7fbe7c5a9640 1 -- 172.21.15.7:0/1288940390 wait complete. 2024-03-19T00:44:59.077 INFO:tasks.cephfs.filesystem:scrub status for tag:281117a7-8b82-4882-a4f3-bd9843e81e66 - {'path': '/', 'tag': '281117a7-8b82-4882-a4f3-bd9843e81e66', 'options': 'recursive,force'} 2024-03-19T00:44:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:59 smithi007 ceph-mon[62854]: pgmap v562: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:59 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:44:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:59 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:59 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:44:59.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:44:59 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:59 smithi204 ceph-mon[66572]: pgmap v562: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:59 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:44:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:59 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:59 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:44:59.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:44:59 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:59 smithi044 ceph-mon[66366]: pgmap v562: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:44:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:59 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:44:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:59 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:44:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:59 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:44:59.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:44:59 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:00.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:00 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:45:00.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:00 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:00.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:00 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:45:00.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:00 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:00.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:00 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:45:00.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:00 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:01.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:01 smithi204 ceph-mon[66572]: pgmap v563: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:45:01.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:01 smithi044 ceph-mon[66366]: pgmap v563: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:45:01.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:01 smithi007 ceph-mon[62854]: pgmap v563: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:45:03.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:03 smithi204 ceph-mon[66572]: pgmap v564: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:03.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:03 smithi044 ceph-mon[66366]: pgmap v564: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:03.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:03 smithi007 ceph-mon[62854]: pgmap v564: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:05.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:05 smithi204 ceph-mon[66572]: pgmap v565: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:45:05.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:05 smithi044 ceph-mon[66366]: pgmap v565: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:45:05.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:05 smithi007 ceph-mon[62854]: pgmap v565: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:45:06.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:06 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:45:06.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:06 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:45:06.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:06 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:45:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:07 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:45:07.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:07 smithi204 ceph-mon[66572]: pgmap v566: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:07 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:45:07.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:07 smithi044 ceph-mon[66366]: pgmap v566: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:07.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:07 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:45:07.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:07 smithi007 ceph-mon[62854]: pgmap v566: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:09.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:09 smithi204 ceph-mon[66572]: pgmap v567: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:09.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:09 smithi044 ceph-mon[66366]: pgmap v567: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:09.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:09 smithi007 ceph-mon[62854]: pgmap v567: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:11.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:11 smithi204 ceph-mon[66572]: pgmap v568: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:11.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:11 smithi044 ceph-mon[66366]: pgmap v568: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:11.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:11 smithi007 ceph-mon[62854]: pgmap v568: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:13 smithi204 ceph-mon[66572]: pgmap v569: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:45:13.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:13 smithi044 ceph-mon[66366]: pgmap v569: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:45:13.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:13 smithi007 ceph-mon[62854]: pgmap v569: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:45:15.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:15 smithi204 ceph-mon[66572]: pgmap v570: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:15.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:15 smithi044 ceph-mon[66366]: pgmap v570: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:15.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:15 smithi007 ceph-mon[62854]: pgmap v570: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:17.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:17 smithi204 ceph-mon[66572]: pgmap v571: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:17.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:17 smithi044 ceph-mon[66366]: pgmap v571: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:17.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:17 smithi007 ceph-mon[62854]: pgmap v571: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:19.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:19 smithi204 ceph-mon[66572]: pgmap v572: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:19.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:19 smithi044 ceph-mon[66366]: pgmap v572: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:19.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:19 smithi007 ceph-mon[62854]: pgmap v572: 129 pgs: 129 active+clean; 2.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:21.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:21 smithi204 ceph-mon[66572]: pgmap v573: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:21.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:21 smithi007 ceph-mon[62854]: pgmap v573: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:21.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:21 smithi044 ceph-mon[66366]: pgmap v573: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:23.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:23 smithi204 ceph-mon[66572]: pgmap v574: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:23.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:23 smithi044 ceph-mon[66366]: pgmap v574: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:23.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:23 smithi007 ceph-mon[62854]: pgmap v574: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:25.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:25 smithi204 ceph-mon[66572]: pgmap v575: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:45:25.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:25 smithi044 ceph-mon[66366]: pgmap v575: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:45:25.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:25 smithi007 ceph-mon[62854]: pgmap v575: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:45:27.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:27 smithi044 ceph-mon[66366]: pgmap v576: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:27.666 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.666 INFO:tasks.workunit.client.0.smithi007.stdout:313/600: open fd = 3 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:314/600: open fd = 3 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:315/600: open fd = 3 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:316/600: open fd = 3 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.667 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:317/600: open fd = 3 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:318/600: open fd = 3 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:319/600: open fd = 3 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.668 INFO:tasks.workunit.client.0.smithi007.stdout:320/600: open fd = 3 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:321/600: open fd = 3 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:322/600: open fd = 3 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.669 INFO:tasks.workunit.client.0.smithi007.stdout:323/600: open fd = 3 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:324/600: open fd = 3 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:325/600: open fd = 3 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:326/600: open fd = 3 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.670 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:327/600: open fd = 3 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:328/600: open fd = 3 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:329/600: open fd = 3 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:330/600: open fd = 3 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.671 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:331/600: open fd = 3 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:332/600: open fd = 3 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:333/600: open fd = 3 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:334/600: open fd = 3 2024-03-19T00:45:27.672 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:335/600: open fd = 3 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:336/600: open fd = 3 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:337/600: open fd = 3 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:338/600: open fd = 3 2024-03-19T00:45:27.673 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:339/600: open fd = 3 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:340/600: open fd = 3 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:341/600: open fd = 3 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.674 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:342/600: open fd = 3 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:343/600: open fd = 3 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:344/600: open fd = 3 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:345/600: open fd = 3 2024-03-19T00:45:27.675 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:346/600: open fd = 3 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:347/600: open fd = 3 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:348/600: open fd = 3 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.676 INFO:tasks.workunit.client.0.smithi007.stdout:349/600: open fd = 3 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:350/600: open fd = 3 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:351/600: open fd = 3 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:352/600: open fd = 3 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.677 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:353/600: open fd = 3 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:354/600: open fd = 3 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:355/600: open fd = 3 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:356/600: open fd = 3 2024-03-19T00:45:27.678 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:357/600: open fd = 3 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:358/600: open fd = 3 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:359/600: open fd = 3 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.679 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:360/600: open fd = 3 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:361/600: open fd = 3 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:362/600: open fd = 3 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.680 INFO:tasks.workunit.client.0.smithi007.stdout:363/600: open fd = 3 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:364/600: open fd = 3 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:365/600: open fd = 3 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:366/600: open fd = 3 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.681 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:367/600: open fd = 3 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:368/600: open fd = 3 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:369/600: open fd = 3 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.682 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:370/600: open fd = 3 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:371/600: open fd = 3 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:372/600: open fd = 3 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:373/600: open fd = 3 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.683 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.684 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.684 INFO:tasks.workunit.client.0.smithi007.stdout:374/600: open fd = 3 2024-03-19T00:45:27.684 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.684 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:45:27.684 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:45:27.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:27 smithi007 ceph-mon[62854]: pgmap v576: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:27.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:27 smithi204 ceph-mon[66572]: pgmap v576: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:29.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:45:29.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 -- 172.21.15.7:0/2322272789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850101930 msgr2=0x7f3850101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 --2- 172.21.15.7:0/2322272789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850101930 0x7f3850101db0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3844009900 tx=0x7f384402bc30 comp rx=0 tx=0).stop 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 -- 172.21.15.7:0/2322272789 shutdown_connections 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 --2- 172.21.15.7:0/2322272789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38501022f0 0x7f3850108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 --2- 172.21.15.7:0/2322272789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850101930 0x7f3850101db0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 --2- 172.21.15.7:0/2322272789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3850100740 0x7f3850100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 -- 172.21.15.7:0/2322272789 >> 172.21.15.7:0/2322272789 conn(0x7f38500fbcb0 msgr2=0x7f38500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 -- 172.21.15.7:0/2322272789 shutdown_connections 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.195+0000 7f3856359640 1 -- 172.21.15.7:0/2322272789 wait complete. 2024-03-19T00:45:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 Processor -- start 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 -- start start 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 0x7f3850197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3850101930 0x7f3850197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38501022f0 0x7f385019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f385010d390 con 0x7f3850100740 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f385010d210 con 0x7f38501022f0 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f3856359640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f385010d510 con 0x7f3850101930 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f384ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 0x7f3850197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f384ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 0x7f3850197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52592/0 (socket says 172.21.15.7:52592) 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f384ffff640 1 -- 172.21.15.7:0/3485416615 learned_addr learned my addr 172.21.15.7:0/3485416615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f384f7fe640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3850101930 0x7f3850197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.196+0000 7f38548f6640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38501022f0 0x7f385019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384ffff640 1 -- 172.21.15.7:0/3485416615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3850101930 msgr2=0x7f3850197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384ffff640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3850101930 0x7f3850197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384ffff640 1 -- 172.21.15.7:0/3485416615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38501022f0 msgr2=0x7f385019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384ffff640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38501022f0 0x7f385019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384ffff640 1 -- 172.21.15.7:0/3485416615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385019f690 con 0x7f3850100740 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f38548f6640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38501022f0 0x7f385019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384ffff640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 0x7f3850197a70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f384000e9c0 tx=0x7f384000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384d7fa640 1 -- 172.21.15.7:0/3485416615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384000cd80 con 0x7f3850100740 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384d7fa640 1 -- 172.21.15.7:0/3485416615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38400035f0 con 0x7f3850100740 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385019f980 con 0x7f3850100740 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f384d7fa640 1 -- 172.21.15.7:0/3485416615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840010690 con 0x7f3850100740 2024-03-19T00:45:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.197+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385019fea0 con 0x7f3850100740 2024-03-19T00:45:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f384d7fa640 1 -- 172.21.15.7:0/3485416615 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f38400037a0 con 0x7f3850100740 2024-03-19T00:45:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 Processor -- start 2024-03-19T00:45:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 -- start start 2024-03-19T00:45:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce82d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce832350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce834e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560cce78d500 con 0x560cce82d850 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560cce735be0 con 0x560cce82ce60 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f3859015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560cce7895a0 con 0x560cce8329c0 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f384d7fa640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f38380a2800 0x7f38380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.200+0000 7f384ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce832350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce832350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52602/0 (socket says 172.21.15.7:52602) 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 -- 172.21.15.7:0/2104127442 learned_addr learned my addr 172.21.15.7:0/2104127442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38548f6640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce82d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384f7fe640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce834e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38548f6640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f38380a2800 0x7f38380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384d7fa640 1 -- 172.21.15.7:0/3485416615 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f38400cf150 con 0x7f3850100740 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 -- 172.21.15.7:0/2104127442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 msgr2=0x560cce834e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce834e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 -- 172.21.15.7:0/2104127442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 msgr2=0x560cce82d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce82d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 -- 172.21.15.7:0/2104127442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cce757000 con 0x560cce82d850 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38548f6640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce82d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384f7fe640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce834e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f384ffff640 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce832350 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3840097ba0 tx=0x7f38400097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3ad2c062db2468 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38548f6640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f38380a2800 0x7f38380a4cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f38480099e0 tx=0x7f3848009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38367fc640 1 -- 172.21.15.7:0/2104127442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840003b30 con 0x560cce82d850 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38367fc640 1 -- 172.21.15.7:0/2104127442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f384009be90 con 0x560cce82d850 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.201+0000 7f38367fc640 1 -- 172.21.15.7:0/2104127442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400a3aa0 con 0x560cce82d850 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 -- 172.21.15.7:0/2104127442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 msgr2=0x560cce832350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce832350 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3840097ba0 tx=0x7f38400097c0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 -- 172.21.15.7:0/2104127442 shutdown_connections 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce834e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce832350 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 --2- 172.21.15.7:0/2104127442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce82d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 -- 172.21.15.7:0/2104127442 >> 172.21.15.7:0/2104127442 conn(0x560cce823fc0 msgr2=0x560cce8243c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 -- 172.21.15.7:0/2104127442 shutdown_connections 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 -- 172.21.15.7:0/2104127442 wait complete. 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.202+0000 7f3859015740 1 -- start start 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 Processor -- start 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 0x560cce8371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82d850 0x560cce837730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce83e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560cce8358b0 con 0x560cce82ce60 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560cce749160 con 0x560cce82d850 2024-03-19T00:45:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560cce835a30 con 0x560cce8329c0 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f3859015740 10 client.0 _ll_register_callbacks cb 0x560cce785a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 0x560cce8371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 0x560cce8371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52608/0 (socket says 172.21.15.7:52608) 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 -- 172.21.15.7:0/2567891956 learned_addr learned my addr 172.21.15.7:0/2567891956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384f7fe640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82d850 0x560cce837730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f38548f6640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce83e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 -- 172.21.15.7:0/2567891956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 msgr2=0x560cce83e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce83e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 -- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82d850 msgr2=0x560cce837730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82d850 0x560cce837730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384ffff640 1 -- 172.21.15.7:0/2567891956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cce56b7d0 con 0x560cce82ce60 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.203+0000 7f384f7fe640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82d850 0x560cce837730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f384ffff640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 0x560cce8371f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f384009b720 tx=0x7f3840097ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b3602dc813663c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f382f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f382f7fe640 1 -- 172.21.15.7:0/2567891956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384009b790 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f382f7fe640 1 -- 172.21.15.7:0/2567891956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f38400a3cb0 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384009be90 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f382f7fe640 1 -- 172.21.15.7:0/2567891956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3840086110 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 10 client.17028 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560cce4a13f0 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f382f7fe640 1 -- 172.21.15.7:0/2567891956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f384009be90 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f382f7fe640 1 -- 172.21.15.7:0/2567891956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f38400a0800 con 0x560cce82ce60 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 10 client.17028 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 20 client.17028 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce842c60 0x560cce845050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 4 client.17028 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:45:29.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.204+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cce79aa40 con 0x560cce842c60 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.205+0000 7f384f7fe640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce842c60 0x560cce845050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.205+0000 7f384f7fe640 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce842c60 0x560cce845050 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110a8fbc549b551f server_cookie=5d36f746126df418 in_seq=0 out_seq=0 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.205+0000 7f382f7fe640 10 client.17028 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.205+0000 7f382f7fe640 1 -- 172.21.15.7:0/2567891956 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560cce79aa40 con 0x560cce842c60 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.205+0000 7f382f7fe640 10 client.17028 handle_command_reply: tid=0 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 client.17028 shutdown 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 msgr2=0x560cce8371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 0x560cce8371f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f384009b720 tx=0x7f3840097ab0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 shutdown_connections 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce8329c0 0x560cce83e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82d850 0x560cce837730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce842c60 0x560cce845050 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 --2- 172.21.15.7:0/2567891956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82ce60 0x560cce8371f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 >> 172.21.15.7:0/2567891956 conn(0x560cce823fc0 msgr2=0x560cce8231e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f382f7fe640 0 client.17028 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 shutdown_connections 2024-03-19T00:45:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 1 -- 172.21.15.7:0/2567891956 wait complete. 2024-03-19T00:45:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.206+0000 7f3859015740 20 client.17028 trim_cache size 0 max 16384 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.223+0000 7f3859015740 1 Processor -- start 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.223+0000 7f3859015740 1 -- start start 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3859015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82ce60 0x560cce82e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3859015740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce83e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3859015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce8329c0 0x560cce837690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3859015740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38501aa9b0 con 0x560cce82d850 2024-03-19T00:45:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3859015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38501a8280 con 0x560cce8329c0 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3859015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38501a60d0 con 0x560cce82ce60 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce83e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce83e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52616/0 (socket says 172.21.15.7:52616) 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 -- 172.21.15.7:0/947758108 learned_addr learned my addr 172.21.15.7:0/947758108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384ffff640 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82ce60 0x560cce82e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f38548f6640 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce8329c0 0x560cce837690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 -- 172.21.15.7:0/947758108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82ce60 msgr2=0x560cce82e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82ce60 0x560cce82e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 -- 172.21.15.7:0/947758108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce8329c0 msgr2=0x560cce837690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce8329c0 0x560cce837690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 -- 172.21.15.7:0/947758108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cce4610f0 con 0x560cce82d850 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f384f7fe640 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce83e730 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3844009900 tx=0x7f384402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6dd7913911dc73 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3835ffb640 1 -- 172.21.15.7:0/947758108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844004100 con 0x560cce82d850 2024-03-19T00:45:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3835ffb640 1 -- 172.21.15.7:0/947758108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38440054f0 con 0x560cce82d850 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.224+0000 7f3835ffb640 1 -- 172.21.15.7:0/947758108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844004b20 con 0x560cce82d850 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 -- 172.21.15.7:0/947758108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 msgr2=0x560cce83e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce83e730 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3844009900 tx=0x7f384402cbf0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 -- 172.21.15.7:0/947758108 shutdown_connections 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce8329c0 0x560cce837690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce82d850 0x560cce83e730 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 --2- 172.21.15.7:0/947758108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82ce60 0x560cce82e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 -- 172.21.15.7:0/947758108 >> 172.21.15.7:0/947758108 conn(0x560cce823fc0 msgr2=0x560cce827220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 -- 172.21.15.7:0/947758108 shutdown_connections 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 -- 172.21.15.7:0/947758108 wait complete. 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.225+0000 7f3859015740 1 -- start start 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 Processor -- start 2024-03-19T00:45:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce83b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82d850 0x560cce83ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 0x560cce8a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38501a6740 con 0x560cce8329c0 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38501a7bb0 con 0x560cce82ce60 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38501a6e10 con 0x560cce82d850 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f3859015740 10 client.0 _ll_register_callbacks cb 0x7f3848011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 0x560cce8a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82d850 0x560cce83ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 0x560cce8a6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52626/0 (socket says 172.21.15.7:52626) 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f384ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce83b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 -- 172.21.15.7:0/4251632352 learned_addr learned my addr 172.21.15.7:0/4251632352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 -- 172.21.15.7:0/4251632352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82d850 msgr2=0x560cce83ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82d850 0x560cce83ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 -- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 msgr2=0x560cce83b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce83b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 -- 172.21.15.7:0/4251632352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cce83c0a0 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f384f7fe640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82d850 0x560cce83ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f384ffff640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce83b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f38548f6640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 0x560cce8a6d00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3848003ec0 tx=0x7f3848004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7e823c053a1c03 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.226+0000 7f382f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f382f7fe640 1 -- 172.21.15.7:0/4251632352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848011020 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f382f7fe640 1 -- 172.21.15.7:0/4251632352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f38480111c0 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38440054f0 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f382f7fe640 1 -- 172.21.15.7:0/4251632352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3848036050 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 10 client.17034 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560cce8a8390 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f382f7fe640 1 -- 172.21.15.7:0/4251632352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38440054f0 con 0x560cce8329c0 2024-03-19T00:45:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f382f7fe640 1 -- 172.21.15.7:0/4251632352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f384801d040 con 0x560cce8329c0 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 10 client.17034 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 20 client.17034 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce8a8df0 0x560cce8ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 4 client.17034 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cce794d90 con 0x560cce8a8df0 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.227+0000 7f384ffff640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce8a8df0 0x560cce8ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f382f7fe640 10 client.17034 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f384ffff640 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce8a8df0 0x560cce8ab210 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d391add6b42e538 server_cookie=29384e5cba60a9d1 in_seq=0 out_seq=0 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f382f7fe640 1 -- 172.21.15.7:0/4251632352 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560cce794d90 con 0x560cce8a8df0 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f382f7fe640 10 client.17034 handle_command_reply: tid=0 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 client.17034 shutdown 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 msgr2=0x560cce8a6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 0x560cce8a6d00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3848003ec0 tx=0x7f3848004ab0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 shutdown_connections 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560cce8a8df0 0x560cce8ab210 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560cce8329c0 0x560cce8a6d00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560cce82d850 0x560cce83ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 --2- 172.21.15.7:0/4251632352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560cce82ce60 0x560cce83b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f382f7fe640 0 client.17034 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 >> 172.21.15.7:0/4251632352 conn(0x560cce823fc0 msgr2=0x560cce827ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 shutdown_connections 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 1 -- 172.21.15.7:0/4251632352 wait complete. 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.228+0000 7f3859015740 20 client.17034 trim_cache size 0 max 16384 2024-03-19T00:45:29.236 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:45:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:45:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:45:29.237 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:45:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f38380a2800 msgr2=0x7f38380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f38380a2800 0x7f38380a4cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f38480099e0 tx=0x7f3848009500 comp rx=0 tx=0).stop 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 msgr2=0x7f3850197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 0x7f3850197a70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f384000e9c0 tx=0x7f384000ee80 comp rx=0 tx=0).stop 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 shutdown_connections 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f38380a2800 0x7f38380a4cc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38501022f0 0x7f385019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3850101930 0x7f3850197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 --2- 172.21.15.7:0/3485416615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3850100740 0x7f3850197a70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 >> 172.21.15.7:0/3485416615 conn(0x7f38500fbcb0 msgr2=0x7f3850104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 shutdown_connections 2024-03-19T00:45:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.229+0000 7f3856359640 1 -- 172.21.15.7:0/3485416615 wait complete. 2024-03-19T00:45:29.240 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:45:29.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.342+0000 7fa0858ff640 1 -- 172.21.15.7:0/410335392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080102300 msgr2=0x7fa080108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.342+0000 7fa0858ff640 1 --2- 172.21.15.7:0/410335392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080102300 0x7fa080108b90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa078009ca0 tx=0x7fa07802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- 172.21.15.7:0/410335392 shutdown_connections 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 --2- 172.21.15.7:0/410335392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080102300 0x7fa080108b90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 --2- 172.21.15.7:0/410335392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 0x7fa080101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 --2- 172.21.15.7:0/410335392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa080100750 0x7fa080100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- 172.21.15.7:0/410335392 >> 172.21.15.7:0/410335392 conn(0x7fa0800fbcc0 msgr2=0x7fa0800fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- 172.21.15.7:0/410335392 shutdown_connections 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- 172.21.15.7:0/410335392 wait complete. 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 Processor -- start 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- start start 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 0x7fa080197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 0x7fa080197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa080102300 0x7fa08019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010c280 con 0x7fa080100750 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010c100 con 0x7fa080101940 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.343+0000 7fa0858ff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010c400 con 0x7fa080102300 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 0x7fa080197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 0x7fa080197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52650/0 (socket says 172.21.15.7:52650) 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 -- 172.21.15.7:0/3749732860 learned_addr learned my addr 172.21.15.7:0/3749732860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07e7fc640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 0x7fa080197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07f7fe640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa080102300 0x7fa08019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 -- 172.21.15.7:0/3749732860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa080102300 msgr2=0x7fa08019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa080102300 0x7fa08019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 -- 172.21.15.7:0/3749732860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 msgr2=0x7fa080197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 0x7fa080197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 -- 172.21.15.7:0/3749732860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08019f630 con 0x7fa080100750 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07e7fc640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 0x7fa080197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa07effd640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 0x7fa080197a10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa07400e9c0 tx=0x7fa07400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400cd80 con 0x7fa080100750 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0740035f0 con 0x7fa080100750 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08019f920 con 0x7fa080100750 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074010690 con 0x7fa080100750 2024-03-19T00:45:29.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.344+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08019fe20 con 0x7fa080100750 2024-03-19T00:45:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.345+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0801a00b0 con 0x7fa080100750 2024-03-19T00:45:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.346+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa0740037a0 con 0x7fa080100750 2024-03-19T00:45:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.347+0000 7fa0848fd640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0680a2800 0x7fa0680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.347+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa0740cf310 con 0x7fa080100750 2024-03-19T00:45:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.347+0000 7fa07e7fc640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0680a2800 0x7fa0680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.347+0000 7fa07e7fc640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0680a2800 0x7fa0680a4cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa080198f20 tx=0x7fa070005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.352+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa07409c050 con 0x7fa080100750 2024-03-19T00:45:29.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:29 smithi007 ceph-mon[62854]: pgmap v577: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.580+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0801a0280 con 0x7fa080100750 2024-03-19T00:45:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.581+0000 7fa0848fd640 1 -- 172.21.15.7:0/3749732860 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fa07409b950 con 0x7fa080100750 2024-03-19T00:45:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:45:29.582 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:45:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:45:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0680a2800 msgr2=0x7fa0680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0680a2800 0x7fa0680a4cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa080198f20 tx=0x7fa070005f90 comp rx=0 tx=0).stop 2024-03-19T00:45:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 msgr2=0x7fa080197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 0x7fa080197a10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa07400e9c0 tx=0x7fa07400ee80 comp rx=0 tx=0).stop 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 shutdown_connections 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa0680a2800 0x7fa0680a4cc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa080102300 0x7fa08019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa080101940 0x7fa080197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 --2- 172.21.15.7:0/3749732860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa080100750 0x7fa080197a10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 >> 172.21.15.7:0/3749732860 conn(0x7fa0800fbcc0 msgr2=0x7fa080068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 shutdown_connections 2024-03-19T00:45:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.583+0000 7fa0858ff640 1 -- 172.21.15.7:0/3749732860 wait complete. 2024-03-19T00:45:29.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:45:29.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:29 smithi044 ceph-mon[66366]: pgmap v577: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.698+0000 7f166c873640 1 -- 172.21.15.7:0/1227281868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 msgr2=0x7f1668101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.698+0000 7f166c873640 1 --2- 172.21.15.7:0/1227281868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668101d90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1650009900 tx=0x7f165002bc30 comp rx=0 tx=0).stop 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- 172.21.15.7:0/1227281868 shutdown_connections 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 --2- 172.21.15.7:0/1227281868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16681022d0 0x7f1668108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 --2- 172.21.15.7:0/1227281868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668101d90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 --2- 172.21.15.7:0/1227281868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1668100720 0x7f1668100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- 172.21.15.7:0/1227281868 >> 172.21.15.7:0/1227281868 conn(0x7f16680fbc50 msgr2=0x7f16680fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- 172.21.15.7:0/1227281868 shutdown_connections 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- 172.21.15.7:0/1227281868 wait complete. 2024-03-19T00:45:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 Processor -- start 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- start start 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1668100720 0x7f1668196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16681022d0 0x7f166819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f166810c250 con 0x7f1668101910 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f166810c0d0 con 0x7f1668100720 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.699+0000 7f166c873640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f166810c3d0 con 0x7f16681022d0 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52658/0 (socket says 172.21.15.7:52658) 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 -- 172.21.15.7:0/1012308878 learned_addr learned my addr 172.21.15.7:0/1012308878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f166659c640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1668100720 0x7f1668196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f1666d9d640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16681022d0 0x7f166819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 -- 172.21.15.7:0/1012308878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16681022d0 msgr2=0x7f166819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16681022d0 0x7f166819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 -- 172.21.15.7:0/1012308878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1668100720 msgr2=0x7f1668196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1668100720 0x7f1668196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 -- 172.21.15.7:0/1012308878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166819e540 con 0x7f1668101910 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165ffff640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668196e60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1650006010 tx=0x7f1650005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165f7fe640 1 -- 172.21.15.7:0/1012308878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650043020 con 0x7f1668101910 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166819e7d0 con 0x7f1668101910 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165f7fe640 1 -- 172.21.15.7:0/1012308878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16500043d0 con 0x7f1668101910 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f165f7fe640 1 -- 172.21.15.7:0/1012308878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650003b80 con 0x7f1668101910 2024-03-19T00:45:29.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.700+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166819ed70 con 0x7f1668101910 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.703+0000 7f165f7fe640 1 -- 172.21.15.7:0/1012308878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1650046020 con 0x7f1668101910 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.703+0000 7f166f45e740 1 Processor -- start 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.703+0000 7f166f45e740 1 -- start start 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166f45e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef32f0 0x55c536ef3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166f45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536ef87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166f45e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef8e50 0x55c536efb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166f45e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c536e574b0 con 0x55c536ef32f0 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166f45e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c536dffd70 con 0x55c536ef8e50 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166f45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c536e53550 con 0x55c536ef3ce0 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f165f7fe640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16540a2800 0x7f16540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f1666d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef32f0 0x55c536ef3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536ef87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f165ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef8e50 0x55c536efb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536ef87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:37642/0 (socket says 172.21.15.7:37642) 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 -- 172.21.15.7:0/4110383504 learned_addr learned my addr 172.21.15.7:0/4110383504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f1666d9d640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16540a2800 0x7f16540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f165f7fe640 1 -- 172.21.15.7:0/1012308878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f16500f0f70 con 0x7f1668101910 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 -- 172.21.15.7:0/4110383504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef8e50 msgr2=0x55c536efb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef8e50 0x55c536efb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 -- 172.21.15.7:0/4110383504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef32f0 msgr2=0x55c536ef3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef32f0 0x55c536ef3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 -- 172.21.15.7:0/4110383504 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c536e22210 con 0x55c536ef3ce0 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f166659c640 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536ef87e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1658009ca0 tx=0x7f1658011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41df8b7beeffce5d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.704+0000 7f1666d9d640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16540a2800 0x7f16540a4cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55c536ef4540 tx=0x7f1660009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f165cff9640 1 -- 172.21.15.7:0/4110383504 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658004220 con 0x55c536ef3ce0 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f165cff9640 1 -- 172.21.15.7:0/4110383504 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1658009e50 con 0x55c536ef3ce0 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f165cff9640 1 -- 172.21.15.7:0/4110383504 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658019650 con 0x55c536ef3ce0 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 -- 172.21.15.7:0/4110383504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 msgr2=0x55c536ef87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536ef87e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1658009ca0 tx=0x7f1658011810 comp rx=0 tx=0).stop 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 -- 172.21.15.7:0/4110383504 shutdown_connections 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef8e50 0x55c536efb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536ef87e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 --2- 172.21.15.7:0/4110383504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef32f0 0x55c536ef3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 -- 172.21.15.7:0/4110383504 >> 172.21.15.7:0/4110383504 conn(0x55c536eee860 msgr2=0x55c536eeec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 -- 172.21.15.7:0/4110383504 shutdown_connections 2024-03-19T00:45:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.705+0000 7f166f45e740 1 -- 172.21.15.7:0/4110383504 wait complete. 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 -- start start 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 Processor -- start 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536efd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536efdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c536efbe00 con 0x55c536ef8e50 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c536e13310 con 0x55c536ef32f0 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c536efbf80 con 0x55c536ef3ce0 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166f45e740 10 client.0 _ll_register_callbacks cb 0x55c536e072d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536efd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f04d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52680/0 (socket says 172.21.15.7:52680) 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 -- 172.21.15.7:0/4000006903 learned_addr learned my addr 172.21.15.7:0/4000006903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f165ffff640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536efdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 -- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 msgr2=0x55c536efdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536efdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 -- 172.21.15.7:0/4000006903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 msgr2=0x55c536efd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536efd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f1666d9d640 1 -- 172.21.15.7:0/4000006903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c536b6b3f0 con 0x55c536ef8e50 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.706+0000 7f166659c640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536efd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f1666d9d640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f04d30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1660011ab0 tx=0x7f1660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d066004a73255a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f164dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f164dffb640 1 -- 172.21.15.7:0/4000006903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660004b20 con 0x55c536ef8e50 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f164dffb640 1 -- 172.21.15.7:0/4000006903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1660004cc0 con 0x55c536ef8e50 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1658009e50 con 0x55c536ef8e50 2024-03-19T00:45:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f164dffb640 1 -- 172.21.15.7:0/4000006903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1660033050 con 0x55c536ef8e50 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 10 client.17052 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c536c357d0 con 0x55c536ef8e50 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f164dffb640 1 -- 172.21.15.7:0/4000006903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1658009e50 con 0x55c536ef8e50 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f164dffb640 1 -- 172.21.15.7:0/4000006903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1660027030 con 0x55c536ef8e50 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 10 client.17052 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 10 client.17052 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 20 client.17052 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f096d0 0x55c536f0bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 4 client.17052 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.707+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c536eb53e0 con 0x55c536f096d0 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.708+0000 7f166659c640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f096d0 0x55c536f0bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.708+0000 7f166659c640 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f096d0 0x55c536f0bb90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0cb469a7fe730e1 server_cookie=f264bc0f9d5e15c0 in_seq=0 out_seq=0 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.708+0000 7f164dffb640 10 client.17052 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:45:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f164dffb640 1 -- 172.21.15.7:0/4000006903 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c536eb53e0 con 0x55c536f096d0 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f164dffb640 10 client.17052 handle_command_reply: tid=0 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 client.17052 shutdown 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 msgr2=0x55c536f04d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f04d30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1660011ab0 tx=0x7f1660004a40 comp rx=0 tx=0).stop 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 shutdown_connections 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f096d0 0x55c536f0bb90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f04d30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f164dffb640 0 client.17052 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef3ce0 0x55c536efdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 --2- 172.21.15.7:0/4000006903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536efd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 >> 172.21.15.7:0/4000006903 conn(0x55c536eee860 msgr2=0x55c536eedba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 shutdown_connections 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 1 -- 172.21.15.7:0/4000006903 wait complete. 2024-03-19T00:45:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.709+0000 7f166f45e740 20 client.17052 trim_cache size 0 max 16384 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 Processor -- start 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 -- start start 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536eeec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 0x55c536eed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef8e50 0x55c536eed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16681abaa0 con 0x55c536ef3ce0 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16681a9370 con 0x55c536ef32f0 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166f45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f16681a71c0 con 0x55c536ef8e50 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f165ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 0x55c536eed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f165ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 0x55c536eed310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52686/0 (socket says 172.21.15.7:52686) 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f165ffff640 1 -- 172.21.15.7:0/3557109440 learned_addr learned my addr 172.21.15.7:0/3557109440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f166659c640 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536eeec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.722+0000 7f1666d9d640 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef8e50 0x55c536eed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f165ffff640 1 -- 172.21.15.7:0/3557109440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef8e50 msgr2=0x55c536eed850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f165ffff640 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef8e50 0x55c536eed850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f165ffff640 1 -- 172.21.15.7:0/3557109440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 msgr2=0x55c536eeec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f165ffff640 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536eeec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f165ffff640 1 -- 172.21.15.7:0/3557109440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c536ec4880 con 0x55c536ef3ce0 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f165ffff640 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 0x55c536eed310 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f16500ee710 tx=0x7f16500902e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88f51649017b53c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f164ffff640 1 -- 172.21.15.7:0/3557109440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16500bd380 con 0x55c536ef3ce0 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f164ffff640 1 -- 172.21.15.7:0/3557109440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16500bdcb0 con 0x55c536ef3ce0 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f164ffff640 1 -- 172.21.15.7:0/3557109440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16500bcf10 con 0x55c536ef3ce0 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 -- 172.21.15.7:0/3557109440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 msgr2=0x55c536eed310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 0x55c536eed310 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f16500ee710 tx=0x7f16500902e0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 -- 172.21.15.7:0/3557109440 shutdown_connections 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef8e50 0x55c536eed850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef3ce0 0x55c536eed310 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 --2- 172.21.15.7:0/3557109440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef32f0 0x55c536eeec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 -- 172.21.15.7:0/3557109440 >> 172.21.15.7:0/3557109440 conn(0x55c536eee860 msgr2=0x55c536ef4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 -- 172.21.15.7:0/3557109440 shutdown_connections 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.723+0000 7f166f45e740 1 -- 172.21.15.7:0/3557109440 wait complete. 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 -- start start 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 Processor -- start 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef32f0 0x55c536f6bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef3ce0 0x55c536f6e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16681a7830 con 0x55c536ef8e50 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16681a8ca0 con 0x55c536ef3ce0 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f16681a7f00 con 0x55c536ef32f0 2024-03-19T00:45:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166f45e740 10 client.0 _ll_register_callbacks cb 0x7f1660011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f165ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef3ce0 0x55c536f6e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f70550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52700/0 (socket says 172.21.15.7:52700) 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 -- 172.21.15.7:0/2491326671 learned_addr learned my addr 172.21.15.7:0/2491326671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f166659c640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef32f0 0x55c536f6bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 -- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef32f0 msgr2=0x55c536f6bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef32f0 0x55c536f6bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 -- 172.21.15.7:0/2491326671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef3ce0 msgr2=0x55c536f6e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef3ce0 0x55c536f6e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f1666d9d640 1 -- 172.21.15.7:0/2491326671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c536f70c90 con 0x55c536ef8e50 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.724+0000 7f165ffff640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef3ce0 0x55c536f6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f1666d9d640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f70550 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1660009bf0 tx=0x7f1660003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab65eb82b30384c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f164dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f164dffb640 1 -- 172.21.15.7:0/2491326671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166003c060 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f164dffb640 1 -- 172.21.15.7:0/2491326671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1660004cb0 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16500bdcb0 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f164dffb640 1 -- 172.21.15.7:0/2491326671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1660043050 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 10 client.17058 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c536f71f70 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f164dffb640 1 -- 172.21.15.7:0/2491326671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16500bdcb0 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f164dffb640 1 -- 172.21.15.7:0/2491326671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1660027040 con 0x55c536ef8e50 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 10 client.17058 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 10 client.17058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 20 client.17058 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f728f0 0x55c536f74d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 4 client.17058 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c536e649f0 con 0x55c536f728f0 2024-03-19T00:45:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.725+0000 7f166659c640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f728f0 0x55c536f74d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f164dffb640 10 client.17058 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166659c640 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f728f0 0x55c536f74d10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd0daa3ae412d94a server_cookie=efebea2e130b0388 in_seq=0 out_seq=0 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f164dffb640 1 -- 172.21.15.7:0/2491326671 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c536e649f0 con 0x55c536f728f0 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f164dffb640 10 client.17058 handle_command_reply: tid=0 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 client.17058 shutdown 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 msgr2=0x55c536f70550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f70550 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1660009bf0 tx=0x7f1660003ec0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 shutdown_connections 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55c536f728f0 0x55c536f74d10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c536ef8e50 0x55c536f70550 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c536ef3ce0 0x55c536f6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f164dffb640 0 client.17058 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 --2- 172.21.15.7:0/2491326671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c536ef32f0 0x55c536f6bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 >> 172.21.15.7:0/2491326671 conn(0x55c536eee860 msgr2=0x55c536eeec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 shutdown_connections 2024-03-19T00:45:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 1 -- 172.21.15.7:0/2491326671 wait complete. 2024-03-19T00:45:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.726+0000 7f166f45e740 20 client.17058 trim_cache size 0 max 16384 2024-03-19T00:45:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16540a2800 msgr2=0x7f16540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16540a2800 0x7f16540a4cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55c536ef4540 tx=0x7f1660009500 comp rx=0 tx=0).stop 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 msgr2=0x7f1668196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668196e60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1650006010 tx=0x7f1650005e40 comp rx=0 tx=0).stop 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 shutdown_connections 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16540a2800 0x7f16540a4cc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16681022d0 0x7f166819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1668101910 0x7f1668196e60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 --2- 172.21.15.7:0/1012308878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1668100720 0x7f1668196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 >> 172.21.15.7:0/1012308878 conn(0x7f16680fbc50 msgr2=0x7f1668104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 shutdown_connections 2024-03-19T00:45:29.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.727+0000 7f166c873640 1 -- 172.21.15.7:0/1012308878 wait complete. 2024-03-19T00:45:29.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:45:29.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:29 smithi204 ceph-mon[66572]: pgmap v577: 129 pgs: 129 active+clean; 2.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 -- 172.21.15.7:0/3780732535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc100740 msgr2=0x7fe7dc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 --2- 172.21.15.7:0/3780732535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc100740 0x7fe7dc100b60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009900 tx=0x7fe7cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 -- 172.21.15.7:0/3780732535 shutdown_connections 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 --2- 172.21.15.7:0/3780732535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 --2- 172.21.15.7:0/3780732535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 0x7fe7dc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 --2- 172.21.15.7:0/3780732535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc100740 0x7fe7dc100b60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 -- 172.21.15.7:0/3780732535 >> 172.21.15.7:0/3780732535 conn(0x7fe7dc0fbc90 msgr2=0x7fe7dc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 -- 172.21.15.7:0/3780732535 shutdown_connections 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 -- 172.21.15.7:0/3780732535 wait complete. 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 Processor -- start 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.842+0000 7fe7e080c640 1 -- start start 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc100740 0x7fe7dc196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 0x7fe7dc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10c260 con 0x7fe7dc1022f0 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10c0e0 con 0x7fe7dc100740 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10c3e0 con 0x7fe7dc101930 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc100740 0x7fe7dc196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc19df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52710/0 (socket says 172.21.15.7:52710) 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 -- 172.21.15.7:0/423219979 learned_addr learned my addr 172.21.15.7:0/423219979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 0x7fe7dc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 -- 172.21.15.7:0/423219979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 msgr2=0x7fe7dc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 0x7fe7dc196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 -- 172.21.15.7:0/423219979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc100740 msgr2=0x7fe7dc196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc100740 0x7fe7dc196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 -- 172.21.15.7:0/423219979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc19e5b0 con 0x7fe7dc1022f0 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7da59c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc100740 0x7fe7dc196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 0x7fe7dc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7dad9d640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc19df40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400eb10 tx=0x7fe7d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7d37fe640 1 -- 172.21.15.7:0/423219979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4004b20 con 0x7fe7dc1022f0 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc069f40 con 0x7fe7dc1022f0 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7d37fe640 1 -- 172.21.15.7:0/423219979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d4004cc0 con 0x7fe7dc1022f0 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7d37fe640 1 -- 172.21.15.7:0/423219979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4010690 con 0x7fe7dc1022f0 2024-03-19T00:45:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.843+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc06a440 con 0x7fe7dc1022f0 2024-03-19T00:45:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7d37fe640 1 -- 172.21.15.7:0/423219979 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe7d4005020 con 0x7fe7dc1022f0 2024-03-19T00:45:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 Processor -- start 2024-03-19T00:45:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 -- start start 2024-03-19T00:45:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e408710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e40d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e4102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e36c4b0 con 0x55fc1e40de50 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e314d70 con 0x55fc1e4082f0 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e368550 con 0x55fc1e408ce0 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.846+0000 7fe7d37fe640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7c80a2800 0x7fe7c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e40d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e408710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e40d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:37692/0 (socket says 172.21.15.7:37692) 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d37fe640 1 -- 172.21.15.7:0/423219979 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7d40cf350 con 0x7fe7dc1022f0 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 -- 172.21.15.7:0/250969956 learned_addr learned my addr 172.21.15.7:0/250969956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e408710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43280/0 (socket says 172.21.15.7:43280) 2024-03-19T00:45:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7dad9d640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7c80a2800 0x7fe7c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e4102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 -- 172.21.15.7:0/250969956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 msgr2=0x55fc1e408710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e408710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 -- 172.21.15.7:0/250969956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 msgr2=0x55fc1e4102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e4102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 -- 172.21.15.7:0/250969956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1e337210 con 0x55fc1e408ce0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e408710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7dad9d640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7c80a2800 0x7fe7c80a4cc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4098360 tx=0x7fe7d4086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7da59c640 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e40d7e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009900 tx=0x7fe7cc0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f79a97f7ae7f15a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d0ff9640 1 -- 172.21.15.7:0/250969956 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc04a070 con 0x55fc1e408ce0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d0ff9640 1 -- 172.21.15.7:0/250969956 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7cc02cc90 con 0x55fc1e408ce0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.847+0000 7fe7d0ff9640 1 -- 172.21.15.7:0/250969956 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc004920 con 0x55fc1e408ce0 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 -- 172.21.15.7:0/250969956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 msgr2=0x55fc1e40d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e40d7e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009900 tx=0x7fe7cc0095f0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 -- 172.21.15.7:0/250969956 shutdown_connections 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e4102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e40d7e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/250969956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e408710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 -- 172.21.15.7:0/250969956 >> 172.21.15.7:0/250969956 conn(0x55fc1e403860 msgr2=0x55fc1e403c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 -- 172.21.15.7:0/250969956 shutdown_connections 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 -- 172.21.15.7:0/250969956 wait complete. 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 -- start start 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.848+0000 7fe7e33f1740 1 Processor -- start 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e412cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e419d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e410da0 con 0x55fc1e40de50 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e328310 con 0x55fc1e4082f0 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e410f20 con 0x55fc1e408ce0 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 10 client.0 _ll_register_callbacks cb 0x55fc1e31c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e419d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e412cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7d3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e419d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52724/0 (socket says 172.21.15.7:52724) 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7d3fff640 1 -- 172.21.15.7:0/1474519623 learned_addr learned my addr 172.21.15.7:0/1474519623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 -- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 msgr2=0x55fc1e4127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 -- 172.21.15.7:0/1474519623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 msgr2=0x55fc1e419d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e419d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 -- 172.21.15.7:0/1474519623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1e0803f0 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e419d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7da59c640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e412cf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55fc1e4047d0 tx=0x7fe7cc005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc015133a518ef server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/1474519623 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc04a050 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7cc02cc90 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.849+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/1474519623 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7cc003900 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/1474519623 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7cc003b00 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 10 client.26827 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc1e14a7d0 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/1474519623 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7cc02cc90 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/1474519623 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe7cc03b440 con 0x55fc1e408ce0 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 10 client.26827 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 10 client.26827 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 20 client.26827 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e41e220 0x55fc1e420610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 4 client.26827 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1e3ca3e0 con 0x55fc1e41e220 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.850+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e41e220 0x55fc1e420610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7c1ffb640 10 client.26827 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e41e220 0x55fc1e420610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=258c520428886af8 server_cookie=b1c15f1b559353f9 in_seq=0 out_seq=0 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/1474519623 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fc1e3ca3e0 con 0x55fc1e41e220 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7c1ffb640 10 client.26827 handle_command_reply: tid=0 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 client.26827 shutdown 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 msgr2=0x55fc1e412cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e412cf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55fc1e4047d0 tx=0x7fe7cc005640 comp rx=0 tx=0).stop 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 shutdown_connections 2024-03-19T00:45:29.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e41e220 0x55fc1e420610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e40de50 0x55fc1e419d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7c1ffb640 0 client.26827 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e408ce0 0x55fc1e412cf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/1474519623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.851+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 >> 172.21.15.7:0/1474519623 conn(0x55fc1e403860 msgr2=0x55fc1e402ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.852+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 shutdown_connections 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.852+0000 7fe7e33f1740 1 -- 172.21.15.7:0/1474519623 wait complete. 2024-03-19T00:45:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.852+0000 7fe7e33f1740 20 client.26827 trim_cache size 0 max 16384 2024-03-19T00:45:29.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 Processor -- start 2024-03-19T00:45:29.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 -- start start 2024-03-19T00:45:29.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e419cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e412cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1aa8c0 con 0x55fc1e408ce0 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1a8190 con 0x55fc1e4082f0 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1a6000 con 0x55fc1e40de50 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e419cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e419cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52732/0 (socket says 172.21.15.7:52732) 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 -- 172.21.15.7:0/566421807 learned_addr learned my addr 172.21.15.7:0/566421807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e412cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 -- 172.21.15.7:0/566421807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 msgr2=0x55fc1e412cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e412cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 -- 172.21.15.7:0/566421807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 msgr2=0x55fc1e4127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 -- 172.21.15.7:0/566421807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1e4090c0 con 0x55fc1e408ce0 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.865+0000 7fe7da59c640 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e419cf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03b840 tx=0x7fe7cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db063546fd122cad server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7c3fff640 1 -- 172.21.15.7:0/566421807 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc04a040 con 0x55fc1e408ce0 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7c3fff640 1 -- 172.21.15.7:0/566421807 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7cc02cc90 con 0x55fc1e408ce0 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7c3fff640 1 -- 172.21.15.7:0/566421807 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc000e60 con 0x55fc1e408ce0 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 -- 172.21.15.7:0/566421807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 msgr2=0x55fc1e419cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e419cf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03b840 tx=0x7fe7cc005fd0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 -- 172.21.15.7:0/566421807 shutdown_connections 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e412cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e419cf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/566421807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e4127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 -- 172.21.15.7:0/566421807 >> 172.21.15.7:0/566421807 conn(0x55fc1e403860 msgr2=0x55fc1e403c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 -- 172.21.15.7:0/566421807 shutdown_connections 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 -- 172.21.15.7:0/566421807 wait complete. 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 -- start start 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.866+0000 7fe7e33f1740 1 Processor -- start 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e417370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e4178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e4188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1a6650 con 0x55fc1e408ce0 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1a7ac0 con 0x55fc1e4082f0 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1a6d20 con 0x55fc1e40de50 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 10 client.0 _ll_register_callbacks cb 0x7fe7cc048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e4178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e4178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52736/0 (socket says 172.21.15.7:52736) 2024-03-19T00:45:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 -- 172.21.15.7:0/3623429132 learned_addr learned my addr 172.21.15.7:0/3623429132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e417370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e4188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 -- 172.21.15.7:0/3623429132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 msgr2=0x55fc1e4188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e4188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 -- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 msgr2=0x55fc1e417370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e417370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 -- 172.21.15.7:0/3623429132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1e418eb0 con 0x55fc1e408ce0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e4188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7da59c640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e4178b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03b7c0 tx=0x7fe7cc005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f971af1f4813d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/3623429132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc05c020 con 0x55fc1e408ce0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/3623429132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7cc03bdc0 con 0x55fc1e408ce0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.867+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7cc02cc90 con 0x55fc1e408ce0 2024-03-19T00:45:29.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/3623429132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7cc067050 con 0x55fc1e408ce0 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 10 client.17076 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc1e484400 con 0x55fc1e408ce0 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/3623429132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7cc02cc90 con 0x55fc1e408ce0 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/3623429132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe7cc03c470 con 0x55fc1e408ce0 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 10 client.17076 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 10 client.17076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 20 client.17076 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e484c20 0x55fc1e487040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 4 client.17076 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc1e3799f0 con 0x55fc1e484c20 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e484c20 0x55fc1e487040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7d3fff640 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e484c20 0x55fc1e487040 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2bd338e7c169dd1 server_cookie=f0ed17de77c38b15 in_seq=0 out_seq=0 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.868+0000 7fe7c1ffb640 10 client.17076 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7c1ffb640 1 -- 172.21.15.7:0/3623429132 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc1e3799f0 con 0x55fc1e484c20 2024-03-19T00:45:29.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7c1ffb640 10 client.17076 handle_command_reply: tid=0 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 client.17076 shutdown 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 msgr2=0x55fc1e4178b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e4178b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc03b7c0 tx=0x7fe7cc005860 comp rx=0 tx=0).stop 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 shutdown_connections 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55fc1e484c20 0x55fc1e487040 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc1e40de50 0x55fc1e4188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc1e408ce0 0x55fc1e4178b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7c1ffb640 0 client.17076 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 --2- 172.21.15.7:0/3623429132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc1e4082f0 0x55fc1e417370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 >> 172.21.15.7:0/3623429132 conn(0x55fc1e403860 msgr2=0x55fc1e412750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 shutdown_connections 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 1 -- 172.21.15.7:0/3623429132 wait complete. 2024-03-19T00:45:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e33f1740 20 client.17076 trim_cache size 0 max 16384 2024-03-19T00:45:29.878 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7c80a2800 msgr2=0x7fe7c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7c80a2800 0x7fe7c80a4cc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4098360 tx=0x7fe7d4086040 comp rx=0 tx=0).stop 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 msgr2=0x7fe7dc19df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc19df40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400eb10 tx=0x7fe7d400c2d0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 shutdown_connections 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7c80a2800 0x7fe7c80a4cc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7dc1022f0 0x7fe7dc19df40 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7dc101930 0x7fe7dc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 --2- 172.21.15.7:0/423219979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc100740 0x7fe7dc196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 >> 172.21.15.7:0/423219979 conn(0x7fe7dc0fbc90 msgr2=0x7fe7dc104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 shutdown_connections 2024-03-19T00:45:29.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.869+0000 7fe7e080c640 1 -- 172.21.15.7:0/423219979 wait complete. 2024-03-19T00:45:29.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:45:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 -- 172.21.15.7:0/2179407360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8101850 msgr2=0x7f00d8101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 --2- 172.21.15.7:0/2179407360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8101850 0x7f00d8101c70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009900 tx=0x7f00cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:45:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 -- 172.21.15.7:0/2179407360 shutdown_connections 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 --2- 172.21.15.7:0/2179407360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8103400 0x7f00d8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 --2- 172.21.15.7:0/2179407360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00d8102a40 0x7f00d8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 --2- 172.21.15.7:0/2179407360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8101850 0x7f00d8101c70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 -- 172.21.15.7:0/2179407360 >> 172.21.15.7:0/2179407360 conn(0x7f00d80fcd80 msgr2=0x7f00d80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 -- 172.21.15.7:0/2179407360 shutdown_connections 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.985+0000 7f00debf4640 1 -- 172.21.15.7:0/2179407360 wait complete. 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 Processor -- start 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 -- start start 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8101850 0x7f00d8075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 0x7f00d8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00d8103400 0x7f00d8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00d810d350 con 0x7f00d8102a40 2024-03-19T00:45:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00d810d1d0 con 0x7f00d8103400 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00debf4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00d810d4d0 con 0x7f00d8101850 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 0x7f00d8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 0x7f00d8073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52758/0 (socket says 172.21.15.7:52758) 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 -- 172.21.15.7:0/4108984131 learned_addr learned my addr 172.21.15.7:0/4108984131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00dc990640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8101850 0x7f00d8075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00dd191640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00d8103400 0x7f00d8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 -- 172.21.15.7:0/4108984131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8101850 msgr2=0x7f00d8075580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8101850 0x7f00d8075580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 -- 172.21.15.7:0/4108984131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00d8103400 msgr2=0x7f00d8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00d8103400 0x7f00d8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 -- 172.21.15.7:0/4108984131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d8074a10 con 0x7f00d8102a40 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00dc990640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8101850 0x7f00d8075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d7fff640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 0x7f00d8073c10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f00c400ea60 tx=0x7f00c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.986+0000 7f00d5ffb640 1 -- 172.21.15.7:0/4108984131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c400cde0 con 0x7f00d8102a40 2024-03-19T00:45:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.987+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d81a1bd0 con 0x7f00d8102a40 2024-03-19T00:45:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.987+0000 7f00d5ffb640 1 -- 172.21.15.7:0/4108984131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c40035f0 con 0x7f00d8102a40 2024-03-19T00:45:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.987+0000 7f00d5ffb640 1 -- 172.21.15.7:0/4108984131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4010690 con 0x7f00d8102a40 2024-03-19T00:45:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.987+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d81a20f0 con 0x7f00d8102a40 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.989+0000 7f00d5ffb640 1 -- 172.21.15.7:0/4108984131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f00c40037a0 con 0x7f00d8102a40 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.989+0000 7f00e17e5740 1 Processor -- start 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.989+0000 7f00e17e5740 1 -- start start 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c6ce0 0x55f3a73cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00d5ffb640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f00c80a2800 0x7f00c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00e17e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3a732a4b0 con 0x55f3a73c62f0 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00e17e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f3a72d2d70 con 0x55f3a73cbe50 2024-03-19T00:45:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00e17e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3a7326550 con 0x55f3a73c6ce0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52774/0 (socket says 172.21.15.7:52774) 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 -- 172.21.15.7:0/806935506 learned_addr learned my addr 172.21.15.7:0/806935506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dc990640 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c6ce0 0x55f3a73cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00d5ffb640 1 -- 172.21.15.7:0/4108984131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f00c40ceff0 con 0x7f00d8102a40 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f00c80a2800 0x7f00c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 -- 172.21.15.7:0/806935506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c6ce0 msgr2=0x55f3a73cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c6ce0 0x55f3a73cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 -- 172.21.15.7:0/806935506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 msgr2=0x55f3a73ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 -- 172.21.15.7:0/806935506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a72f5210 con 0x55f3a73c62f0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dc990640 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c6ce0 0x55f3a73cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00dd191640 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73c6710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f00d00099e0 tx=0x7f00d0011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b3717d0cf31547 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.990+0000 7f00c2ffd640 1 -- 172.21.15.7:0/806935506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0022020 con 0x55f3a73c62f0 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00dd191640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f00c80a2800 0x7f00c80a4cc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f00d0023000 tx=0x7f00d001f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00c2ffd640 1 -- 172.21.15.7:0/806935506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00d00114c0 con 0x55f3a73c62f0 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00c2ffd640 1 -- 172.21.15.7:0/806935506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d001dbc0 con 0x55f3a73c62f0 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 -- 172.21.15.7:0/806935506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 msgr2=0x55f3a73c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73c6710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f00d00099e0 tx=0x7f00d0011e00 comp rx=0 tx=0).stop 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 -- 172.21.15.7:0/806935506 shutdown_connections 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c6ce0 0x55f3a73cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 --2- 172.21.15.7:0/806935506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73c6710 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 -- 172.21.15.7:0/806935506 >> 172.21.15.7:0/806935506 conn(0x55f3a73c1860 msgr2=0x55f3a73c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 -- 172.21.15.7:0/806935506 shutdown_connections 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.991+0000 7f00e17e5740 1 -- 172.21.15.7:0/806935506 wait complete. 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 -- start start 2024-03-19T00:45:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 Processor -- start 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3a73cee00 con 0x55f3a73c62f0 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f3a72e6310 con 0x55f3a73c6ce0 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3a73cef80 con 0x55f3a73cbe50 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00e17e5740 10 client.0 _ll_register_callbacks cb 0x55f3a72da2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52782/0 (socket says 172.21.15.7:52782) 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 -- 172.21.15.7:0/3104593381 learned_addr learned my addr 172.21.15.7:0/3104593381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dd191640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00d7fff640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 -- 172.21.15.7:0/3104593381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 msgr2=0x55f3a73d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 -- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 msgr2=0x55f3a73d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.992+0000 7f00dc990640 1 -- 172.21.15.7:0/3104593381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a703e3f0 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00dd191640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00dc990640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d07a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009900 tx=0x7f00cc005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3324bec91640b8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00b7fff640 1 -- 172.21.15.7:0/3104593381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc046070 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00b7fff640 1 -- 172.21.15.7:0/3104593381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f00cc0046c0 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d00114c0 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00b7fff640 1 -- 172.21.15.7:0/3104593381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00cc005d50 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 10 client.17088 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3a71087d0 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00b7fff640 1 -- 172.21.15.7:0/3104593381 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00d00114c0 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00b7fff640 1 -- 172.21.15.7:0/3104593381 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f00cc003930 con 0x55f3a73c62f0 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 10 client.17088 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 20 client.17088 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a73dc5c0 0x55f3a73dea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 4 client.17088 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.993+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a73883e0 con 0x55f3a73dc5c0 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.994+0000 7f00d7fff640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a73dc5c0 0x55f3a73dea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.994+0000 7f00d7fff640 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a73dc5c0 0x55f3a73dea60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcac386eaab4a3e server_cookie=f95d2029bc649c10 in_seq=0 out_seq=0 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.994+0000 7f00b7fff640 10 client.17088 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00b7fff640 1 -- 172.21.15.7:0/3104593381 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f3a73883e0 con 0x55f3a73dc5c0 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00b7fff640 10 client.17088 handle_command_reply: tid=0 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 client.17088 shutdown 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 msgr2=0x55f3a73d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d07a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009900 tx=0x7f00cc005250 comp rx=0 tx=0).stop 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 shutdown_connections 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a73dc5c0 0x55f3a73dea60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 --2- 172.21.15.7:0/3104593381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d07a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00b7fff640 0 client.17088 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 >> 172.21.15.7:0/3104593381 conn(0x55f3a73c1860 msgr2=0x55f3a73c0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 shutdown_connections 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 1 -- 172.21.15.7:0/3104593381 wait complete. 2024-03-19T00:45:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:29.995+0000 7f00e17e5740 20 client.17088 trim_cache size 0 max 16384 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 Processor -- start 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 -- start start 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c62f0 0x55f3a73c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81aedb0 con 0x55f3a73c6ce0 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81ac680 con 0x55f3a73cbe50 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.008+0000 7f00e17e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81aa4d0 con 0x55f3a73c62f0 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52790/0 (socket says 172.21.15.7:52790) 2024-03-19T00:45:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 -- 172.21.15.7:0/1338771393 learned_addr learned my addr 172.21.15.7:0/1338771393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00dc990640 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c62f0 0x55f3a73c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00dd191640 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 -- 172.21.15.7:0/1338771393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c62f0 msgr2=0x55f3a73c0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c62f0 0x55f3a73c0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 -- 172.21.15.7:0/1338771393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 msgr2=0x55f3a73e3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73e3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 -- 172.21.15.7:0/1338771393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a73cf0b0 con 0x55f3a73c6ce0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00d7fff640 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d7ce0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f00c4097fe0 tx=0x7f00c4007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb1455ddfbe8fed server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00c27fc640 1 -- 172.21.15.7:0/1338771393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c409bd10 con 0x55f3a73c6ce0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00c27fc640 1 -- 172.21.15.7:0/1338771393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c4097040 con 0x55f3a73c6ce0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.009+0000 7f00c27fc640 1 -- 172.21.15.7:0/1338771393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c409b870 con 0x55f3a73c6ce0 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- 172.21.15.7:0/1338771393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 msgr2=0x55f3a73d7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d7ce0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f00c4097fe0 tx=0x7f00c4007b90 comp rx=0 tx=0).stop 2024-03-19T00:45:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- 172.21.15.7:0/1338771393 shutdown_connections 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73cbe50 0x55f3a73e3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d7ce0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- 172.21.15.7:0/1338771393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73c62f0 0x55f3a73c0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- 172.21.15.7:0/1338771393 >> 172.21.15.7:0/1338771393 conn(0x55f3a73c1860 msgr2=0x55f3a73c0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- 172.21.15.7:0/1338771393 shutdown_connections 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- 172.21.15.7:0/1338771393 wait complete. 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- start start 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 Processor -- start 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81aab40 con 0x55f3a73c62f0 2024-03-19T00:45:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81abfb0 con 0x55f3a73c6ce0 2024-03-19T00:45:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.010+0000 7f00e17e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81ab210 con 0x55f3a73cbe50 2024-03-19T00:45:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00e17e5740 10 client.0 _ll_register_callbacks cb 0x7f00cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52804/0 (socket says 172.21.15.7:52804) 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 -- 172.21.15.7:0/249774210 learned_addr learned my addr 172.21.15.7:0/249774210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00d7fff640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dd191640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 -- 172.21.15.7:0/249774210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 msgr2=0x55f3a73d6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 -- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 msgr2=0x55f3a73d5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 -- 172.21.15.7:0/249774210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a73d7080 con 0x55f3a73c62f0 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dd191640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:45:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00dc990640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d55b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55f3a7397a20 tx=0x7f00cc005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa20dfaa9ff8fb23 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00b7fff640 1 -- 172.21.15.7:0/249774210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc05b060 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00b7fff640 1 -- 172.21.15.7:0/249774210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f00cc0037f0 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c4097040 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00b7fff640 1 -- 172.21.15.7:0/249774210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00cc046070 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00e17e5740 10 client.17094 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3a7442b50 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.011+0000 7f00b7fff640 1 -- 172.21.15.7:0/249774210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c4097040 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00b7fff640 1 -- 172.21.15.7:0/249774210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f00cc0046c0 con 0x55f3a73c62f0 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 10 client.17094 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 20 client.17094 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a7443530 0x55f3a7445950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 4 client.17094 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3a73379f0 con 0x55f3a7443530 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00d7fff640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a7443530 0x55f3a7445950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00d7fff640 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a7443530 0x55f3a7445950 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc5da0aafc517c2 server_cookie=bff05e42fbba5a70 in_seq=0 out_seq=0 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00b7fff640 10 client.17094 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00b7fff640 1 -- 172.21.15.7:0/249774210 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3a73379f0 con 0x55f3a7443530 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00b7fff640 10 client.17094 handle_command_reply: tid=0 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 1 client.17094 shutdown 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 msgr2=0x55f3a73d55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.012+0000 7f00e17e5740 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d55b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55f3a7397a20 tx=0x7f00cc005a60 comp rx=0 tx=0).stop 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 shutdown_connections 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3a73cbe50 0x55f3a73d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f3a73c6ce0 0x55f3a73d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55f3a7443530 0x55f3a7445950 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 --2- 172.21.15.7:0/249774210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3a73c62f0 0x55f3a73d55b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 >> 172.21.15.7:0/249774210 conn(0x55f3a73c1860 msgr2=0x55f3a73c0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00b7fff640 0 client.17094 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 shutdown_connections 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 1 -- 172.21.15.7:0/249774210 wait complete. 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00e17e5740 20 client.17094 trim_cache size 0 max 16384 2024-03-19T00:45:30.023 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f00c80a2800 msgr2=0x7f00c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f00c80a2800 0x7f00c80a4cc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f00d0023000 tx=0x7f00d001f000 comp rx=0 tx=0).stop 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 msgr2=0x7f00d8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 0x7f00d8073c10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f00c400ea60 tx=0x7f00c400ef20 comp rx=0 tx=0).stop 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 shutdown_connections 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f00c80a2800 0x7f00c80a4cc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00d8103400 0x7f00d8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d8102a40 0x7f00d8073c10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 --2- 172.21.15.7:0/4108984131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00d8101850 0x7f00d8075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 >> 172.21.15.7:0/4108984131 conn(0x7f00d80fcd80 msgr2=0x7f00d8105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 shutdown_connections 2024-03-19T00:45:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:30.013+0000 7f00debf4640 1 -- 172.21.15.7:0/4108984131 wait complete. 2024-03-19T00:45:30.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:30 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3749732860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:45:30.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:30 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3749732860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:45:30.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:30 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3749732860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:45:31.026 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:45:31.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:45:31.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 -- 172.21.15.7:0/385725751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4101850 msgr2=0x7fdcd4101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/385725751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4101850 0x7fdcd4101c70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002bc30 comp rx=0 tx=0).stop 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 -- 172.21.15.7:0/385725751 shutdown_connections 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/385725751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd4103400 0x7fdcd4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/385725751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 0x7fdcd4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/385725751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4101850 0x7fdcd4101c70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 -- 172.21.15.7:0/385725751 >> 172.21.15.7:0/385725751 conn(0x7fdcd40fcde0 msgr2=0x7fdcd40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 -- 172.21.15.7:0/385725751 shutdown_connections 2024-03-19T00:45:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.145+0000 7fdcdbcac640 1 -- 172.21.15.7:0/385725751 wait complete. 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 Processor -- start 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 -- start start 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd4101850 0x7fdcd4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 0x7fdcd4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 0x7fdcd419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd410d380 con 0x7fdcd4103400 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd410d200 con 0x7fdcd4102a40 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcdbcac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd410d500 con 0x7fdcd4101850 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 0x7fdcd419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 0x7fdcd4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcd9a48640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd4101850 0x7fdcd4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 0x7fdcd419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52820/0 (socket says 172.21.15.7:52820) 2024-03-19T00:45:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 -- 172.21.15.7:0/3977280422 learned_addr learned my addr 172.21.15.7:0/3977280422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 0x7fdcd4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43390/0 (socket says 172.21.15.7:43390) 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 -- 172.21.15.7:0/3977280422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd4101850 msgr2=0x7fdcd4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd4101850 0x7fdcd4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 -- 172.21.15.7:0/3977280422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 msgr2=0x7fdcd4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 0x7fdcd4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 -- 172.21.15.7:0/3977280422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd419f6b0 con 0x7fdcd4103400 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.146+0000 7fdcda249640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 0x7fdcd419f040 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00eb10 tx=0x7fdccc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.147+0000 7fdccaffd640 1 -- 172.21.15.7:0/3977280422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004b20 con 0x7fdcd4103400 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.147+0000 7fdccaffd640 1 -- 172.21.15.7:0/3977280422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdccc004cc0 con 0x7fdcd4103400 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.147+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd419f9a0 con 0x7fdcd4103400 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.147+0000 7fdccaffd640 1 -- 172.21.15.7:0/3977280422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc010690 con 0x7fdcd4103400 2024-03-19T00:45:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.147+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd419fef0 con 0x7fdcd4103400 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.149+0000 7fdccaffd640 1 -- 172.21.15.7:0/3977280422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fdccc005020 con 0x7fdcd4103400 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.149+0000 7fdcde974740 1 Processor -- start 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 -- start start 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f4b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559dc3eab160 con 0x559dc3f46d90 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559dc3e53e10 con 0x559dc3f4bf00 2024-03-19T00:45:31.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcde974740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559dc3ea7200 con 0x559dc3f463a0 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdccaffd640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdcc00a2800 0x7fdcc00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9a48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f4b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:37786/0 (socket says 172.21.15.7:37786) 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 -- 172.21.15.7:0/4159614995 learned_addr learned my addr 172.21.15.7:0/4159614995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcda249640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdcc00a2800 0x7fdcc00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdccaffd640 1 -- 172.21.15.7:0/3977280422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdccc0cf4e0 con 0x7fdcd4103400 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 -- 172.21.15.7:0/4159614995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 msgr2=0x559dc3f4e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f4e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 -- 172.21.15.7:0/4159614995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 msgr2=0x559dc3f4b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f4b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 -- 172.21.15.7:0/4159614995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc3c877e0 con 0x559dc3f463a0 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.150+0000 7fdcd9247640 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f467c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40058c0 tx=0x7fdcc4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff8841b0972c27f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.151+0000 7fdcda249640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdcc00a2800 0x7fdcc00a4cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0ccc20 tx=0x7fdccc090fa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.151+0000 7fdcbffff640 1 -- 172.21.15.7:0/4159614995 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4004030 con 0x559dc3f463a0 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.151+0000 7fdcbffff640 1 -- 172.21.15.7:0/4159614995 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc4005020 con 0x559dc3f463a0 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.151+0000 7fdcbffff640 1 -- 172.21.15.7:0/4159614995 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4004b20 con 0x559dc3f463a0 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.151+0000 7fdcde974740 1 -- 172.21.15.7:0/4159614995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 msgr2=0x559dc3f467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.151+0000 7fdcde974740 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f467c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40058c0 tx=0x7fdcc4005850 comp rx=0 tx=0).stop 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 -- 172.21.15.7:0/4159614995 shutdown_connections 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f4e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f4b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 --2- 172.21.15.7:0/4159614995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f467c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 -- 172.21.15.7:0/4159614995 >> 172.21.15.7:0/4159614995 conn(0x559dc3f41910 msgr2=0x559dc3f41d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 -- 172.21.15.7:0/4159614995 shutdown_connections 2024-03-19T00:45:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 -- 172.21.15.7:0/4159614995 wait complete. 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 -- start start 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 Processor -- start 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f50d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f57d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.152+0000 7fdcde974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559dc3f4eeb0 con 0x559dc3f46d90 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcde974740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559dc3e671a0 con 0x559dc3f4bf00 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcde974740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559dc3f4f030 con 0x559dc3f463a0 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcde974740 10 client.0 _ll_register_callbacks cb 0x559dc3bbec60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9a48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f50d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:37802/0 (socket says 172.21.15.7:37802) 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 -- 172.21.15.7:0/1034754770 learned_addr learned my addr 172.21.15.7:0/1034754770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f57d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 -- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 msgr2=0x559dc3f57d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f57d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 -- 172.21.15.7:0/1034754770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 msgr2=0x559dc3f50d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f50d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 -- 172.21.15.7:0/1034754770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc3ce7f70 con 0x559dc3f463a0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcd9247640 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f507d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4004da0 tx=0x7fdcc4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f942909f1d433e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/1034754770 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4004e00 con 0x559dc3f463a0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc4005020 con 0x559dc3f463a0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/1034754770 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdcc4019040 con 0x559dc3f463a0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/1034754770 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcc4037050 con 0x559dc3f463a0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.153+0000 7fdcde974740 10 client.26863 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dc3b7f170 con 0x559dc3f463a0 2024-03-19T00:45:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/1034754770 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc4005020 con 0x559dc3f463a0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/1034754770 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdcc402c020 con 0x559dc3f463a0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 10 client.26863 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 20 client.26863 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3f5c3c0 0x559dc3f5e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 4 client.26863 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dc3eb86a0 con 0x559dc3f5c3c0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3f5c3c0 0x559dc3f5e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.155+0000 7fdcbd7fa640 10 client.26863 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.154+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3f5c3c0 0x559dc3f5e820 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b846e9298de41e8 server_cookie=594de0522a4e718d in_seq=0 out_seq=0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.155+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/1034754770 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559dc3eb86a0 con 0x559dc3f5c3c0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.155+0000 7fdcbd7fa640 10 client.26863 handle_command_reply: tid=0 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.155+0000 7fdcde974740 1 client.26863 shutdown 2024-03-19T00:45:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 msgr2=0x559dc3f507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f507d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4004da0 tx=0x7fdcc4005f90 comp rx=0 tx=0).stop 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 shutdown_connections 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f4bf00 0x559dc3f57d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3f5c3c0 0x559dc3f5e820 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f46d90 0x559dc3f50d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 --2- 172.21.15.7:0/1034754770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3f507d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcbd7fa640 0 client.26863 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 >> 172.21.15.7:0/1034754770 conn(0x559dc3f41910 msgr2=0x559dc3f40b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 shutdown_connections 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 1 -- 172.21.15.7:0/1034754770 wait complete. 2024-03-19T00:45:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.156+0000 7fdcde974740 20 client.26863 trim_cache size 0 max 16384 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.173+0000 7fdcde974740 1 Processor -- start 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 -- start start 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 0x559dc3f50a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3f630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f4bf00 0x559dc3f63620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41acb90 con 0x559dc3f463a0 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41aa460 con 0x559dc3f46d90 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcde974740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41a82b0 con 0x559dc3f4bf00 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 0x559dc3f50a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3f630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 0x559dc3f50a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52860/0 (socket says 172.21.15.7:52860) 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9247640 1 -- 172.21.15.7:0/1862913844 learned_addr learned my addr 172.21.15.7:0/1862913844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9247640 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f4bf00 0x559dc3f63620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 -- 172.21.15.7:0/1862913844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f4bf00 msgr2=0x559dc3f63620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f4bf00 0x559dc3f63620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 -- 172.21.15.7:0/1862913844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 msgr2=0x559dc3f50a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 0x559dc3f50a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 -- 172.21.15.7:0/1862913844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc3f3c990 con 0x559dc3f46d90 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9247640 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 0x559dc3f50a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3f630e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0036040 tx=0x7fdcd002c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba4dc7bfa7bdfe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.174+0000 7fdcbf7fe640 1 -- 172.21.15.7:0/1862913844 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd002cb00 con 0x559dc3f46d90 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcbf7fe640 1 -- 172.21.15.7:0/1862913844 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcd0004040 con 0x559dc3f46d90 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcbf7fe640 1 -- 172.21.15.7:0/1862913844 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0005430 con 0x559dc3f46d90 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 -- 172.21.15.7:0/1862913844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 msgr2=0x559dc3f630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3f630e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0036040 tx=0x7fdcd002c3e0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 -- 172.21.15.7:0/1862913844 shutdown_connections 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f4bf00 0x559dc3f63620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3f630e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 --2- 172.21.15.7:0/1862913844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f463a0 0x559dc3f50a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 -- 172.21.15.7:0/1862913844 >> 172.21.15.7:0/1862913844 conn(0x559dc3f41910 msgr2=0x559dc3f45400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 -- 172.21.15.7:0/1862913844 shutdown_connections 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 -- 172.21.15.7:0/1862913844 wait complete. 2024-03-19T00:45:31.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.175+0000 7fdcde974740 1 -- start start 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 Processor -- start 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3fbeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3fbf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f4bf00 0x559dc3fc60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41a8920 con 0x559dc3f4bf00 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41a9d90 con 0x559dc3f46d90 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41a8ff0 con 0x559dc3f463a0 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcde974740 10 client.0 _ll_register_callbacks cb 0x7fdccc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9247640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f4bf00 0x559dc3fc60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3fbf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3fbf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43434/0 (socket says 172.21.15.7:43434) 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 -- 172.21.15.7:0/2255699436 learned_addr learned my addr 172.21.15.7:0/2255699436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9247640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3fbeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 -- 172.21.15.7:0/2255699436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 msgr2=0x559dc3fbeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3fbeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 -- 172.21.15.7:0/2255699436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f4bf00 msgr2=0x559dc3fc60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f4bf00 0x559dc3fc60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9a48640 1 -- 172.21.15.7:0/2255699436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc3fc66e0 con 0x559dc3f46d90 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.176+0000 7fdcd9247640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f4bf00 0x559dc3fc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcd9a48640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3fbf0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdcd002bc70 tx=0x7fdcd00041f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0b1a35939f950e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/2255699436 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0045030 con 0x559dc3f46d90 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/2255699436 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdcd004b420 con 0x559dc3f46d90 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd0004040 con 0x559dc3f46d90 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/2255699436 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcd004b620 con 0x559dc3f46d90 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 10 client.26957 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dc3fc7a50 con 0x559dc3f46d90 2024-03-19T00:45:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/2255699436 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcd0004040 con 0x559dc3f46d90 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/2255699436 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdcd00047e0 con 0x559dc3f46d90 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 10 client.26957 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 20 client.26957 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3fc84a0 0x559dc3fca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 4 client.26957 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559dc3eb29f0 con 0x559dc3fc84a0 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.177+0000 7fdcd9247640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3fc84a0 0x559dc3fca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.178+0000 7fdcd9247640 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3fc84a0 0x559dc3fca8c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f864f35075b457b2 server_cookie=559dfbcc9785cd08 in_seq=0 out_seq=0 2024-03-19T00:45:31.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.178+0000 7fdcbd7fa640 10 client.26957 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcbd7fa640 1 -- 172.21.15.7:0/2255699436 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559dc3eb29f0 con 0x559dc3fc84a0 2024-03-19T00:45:31.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcbd7fa640 10 client.26957 handle_command_reply: tid=0 2024-03-19T00:45:31.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 client.26957 shutdown 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 msgr2=0x559dc3fbf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3fbf0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdcd002bc70 tx=0x7fdcd00041f0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 shutdown_connections 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x559dc3fc84a0 0x559dc3fca8c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc3f4bf00 0x559dc3fc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559dc3f46d90 0x559dc3fbf0b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcbd7fa640 0 client.26957 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 --2- 172.21.15.7:0/2255699436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dc3f463a0 0x559dc3fbeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 >> 172.21.15.7:0/2255699436 conn(0x559dc3f41910 msgr2=0x559dc3f402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 shutdown_connections 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 1 -- 172.21.15.7:0/2255699436 wait complete. 2024-03-19T00:45:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.179+0000 7fdcde974740 20 client.26957 trim_cache size 0 max 16384 2024-03-19T00:45:31.187 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:45:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:45:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2be0d1b9-bb91-401d-927e-e943e1ff32f7", 2024-03-19T00:45:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:45:31.187 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdcc00a2800 msgr2=0x7fdcc00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdcc00a2800 0x7fdcc00a4cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0ccc20 tx=0x7fdccc090fa0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 msgr2=0x7fdcd419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 0x7fdcd419f040 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00eb10 tx=0x7fdccc00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 shutdown_connections 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdcc00a2800 0x7fdcc00a4cc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd4103400 0x7fdcd419f040 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcd4102a40 0x7fdcd4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 --2- 172.21.15.7:0/3977280422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd4101850 0x7fdcd4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 >> 172.21.15.7:0/3977280422 conn(0x7fdcd40fcde0 msgr2=0x7fdcd4105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 shutdown_connections 2024-03-19T00:45:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.180+0000 7fdcdbcac640 1 -- 172.21.15.7:0/3977280422 wait complete. 2024-03-19T00:45:31.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:45:31.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 -- 172.21.15.7:0/547855956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c100740 msgr2=0x7fb13c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 --2- 172.21.15.7:0/547855956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c100740 0x7fb13c100b60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009900 tx=0x7fb12c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 -- 172.21.15.7:0/547855956 shutdown_connections 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 --2- 172.21.15.7:0/547855956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c1022f0 0x7fb13c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 --2- 172.21.15.7:0/547855956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c101930 0x7fb13c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 --2- 172.21.15.7:0/547855956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c100740 0x7fb13c100b60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 -- 172.21.15.7:0/547855956 >> 172.21.15.7:0/547855956 conn(0x7fb13c0fbcb0 msgr2=0x7fb13c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 -- 172.21.15.7:0/547855956 shutdown_connections 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 -- 172.21.15.7:0/547855956 wait complete. 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 Processor -- start 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.298+0000 7fb1415c6640 1 -- start start 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c100740 0x7fb13c10b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c101930 0x7fb13c109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 0x7fb13c109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10d390 con 0x7fb13c1022f0 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10d210 con 0x7fb13c100740 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10d510 con 0x7fb13c101930 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 0x7fb13c109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 0x7fb13c109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52888/0 (socket says 172.21.15.7:52888) 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 -- 172.21.15.7:0/3518946815 learned_addr learned my addr 172.21.15.7:0/3518946815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb133fff640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c101930 0x7fb13c109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13affd640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c100740 0x7fb13c10b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 -- 172.21.15.7:0/3518946815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c101930 msgr2=0x7fb13c109700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c101930 0x7fb13c109700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 -- 172.21.15.7:0/3518946815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c100740 msgr2=0x7fb13c10b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c100740 0x7fb13c10b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 -- 172.21.15.7:0/3518946815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c10a4d0 con 0x7fb13c1022f0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb133fff640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c101930 0x7fb13c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb13b7fe640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 0x7fb13c109c40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb13400eb10 tx=0x7fb13400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb138ff9640 1 -- 172.21.15.7:0/3518946815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134004b20 con 0x7fb13c1022f0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb138ff9640 1 -- 172.21.15.7:0/3518946815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb134004cc0 con 0x7fb13c1022f0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c1a1ba0 con 0x7fb13c1022f0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb138ff9640 1 -- 172.21.15.7:0/3518946815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134010690 con 0x7fb13c1022f0 2024-03-19T00:45:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.299+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c1a20f0 con 0x7fb13c1022f0 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb138ff9640 1 -- 172.21.15.7:0/3518946815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb134005020 con 0x7fb13c1022f0 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 Processor -- start 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 -- start start 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5905a50 0x5608a5905e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a590af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a590da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5608a586a500 con 0x5608a590b5b0 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5608a5812be0 con 0x5608a5906440 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb1441b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5608a58665a0 con 0x5608a5905a50 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a590da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb13affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a590af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a590da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52898/0 (socket says 172.21.15.7:52898) 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb133fff640 1 -- 172.21.15.7:0/2617958976 learned_addr learned my addr 172.21.15.7:0/2617958976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb138ff9640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb1280a2800 0x7fb1280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.302+0000 7fb133fff640 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5905a50 0x5608a5905e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb13b7fe640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb1280a2800 0x7fb1280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb133fff640 1 -- 172.21.15.7:0/2617958976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5905a50 msgr2=0x5608a5905e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb133fff640 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5905a50 0x5608a5905e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb133fff640 1 -- 172.21.15.7:0/2617958976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 msgr2=0x5608a590af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb133fff640 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a590af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb133fff640 1 -- 172.21.15.7:0/2617958976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608a5834000 con 0x5608a590b5b0 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb13affd640 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a590af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb138ff9640 1 -- 172.21.15.7:0/3518946815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb1340cf4e0 con 0x7fb13c1022f0 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb133fff640 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a590da40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb124009fd0 tx=0x7fb124009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfc7f2880124175 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb1317fa640 1 -- 172.21.15.7:0/2617958976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12400fc50 con 0x5608a590b5b0 2024-03-19T00:45:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb13b7fe640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb1280a2800 0x7fb1280a4cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb1340984f0 tx=0x7fb1340668e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb1317fa640 1 -- 172.21.15.7:0/2617958976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb12400fdf0 con 0x5608a590b5b0 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.303+0000 7fb1317fa640 1 -- 172.21.15.7:0/2617958976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12401d5a0 con 0x5608a590b5b0 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- 172.21.15.7:0/2617958976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 msgr2=0x5608a590da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a590da40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb124009fd0 tx=0x7fb124009ba0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- 172.21.15.7:0/2617958976 shutdown_connections 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a590da40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a590af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- 172.21.15.7:0/2617958976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5905a50 0x5608a5905e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- 172.21.15.7:0/2617958976 >> 172.21.15.7:0/2617958976 conn(0x5608a5900fc0 msgr2=0x5608a59013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- 172.21.15.7:0/2617958976 shutdown_connections 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- 172.21.15.7:0/2617958976 wait complete. 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- start start 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 Processor -- start 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a590fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a59173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.304+0000 7fb1441b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5608a590e560 con 0x5608a5905a50 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1441b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5608a5826160 con 0x5608a5906440 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1441b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5608a590e6e0 con 0x5608a590b5b0 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1441b2740 10 client.0 _ll_register_callbacks cb 0x5608a5862a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a590fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a590fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52912/0 (socket says 172.21.15.7:52912) 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 -- 172.21.15.7:0/3144289953 learned_addr learned my addr 172.21.15.7:0/3144289953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb13affd640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a59173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 -- 172.21.15.7:0/3144289953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 msgr2=0x5608a59173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a59173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 -- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 msgr2=0x5608a59103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 -- 172.21.15.7:0/3144289953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608a56487d0 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb133fff640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a590fe70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb1240057c0 tx=0x7fb1240096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c082574a5bf307 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1227fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1227fc640 1 -- 172.21.15.7:0/3144289953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12400eae0 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1227fc640 1 -- 172.21.15.7:0/3144289953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb124004020 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12400fdf0 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1227fc640 1 -- 172.21.15.7:0/3144289953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb124004220 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1441b2740 10 client.17124 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.305+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608a557e3f0 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1227fc640 1 -- 172.21.15.7:0/3144289953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb12400fdf0 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1227fc640 1 -- 172.21.15.7:0/3144289953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb12402b020 con 0x5608a5905a50 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1441b2740 10 client.17124 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1441b2740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1441b2740 20 client.17124 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a591b9d0 0x5608a591ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1441b2740 4 client.17124 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608a5877a40 con 0x5608a591b9d0 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb13affd640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a591b9d0 0x5608a591ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb1227fc640 10 client.17124 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.306+0000 7fb13affd640 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a591b9d0 0x5608a591ddf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4724c3cc61058fb server_cookie=6968085e345c3989 in_seq=0 out_seq=0 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.307+0000 7fb1227fc640 1 -- 172.21.15.7:0/3144289953 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5608a5877a40 con 0x5608a591b9d0 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.307+0000 7fb1227fc640 10 client.17124 handle_command_reply: tid=0 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.307+0000 7fb1441b2740 1 client.17124 shutdown 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 msgr2=0x5608a590fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a590fe70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb1240057c0 tx=0x7fb1240096c0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 shutdown_connections 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a59173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a591b9d0 0x5608a591ddf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3144289953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a590fe70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1227fc640 0 client.17124 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 >> 172.21.15.7:0/3144289953 conn(0x5608a5900fc0 msgr2=0x5608a59001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 shutdown_connections 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 1 -- 172.21.15.7:0/3144289953 wait complete. 2024-03-19T00:45:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.308+0000 7fb1441b2740 20 client.17124 trim_cache size 0 max 16384 2024-03-19T00:45:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 Processor -- start 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 -- start start 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a5910190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59173b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a58ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1aee50 con 0x5608a5905a50 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ac720 con 0x5608a5906440 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.325+0000 7fb1441b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1aa570 con 0x5608a590b5b0 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a5910190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a5910190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52922/0 (socket says 172.21.15.7:52922) 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb13affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 -- 172.21.15.7:0/3697824317 learned_addr learned my addr 172.21.15.7:0/3697824317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a58ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 -- 172.21.15.7:0/3697824317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 msgr2=0x5608a58ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a58ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 -- 172.21.15.7:0/3697824317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 msgr2=0x5608a59173b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59173b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 -- 172.21.15.7:0/3697824317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608a553e0f0 con 0x5608a5905a50 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb13affd640 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb133fff640 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a5910190 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb1240059f0 tx=0x7fb124005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648a309f46d51a55 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb130ff9640 1 -- 172.21.15.7:0/3697824317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124023070 con 0x5608a5905a50 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb130ff9640 1 -- 172.21.15.7:0/3697824317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb124024040 con 0x5608a5905a50 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.326+0000 7fb130ff9640 1 -- 172.21.15.7:0/3697824317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124004f80 con 0x5608a5905a50 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- 172.21.15.7:0/3697824317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 msgr2=0x5608a5910190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a5910190 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb1240059f0 tx=0x7fb124005b90 comp rx=0 tx=0).stop 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- 172.21.15.7:0/3697824317 shutdown_connections 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a590b5b0 0x5608a58ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5906440 0x5608a59173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- 172.21.15.7:0/3697824317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a5905a50 0x5608a5910190 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- 172.21.15.7:0/3697824317 >> 172.21.15.7:0/3697824317 conn(0x5608a5900fc0 msgr2=0x5608a590fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- 172.21.15.7:0/3697824317 shutdown_connections 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- 172.21.15.7:0/3697824317 wait complete. 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- start start 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 Processor -- start 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 0x5608a5914c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5906440 0x5608a5915150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a5916160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1aabe0 con 0x5608a590b5b0 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ac050 con 0x5608a5905a50 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.327+0000 7fb1441b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ab2b0 con 0x5608a5906440 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1441b2740 10 client.0 _ll_register_callbacks cb 0x7fb1340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a5916160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 0x5608a5914c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 0x5608a5914c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43498/0 (socket says 172.21.15.7:43498) 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 -- 172.21.15.7:0/334481622 learned_addr learned my addr 172.21.15.7:0/334481622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb13affd640 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5906440 0x5608a5915150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 -- 172.21.15.7:0/334481622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5906440 msgr2=0x5608a5915150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5906440 0x5608a5915150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 -- 172.21.15.7:0/334481622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 msgr2=0x5608a5916160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a5916160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 -- 172.21.15.7:0/334481622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608a5916750 con 0x5608a5905a50 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb133fff640 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 0x5608a5914c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb1240046a0 tx=0x7fb124005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db0bce485c1f4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1227fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1227fc640 1 -- 172.21.15.7:0/334481622 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124040030 con 0x5608a5905a50 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1227fc640 1 -- 172.21.15.7:0/334481622 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb124011250 con 0x5608a5905a50 2024-03-19T00:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124024040 con 0x5608a5905a50 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1227fc640 1 -- 172.21.15.7:0/334481622 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb124023030 con 0x5608a5905a50 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1441b2740 10 client.26975 fetch_fsmap learned FSMap version 27 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.328+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608a5981e10 con 0x5608a5905a50 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1227fc640 1 -- 172.21.15.7:0/334481622 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb124024040 con 0x5608a5905a50 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1227fc640 1 -- 172.21.15.7:0/334481622 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb12402b070 con 0x5608a5905a50 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 10 client.26975 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 20 client.26975 populate_metadata read hostname 'smithi007' 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a5982580 0x5608a59849a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 4 client.26975 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608a5871d90 con 0x5608a5982580 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb13affd640 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a5982580 0x5608a59849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb13affd640 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a5982580 0x5608a59849a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8880608d29e1bc server_cookie=bb70b7225c4e64ee in_seq=0 out_seq=0 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1227fc640 10 client.26975 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1227fc640 1 -- 172.21.15.7:0/334481622 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5608a5871d90 con 0x5608a5982580 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1227fc640 10 client.26975 handle_command_reply: tid=0 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.329+0000 7fb1441b2740 1 client.26975 shutdown 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 msgr2=0x5608a5914c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 0x5608a5914c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb1240046a0 tx=0x7fb124005b90 comp rx=0 tx=0).stop 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 shutdown_connections 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5608a5982580 0x5608a59849a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5608a590b5b0 0x5608a5916160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1227fc640 0 client.26975 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608a5906440 0x5608a5915150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 --2- 172.21.15.7:0/334481622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608a5905a50 0x5608a5914c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 >> 172.21.15.7:0/334481622 conn(0x5608a5900fc0 msgr2=0x5608a58ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 shutdown_connections 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 1 -- 172.21.15.7:0/334481622 wait complete. 2024-03-19T00:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1441b2740 20 client.26975 trim_cache size 0 max 16384 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "2be0d1b9-bb91-401d-927e-e943e1ff32f7": { 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2be0d1b9-bb91-401d-927e-e943e1ff32f7", 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:45:31.338 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb1280a2800 msgr2=0x7fb1280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb1280a2800 0x7fb1280a4cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb1340984f0 tx=0x7fb1340668e0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 msgr2=0x7fb13c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 0x7fb13c109c40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb13400eb10 tx=0x7fb13400c2d0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 shutdown_connections 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb1280a2800 0x7fb1280a4cc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb13c1022f0 0x7fb13c109c40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb13c101930 0x7fb13c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 --2- 172.21.15.7:0/3518946815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c100740 0x7fb13c10b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 >> 172.21.15.7:0/3518946815 conn(0x7fb13c0fbcb0 msgr2=0x7fb13c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 shutdown_connections 2024-03-19T00:45:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:45:31.330+0000 7fb1415c6640 1 -- 172.21.15.7:0/3518946815 wait complete. 2024-03-19T00:45:31.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:31 smithi007 ceph-mon[62854]: pgmap v578: 129 pgs: 129 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:31.342 INFO:tasks.cephfs.filesystem:scrub status for tag:2be0d1b9-bb91-401d-927e-e943e1ff32f7 - {'path': '/', 'tag': '2be0d1b9-bb91-401d-927e-e943e1ff32f7', 'options': 'recursive,force'} 2024-03-19T00:45:31.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:31 smithi044 ceph-mon[66366]: pgmap v578: 129 pgs: 129 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:31 smithi204 ceph-mon[66572]: pgmap v578: 129 pgs: 129 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:32 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:45:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:32 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:32 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:45:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:32 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:45:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:32 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:32 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:32 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:45:32.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:32 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:32 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:45:32.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:32 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:45:32.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:32 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:32 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:32 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:45:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:32 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:32 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:45:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:32 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:45:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:32 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:32.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:32 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:45:33.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:33 smithi044 ceph-mon[66366]: pgmap v579: 129 pgs: 129 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:33.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:33 smithi007 ceph-mon[62854]: pgmap v579: 129 pgs: 129 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:33.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:33 smithi204 ceph-mon[66572]: pgmap v579: 129 pgs: 129 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:45:35.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:35 smithi044 ceph-mon[66366]: pgmap v580: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-03-19T00:45:35.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:35 smithi007 ceph-mon[62854]: pgmap v580: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-03-19T00:45:35.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:35 smithi204 ceph-mon[66572]: pgmap v580: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-03-19T00:45:37.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:37 smithi044 ceph-mon[66366]: pgmap v581: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:45:37.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:37 smithi007 ceph-mon[62854]: pgmap v581: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:45:37.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:37 smithi204 ceph-mon[66572]: pgmap v581: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:45:39.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:39 smithi044 ceph-mon[66366]: pgmap v582: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:45:39.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:39 smithi007 ceph-mon[62854]: pgmap v582: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:45:39.764 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:39 smithi204 ceph-mon[66572]: pgmap v582: 129 pgs: 129 active+clean; 2.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:45:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:41 smithi044 ceph-mon[66366]: pgmap v583: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:45:41.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:41 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:45:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:41 smithi007 ceph-mon[62854]: pgmap v583: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:45:41.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:41 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:45:41.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:41 smithi204 ceph-mon[66572]: pgmap v583: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:45:41.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:41 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:45:42.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:42 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:45:42.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:42 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:45:42.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:42 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:45:43.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:43 smithi044 ceph-mon[66366]: pgmap v584: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:45:43.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:43 smithi204 ceph-mon[66572]: pgmap v584: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:45:43.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:43 smithi007 ceph-mon[62854]: pgmap v584: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:45:45.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:45 smithi044 ceph-mon[66366]: pgmap v585: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:45.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:45 smithi007 ceph-mon[62854]: pgmap v585: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:45.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:45 smithi204 ceph-mon[66572]: pgmap v585: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:47.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:47 smithi044 ceph-mon[66366]: pgmap v586: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:47.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:47 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:45:47.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:47 smithi007 ceph-mon[62854]: pgmap v586: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:47.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:47 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:45:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:47 smithi204 ceph-mon[66572]: pgmap v586: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:45:47.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:47 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: Adjusting osd_memory_target on smithi204 to 176.1M 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: Unable to set osd_memory_target on smithi204 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:45:48.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: Adjusting osd_memory_target on smithi204 to 176.1M 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: Unable to set osd_memory_target on smithi204 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:45:48.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: Adjusting osd_memory_target on smithi204 to 176.1M 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: Unable to set osd_memory_target on smithi204 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:45:48.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:45:49.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:49 smithi007 ceph-mon[62854]: pgmap v587: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:49 smithi204 ceph-mon[66572]: pgmap v587: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:49 smithi044 ceph-mon[66366]: pgmap v587: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:51 smithi007 ceph-mon[62854]: pgmap v588: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:51 smithi204 ceph-mon[66572]: pgmap v588: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:51 smithi044 ceph-mon[66366]: pgmap v588: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:45:53.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:53 smithi007 ceph-mon[62854]: pgmap v589: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:54.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:53 smithi204 ceph-mon[66572]: pgmap v589: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:53 smithi044 ceph-mon[66366]: pgmap v589: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:55.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:55 smithi007 ceph-mon[62854]: pgmap v590: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:45:56.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:55 smithi204 ceph-mon[66572]: pgmap v590: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:45:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:55 smithi044 ceph-mon[66366]: pgmap v590: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:45:57.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:57 smithi007 ceph-mon[62854]: pgmap v591: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:58.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:57 smithi204 ceph-mon[66572]: pgmap v591: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:57 smithi044 ceph-mon[66366]: pgmap v591: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:45:59.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:45:59 smithi007 ceph-mon[62854]: pgmap v592: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:00.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:45:59 smithi204 ceph-mon[66572]: pgmap v592: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:45:59 smithi044 ceph-mon[66366]: pgmap v592: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:01.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:46:01.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:46:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 -- 172.21.15.7:0/4005523828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c1022f0 msgr2=0x7f2d4c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/4005523828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c108b80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009ca0 tx=0x7f2d4402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 -- 172.21.15.7:0/4005523828 shutdown_connections 2024-03-19T00:46:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/4005523828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c108b80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/4005523828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c101930 0x7f2d4c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/4005523828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d4c100740 0x7f2d4c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 -- 172.21.15.7:0/4005523828 >> 172.21.15.7:0/4005523828 conn(0x7f2d4c0fbcd0 msgr2=0x7f2d4c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 -- 172.21.15.7:0/4005523828 shutdown_connections 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.459+0000 7f2d53cb8640 1 -- 172.21.15.7:0/4005523828 wait complete. 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 Processor -- start 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 -- start start 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 0x7f2d4c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d4c101930 0x7f2d4c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c10d390 con 0x7f2d4c1022f0 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c10d210 con 0x7f2d4c101930 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d53cb8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c10d510 con 0x7f2d4c100740 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 0x7f2d4c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 0x7f2d4c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:53280/0 (socket says 172.21.15.7:53280) 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 -- 172.21.15.7:0/1129872108 learned_addr learned my addr 172.21.15.7:0/1129872108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d52255640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51253640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d4c101930 0x7f2d4c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 -- 172.21.15.7:0/1129872108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d4c101930 msgr2=0x7f2d4c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d4c101930 0x7f2d4c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 -- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c1022f0 msgr2=0x7f2d4c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 -- 172.21.15.7:0/1129872108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c19f610 con 0x7f2d4c100740 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d52255640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.460+0000 7f2d51a54640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 0x7f2d4c1979f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e9c0 tx=0x7f2d4800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.461+0000 7f2d42ffd640 1 -- 172.21.15.7:0/1129872108 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800cd80 con 0x7f2d4c100740 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.461+0000 7f2d42ffd640 1 -- 172.21.15.7:0/1129872108 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d480035f0 con 0x7f2d4c100740 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.461+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c19f900 con 0x7f2d4c100740 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.461+0000 7f2d42ffd640 1 -- 172.21.15.7:0/1129872108 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48010690 con 0x7f2d4c100740 2024-03-19T00:46:01.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.461+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c19fea0 con 0x7f2d4c100740 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.463+0000 7f2d42ffd640 1 -- 172.21.15.7:0/1129872108 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f2d480037a0 con 0x7f2d4c100740 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.463+0000 7f2d56978740 1 Processor -- start 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.463+0000 7f2d56978740 1 -- start start 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d56978740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c748ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d56978740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c748ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d56978740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7492a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d56978740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5643c73ef500 con 0x5643c748aa50 2024-03-19T00:46:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d56978740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5643c7397be0 con 0x5643c74905b0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d56978740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643c73eb5a0 con 0x5643c748b440 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d42ffd640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2d380a2880 0x7f2d380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d52255640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c748ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d52255640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c748ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37914/0 (socket says 172.21.15.7:37914) 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d52255640 1 -- 172.21.15.7:0/1676110808 learned_addr learned my addr 172.21.15.7:0/1676110808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51a54640 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c748ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7492a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d52255640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2d380a2880 0x7f2d380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 -- 172.21.15.7:0/1676110808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 msgr2=0x5643c748ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c748ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 -- 172.21.15.7:0/1676110808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 msgr2=0x5643c748ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d42ffd640 1 -- 172.21.15.7:0/1129872108 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d480cf130 con 0x7f2d4c100740 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c748ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 -- 172.21.15.7:0/1676110808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c73b9000 con 0x5643c74905b0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.464+0000 7f2d51253640 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7492a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009bf0 tx=0x7f2d3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4618f4888859d775 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d52255640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2d380a2880 0x7f2d380a4d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5643c748bca0 tx=0x7f2d4402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d37fff640 1 -- 172.21.15.7:0/1676110808 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004050 con 0x5643c74905b0 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d37fff640 1 -- 172.21.15.7:0/1676110808 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d3c0054d0 con 0x5643c74905b0 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d37fff640 1 -- 172.21.15.7:0/1676110808 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004cf0 con 0x5643c74905b0 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 -- 172.21.15.7:0/1676110808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 msgr2=0x5643c7492a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7492a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009bf0 tx=0x7f2d3c011810 comp rx=0 tx=0).stop 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 -- 172.21.15.7:0/1676110808 shutdown_connections 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7492a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c748ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 --2- 172.21.15.7:0/1676110808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c748ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.465+0000 7f2d56978740 1 -- 172.21.15.7:0/1676110808 >> 172.21.15.7:0/1676110808 conn(0x5643c7485fc0 msgr2=0x5643c74863c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 -- 172.21.15.7:0/1676110808 shutdown_connections 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 -- 172.21.15.7:0/1676110808 wait complete. 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 -- start start 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 Processor -- start 2024-03-19T00:46:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c7494ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748b440 0x5643c74953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c74905b0 0x5643c749c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5643c7493560 con 0x5643c748aa50 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5643c73ab160 con 0x5643c748b440 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643c74936e0 con 0x5643c74905b0 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.466+0000 7f2d56978740 10 client.0 _ll_register_callbacks cb 0x5643c73e7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c7494ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51253640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748b440 0x5643c74953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d52255640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c74905b0 0x5643c749c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c7494ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37920/0 (socket says 172.21.15.7:37920) 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51253640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748b440 0x5643c74953e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47738/0 (socket says 172.21.15.7:47738) 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 -- 172.21.15.7:0/379110479 learned_addr learned my addr 172.21.15.7:0/379110479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 -- 172.21.15.7:0/379110479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c74905b0 msgr2=0x5643c749c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c74905b0 0x5643c749c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 -- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748b440 msgr2=0x5643c74953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748b440 0x5643c74953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 -- 172.21.15.7:0/379110479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c71cd7d0 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d51a54640 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c7494ea0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2d4809b6a0 tx=0x7f2d4809b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a5115a579a17a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d357fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d357fa640 1 -- 172.21.15.7:0/379110479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4809b710 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3c0054d0 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d357fa640 1 -- 172.21.15.7:0/379110479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d480cc980 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d357fa640 1 -- 172.21.15.7:0/379110479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d4809f860 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d56978740 10 client.17142 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.467+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643c71033f0 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d357fa640 1 -- 172.21.15.7:0/379110479 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d3c0054d0 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d357fa640 1 -- 172.21.15.7:0/379110479 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d480d11a0 con 0x5643c748aa50 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d56978740 10 client.17142 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d56978740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d56978740 20 client.17142 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d56978740 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c74a0d40 0x5643c74a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d56978740 4 client.17142 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c73fca40 con 0x5643c74a0d40 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d51253640 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c74a0d40 0x5643c74a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d51253640 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c74a0d40 0x5643c74a31d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45533a63a12ee64 server_cookie=47c01f410d889bfc in_seq=0 out_seq=0 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.468+0000 7f2d357fa640 10 client.17142 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d357fa640 1 -- 172.21.15.7:0/379110479 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5643c73fca40 con 0x5643c74a0d40 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d357fa640 10 client.17142 handle_command_reply: tid=0 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 client.17142 shutdown 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 msgr2=0x5643c7494ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c7494ea0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2d4809b6a0 tx=0x7f2d4809b070 comp rx=0 tx=0).stop 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 shutdown_connections 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c74905b0 0x5643c749c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748b440 0x5643c74953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c74a0d40 0x5643c74a31d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 --2- 172.21.15.7:0/379110479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748aa50 0x5643c7494ea0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 >> 172.21.15.7:0/379110479 conn(0x5643c7485fc0 msgr2=0x5643c74851e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d357fa640 0 client.17142 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 shutdown_connections 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 1 -- 172.21.15.7:0/379110479 wait complete. 2024-03-19T00:46:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.469+0000 7f2d56978740 20 client.17142 trim_cache size 0 max 16384 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 Processor -- start 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 -- start start 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748aa50 0x5643c7489ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 0x5643c749c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7493a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1acb50 con 0x5643c748b440 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1aa420 con 0x5643c74905b0 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d56978740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a8270 con 0x5643c748aa50 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d51253640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 0x5643c749c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d51253640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 0x5643c749c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37930/0 (socket says 172.21.15.7:37930) 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d51253640 1 -- 172.21.15.7:0/2311240351 learned_addr learned my addr 172.21.15.7:0/2311240351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d52255640 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7493a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.487+0000 7f2d51a54640 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748aa50 0x5643c7489ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d51253640 1 -- 172.21.15.7:0/2311240351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748aa50 msgr2=0x5643c7489ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d51253640 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748aa50 0x5643c7489ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d51253640 1 -- 172.21.15.7:0/2311240351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 msgr2=0x5643c7493a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d51253640 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7493a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d51253640 1 -- 172.21.15.7:0/2311240351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c70c30f0 con 0x5643c748b440 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d51253640 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 0x5643c749c370 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009e40 tx=0x7f2d3c004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c188f47d06cf9f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d377fe640 1 -- 172.21.15.7:0/2311240351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004050 con 0x5643c748b440 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d377fe640 1 -- 172.21.15.7:0/2311240351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d3c0041f0 con 0x5643c748b440 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.488+0000 7f2d377fe640 1 -- 172.21.15.7:0/2311240351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0035f0 con 0x5643c748b440 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- 172.21.15.7:0/2311240351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 msgr2=0x5643c749c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 0x5643c749c370 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009e40 tx=0x7f2d3c004500 comp rx=0 tx=0).stop 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- 172.21.15.7:0/2311240351 shutdown_connections 2024-03-19T00:46:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c74905b0 0x5643c7493a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c748b440 0x5643c749c370 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- 172.21.15.7:0/2311240351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748aa50 0x5643c7489ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- 172.21.15.7:0/2311240351 >> 172.21.15.7:0/2311240351 conn(0x5643c7485fc0 msgr2=0x5643c74a7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- 172.21.15.7:0/2311240351 shutdown_connections 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- 172.21.15.7:0/2311240351 wait complete. 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- start start 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 Processor -- start 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748aa50 0x5643c7499b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c749a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 0x5643c749af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a88e0 con 0x5643c74905b0 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a9d50 con 0x5643c748aa50 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a8fb0 con 0x5643c748b440 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d56978740 10 client.0 _ll_register_callbacks cb 0x7f2d480046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d52255640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 0x5643c749af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d52255640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 0x5643c749af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37938/0 (socket says 172.21.15.7:37938) 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d52255640 1 -- 172.21.15.7:0/3909925025 learned_addr learned my addr 172.21.15.7:0/3909925025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d51253640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c749a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.489+0000 7f2d51a54640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748aa50 0x5643c7499b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d52255640 1 -- 172.21.15.7:0/3909925025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 msgr2=0x5643c749a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d52255640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c749a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d52255640 1 -- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748aa50 msgr2=0x5643c7499b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d52255640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748aa50 0x5643c7499b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d52255640 1 -- 172.21.15.7:0/3909925025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c749b570 con 0x5643c74905b0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d51253640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c749a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d51a54640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748aa50 0x5643c7499b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d52255640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 0x5643c749af80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d44032420 tx=0x7f2d440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5759236fdd11fcf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d357fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d357fa640 1 -- 172.21.15.7:0/3909925025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44039040 con 0x5643c74905b0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d357fa640 1 -- 172.21.15.7:0/3909925025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d44034640 con 0x5643c74905b0 2024-03-19T00:46:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3c0041f0 con 0x5643c74905b0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d357fa640 1 -- 172.21.15.7:0/3909925025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d44060050 con 0x5643c74905b0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d56978740 10 client.17148 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643c75070d0 con 0x5643c74905b0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.490+0000 7f2d357fa640 1 -- 172.21.15.7:0/3909925025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d3c0041f0 con 0x5643c74905b0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d357fa640 1 -- 172.21.15.7:0/3909925025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d4403f070 con 0x5643c74905b0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 10 client.17148 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 20 client.17148 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c7507af0 0x5643c7509f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 4 client.17148 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643c73f6d90 con 0x5643c7507af0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d51a54640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c7507af0 0x5643c7509f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d357fa640 10 client.17148 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d51a54640 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c7507af0 0x5643c7509f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9ebbe9a00c85a8 server_cookie=d5f790c788a95c22 in_seq=0 out_seq=0 2024-03-19T00:46:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d357fa640 1 -- 172.21.15.7:0/3909925025 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643c73f6d90 con 0x5643c7507af0 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d357fa640 10 client.17148 handle_command_reply: tid=0 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 1 client.17148 shutdown 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 msgr2=0x5643c749af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.491+0000 7f2d56978740 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 0x5643c749af80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d44032420 tx=0x7f2d440095c0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 shutdown_connections 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5643c7507af0 0x5643c7509f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643c74905b0 0x5643c749af80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643c748b440 0x5643c749a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d357fa640 0 client.17148 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 --2- 172.21.15.7:0/3909925025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5643c748aa50 0x5643c7499b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 >> 172.21.15.7:0/3909925025 conn(0x5643c7485fc0 msgr2=0x5643c74a8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 shutdown_connections 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 1 -- 172.21.15.7:0/3909925025 wait complete. 2024-03-19T00:46:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d56978740 20 client.17148 trim_cache size 0 max 16384 2024-03-19T00:46:01.499 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:46:01.499 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:46:01.499 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:46:01.499 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2d380a2880 msgr2=0x7f2d380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2d380a2880 0x7f2d380a4d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5643c748bca0 tx=0x7f2d4402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 msgr2=0x7f2d4c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 0x7f2d4c1979f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e9c0 tx=0x7f2d4800ee80 comp rx=0 tx=0).stop 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 shutdown_connections 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f2d380a2880 0x7f2d380a4d40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d4c1022f0 0x7f2d4c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d4c101930 0x7f2d4c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 --2- 172.21.15.7:0/1129872108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d4c100740 0x7f2d4c1979f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 >> 172.21.15.7:0/1129872108 conn(0x7f2d4c0fbcd0 msgr2=0x7f2d4c104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 shutdown_connections 2024-03-19T00:46:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.492+0000 7f2d53cb8640 1 -- 172.21.15.7:0/1129872108 wait complete. 2024-03-19T00:46:01.504 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:46:01.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:46:01.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.606+0000 7fba77e63640 1 -- 172.21.15.7:0/1025798475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 msgr2=0x7fba70109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.606+0000 7fba77e63640 1 --2- 172.21.15.7:0/1025798475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba70109cf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ca0 tx=0x7fba6802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- 172.21.15.7:0/1025798475 shutdown_connections 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 --2- 172.21.15.7:0/1025798475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba70109cf0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 --2- 172.21.15.7:0/1025798475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 0x7fba70102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 --2- 172.21.15.7:0/1025798475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba701018b0 0x7fba70101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- 172.21.15.7:0/1025798475 >> 172.21.15.7:0/1025798475 conn(0x7fba700fce40 msgr2=0x7fba700ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- 172.21.15.7:0/1025798475 shutdown_connections 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- 172.21.15.7:0/1025798475 wait complete. 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 Processor -- start 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- start start 2024-03-19T00:46:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba701018b0 0x7fba701979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 0x7fba70197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba7019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010d370 con 0x7fba70103460 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010d1f0 con 0x7fba70102aa0 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.607+0000 7fba77e63640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010d4f0 con 0x7fba701018b0 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba7019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba7019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37958/0 (socket says 172.21.15.7:37958) 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 -- 172.21.15.7:0/2977871658 learned_addr learned my addr 172.21.15.7:0/2977871658 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba75bff640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba701018b0 0x7fba701979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba753fe640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 0x7fba70197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 -- 172.21.15.7:0/2977871658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba701018b0 msgr2=0x7fba701979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba701018b0 0x7fba701979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 -- 172.21.15.7:0/2977871658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 msgr2=0x7fba70197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 0x7fba70197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 -- 172.21.15.7:0/2977871658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7019f600 con 0x7fba70103460 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba753fe640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 0x7fba70197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba76400640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba7019ef90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fba68032380 tx=0x7fba68032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68041020 con 0x7fba70103460 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba68003e20 con 0x7fba70103460 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68039040 con 0x7fba70103460 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7019f890 con 0x7fba70103460 2024-03-19T00:46:01.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.608+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7019fdd0 con 0x7fba70103460 2024-03-19T00:46:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.609+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba70108c20 con 0x7fba70103460 2024-03-19T00:46:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.610+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fba68044020 con 0x7fba70103460 2024-03-19T00:46:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.611+0000 7fba66ffd640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fba5c0a2880 0x7fba5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.611+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fba680f0eb0 con 0x7fba70103460 2024-03-19T00:46:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.611+0000 7fba75bff640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fba5c0a2880 0x7fba5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.611+0000 7fba75bff640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fba5c0a2880 0x7fba5c0a4d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fba70102900 tx=0x7fba6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.615+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba6804b020 con 0x7fba70103460 2024-03-19T00:46:01.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.838+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba70062420 con 0x7fba70103460 2024-03-19T00:46:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.839+0000 7fba66ffd640 1 -- 172.21.15.7:0/2977871658 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fba680bdb90 con 0x7fba70103460 2024-03-19T00:46:01.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:46:01.840 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:46:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:46:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fba5c0a2880 msgr2=0x7fba5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fba5c0a2880 0x7fba5c0a4d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fba70102900 tx=0x7fba6c005f90 comp rx=0 tx=0).stop 2024-03-19T00:46:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 msgr2=0x7fba7019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba7019ef90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fba68032380 tx=0x7fba68032820 comp rx=0 tx=0).stop 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 shutdown_connections 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fba5c0a2880 0x7fba5c0a4d40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba70103460 0x7fba7019ef90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba70102aa0 0x7fba70197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 --2- 172.21.15.7:0/2977871658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba701018b0 0x7fba701979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 >> 172.21.15.7:0/2977871658 conn(0x7fba700fce40 msgr2=0x7fba70105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 shutdown_connections 2024-03-19T00:46:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.841+0000 7fba77e63640 1 -- 172.21.15.7:0/2977871658 wait complete. 2024-03-19T00:46:01.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:46:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 -- 172.21.15.7:0/2685575879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8102a10 msgr2=0x7f47b8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 --2- 172.21.15.7:0/2685575879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8102a10 0x7f47b8102e90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009900 tx=0x7f47a002bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 -- 172.21.15.7:0/2685575879 shutdown_connections 2024-03-19T00:46:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 --2- 172.21.15.7:0/2685575879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b81033d0 0x7f47b8109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 --2- 172.21.15.7:0/2685575879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8102a10 0x7f47b8102e90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 --2- 172.21.15.7:0/2685575879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47b8101820 0x7f47b8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 -- 172.21.15.7:0/2685575879 >> 172.21.15.7:0/2685575879 conn(0x7f47b80fcd50 msgr2=0x7f47b80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 -- 172.21.15.7:0/2685575879 shutdown_connections 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.957+0000 7f47bcab5640 1 -- 172.21.15.7:0/2685575879 wait complete. 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 Processor -- start 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 -- start start 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 0x7f47b8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8102a10 0x7f47b8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47b81033d0 0x7f47b819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810d350 con 0x7f47b8101820 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810d1d0 con 0x7f47b81033d0 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47bcab5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810d4d0 con 0x7f47b8102a10 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 0x7f47b8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 0x7f47b8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37978/0 (socket says 172.21.15.7:37978) 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47b659c640 1 -- 172.21.15.7:0/1984445584 learned_addr learned my addr 172.21.15.7:0/1984445584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47affff640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8102a10 0x7f47b8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.958+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47b81033d0 0x7f47b819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47b659c640 1 -- 172.21.15.7:0/1984445584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8102a10 msgr2=0x7f47b8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47b659c640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8102a10 0x7f47b8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47b659c640 1 -- 172.21.15.7:0/1984445584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47b81033d0 msgr2=0x7f47b819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47b659c640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47b81033d0 0x7f47b819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47b659c640 1 -- 172.21.15.7:0/1984445584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b819f670 con 0x7f47b8101820 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47b659c640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 0x7f47b8197a50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9c0 tx=0x7f47a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47af7fe640 1 -- 172.21.15.7:0/1984445584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a800cd80 con 0x7f47b8101820 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47af7fe640 1 -- 172.21.15.7:0/1984445584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a80035f0 con 0x7f47b8101820 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b819f960 con 0x7f47b8101820 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47af7fe640 1 -- 172.21.15.7:0/1984445584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8010690 con 0x7f47b8101820 2024-03-19T00:46:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.959+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b819fe80 con 0x7f47b8101820 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.961+0000 7f47af7fe640 1 -- 172.21.15.7:0/1984445584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f47a80037a0 con 0x7f47b8101820 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 Processor -- start 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47af7fe640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f47a40a2800 0x7f47a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f47a40a2800 0x7f47a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 -- start start 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47af7fe640 1 -- 172.21.15.7:0/1984445584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f47a80cf150 con 0x7f47b8101820 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 0x56421939ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x56421939fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193a0260 0x5642193a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5642192fa4b0 con 0x56421939a700 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642192a2d70 con 0x5642193a0260 2024-03-19T00:46:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47bf776740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642192f6550 con 0x56421939b0f0 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 0x56421939ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 0x56421939ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37990/0 (socket says 172.21.15.7:37990) 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 -- 172.21.15.7:0/4128620547 learned_addr learned my addr 172.21.15.7:0/4128620547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47affff640 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x56421939fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193a0260 0x5642193a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 -- 172.21.15.7:0/4128620547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 msgr2=0x56421939fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x56421939fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f47a40a2800 0x7f47a40a4cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f47b0005fd0 tx=0x7f47b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 -- 172.21.15.7:0/4128620547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193a0260 msgr2=0x5642193a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193a0260 0x5642193a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.962+0000 7f47b659c640 1 -- 172.21.15.7:0/4128620547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642192c5210 con 0x56421939a700 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.963+0000 7f47b659c640 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 0x56421939ab20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f47a80a3a00 tx=0x7f47a809b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cbdb4ce826244b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.963+0000 7f47acff9640 1 -- 172.21.15.7:0/4128620547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80b4070 con 0x56421939a700 2024-03-19T00:46:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.963+0000 7f47acff9640 1 -- 172.21.15.7:0/4128620547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a809b790 con 0x56421939a700 2024-03-19T00:46:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.963+0000 7f47acff9640 1 -- 172.21.15.7:0/4128620547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8097e00 con 0x56421939a700 2024-03-19T00:46:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- 172.21.15.7:0/4128620547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 msgr2=0x56421939ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 0x56421939ab20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f47a80a3a00 tx=0x7f47a809b390 comp rx=0 tx=0).stop 2024-03-19T00:46:01.967 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:01 smithi007 ceph-mon[62854]: pgmap v593: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- 172.21.15.7:0/4128620547 shutdown_connections 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193a0260 0x5642193a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x56421939fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- 172.21.15.7:0/4128620547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939a700 0x56421939ab20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- 172.21.15.7:0/4128620547 >> 172.21.15.7:0/4128620547 conn(0x564219391860 msgr2=0x564219391c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- 172.21.15.7:0/4128620547 shutdown_connections 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- 172.21.15.7:0/4128620547 wait complete. 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- start start 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 Processor -- start 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5642193a3210 con 0x5642193a0260 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642192b6310 con 0x5642193aa2e0 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.964+0000 7f47bf776740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642193a3390 con 0x56421939b0f0 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47bf776740 10 client.0 _ll_register_callbacks cb 0x5642192aa2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37992/0 (socket says 172.21.15.7:37992) 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 -- 172.21.15.7:0/1762303634 learned_addr learned my addr 172.21.15.7:0/1762303634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:53362/0 (socket says 172.21.15.7:53362) 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 -- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 msgr2=0x5642193a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 -- 172.21.15.7:0/1762303634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 msgr2=0x5642193ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 -- 172.21.15.7:0/1762303634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56421900e3f0 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47affff640 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193a52b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009900 tx=0x7f47a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1829aa8f4e53166b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f479dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f479dffb640 1 -- 172.21.15.7:0/1762303634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0005030 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f479dffb640 1 -- 172.21.15.7:0/1762303634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f47a00051d0 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a809b790 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f479dffb640 1 -- 172.21.15.7:0/1762303634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47a0052050 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47bf776740 10 client.17166 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.965+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642190d87d0 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f479dffb640 1 -- 172.21.15.7:0/1762303634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a809b790 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f479dffb640 1 -- 172.21.15.7:0/1762303634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f47a00035f0 con 0x5642193a0260 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47bf776740 10 client.17166 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47bf776740 10 client.17166 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47bf776740 20 client.17166 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47bf776740 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5642193b1070 0x5642193b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47bf776740 4 client.17166 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642193583e0 con 0x5642193b1070 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47b659c640 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5642193b1070 0x5642193b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f479dffb640 10 client.17166 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.966+0000 7f47b659c640 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5642193b1070 0x5642193b34b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cad22403a92f940b server_cookie=d06dd68d748dd3b0 in_seq=0 out_seq=0 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f479dffb640 1 -- 172.21.15.7:0/1762303634 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5642193583e0 con 0x5642193b1070 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f479dffb640 10 client.17166 handle_command_reply: tid=0 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 client.17166 shutdown 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 msgr2=0x5642193a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193a52b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009900 tx=0x7f47a0004270 comp rx=0 tx=0).stop 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 shutdown_connections 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5642193b1070 0x5642193b34b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f479dffb640 0 client.17166 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193a52b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 --2- 172.21.15.7:0/1762303634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 >> 172.21.15.7:0/1762303634 conn(0x564219391860 msgr2=0x564219390ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 shutdown_connections 2024-03-19T00:46:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 1 -- 172.21.15.7:0/1762303634 wait complete. 2024-03-19T00:46:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.967+0000 7f47bf776740 20 client.17166 trim_cache size 0 max 16384 2024-03-19T00:46:01.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.980+0000 7f47bf776740 1 Processor -- start 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 -- start start 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 0x56421938ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642193a0260 0x56421939c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x56421939cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81aa8f0 con 0x56421939b0f0 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a81c0 con 0x5642193aa2e0 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47bf776740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a6010 con 0x5642193a0260 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 0x56421938ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 0x56421938ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38000/0 (socket says 172.21.15.7:38000) 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 -- 172.21.15.7:0/2091198915 learned_addr learned my addr 172.21.15.7:0/2091198915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47affff640 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642193a0260 0x56421939c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x56421939cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 -- 172.21.15.7:0/2091198915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642193a0260 msgr2=0x56421939c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642193a0260 0x56421939c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 -- 172.21.15.7:0/2091198915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 msgr2=0x56421939cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x56421939cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 -- 172.21.15.7:0/2091198915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564219367880 con 0x56421939b0f0 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x56421939cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f47b659c640 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 0x56421938ca30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a809b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19773809de23acaa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f479ffff640 1 -- 172.21.15.7:0/2091198915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80b4040 con 0x56421939b0f0 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f479ffff640 1 -- 172.21.15.7:0/2091198915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a809b790 con 0x56421939b0f0 2024-03-19T00:46:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.981+0000 7f479ffff640 1 -- 172.21.15.7:0/2091198915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8097e00 con 0x56421939b0f0 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- 172.21.15.7:0/2091198915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 msgr2=0x56421938ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 0x56421938ca30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a809b400 comp rx=0 tx=0).stop 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- 172.21.15.7:0/2091198915 shutdown_connections 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x56421939cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642193a0260 0x56421939c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- 172.21.15.7:0/2091198915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56421939b0f0 0x56421938ca30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- 172.21.15.7:0/2091198915 >> 172.21.15.7:0/2091198915 conn(0x564219391860 msgr2=0x5642193a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- 172.21.15.7:0/2091198915 shutdown_connections 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- 172.21.15.7:0/2091198915 wait complete. 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- start start 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 Processor -- start 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a6680 con 0x5642193a0260 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a7af0 con 0x5642193aa2e0 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a6d50 con 0x56421939b0f0 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.982+0000 7f47bf776740 10 client.0 _ll_register_callbacks cb 0x7f47a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38008/0 (socket says 172.21.15.7:38008) 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 -- 172.21.15.7:0/2977326494 learned_addr learned my addr 172.21.15.7:0/2977326494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47b659c640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 -- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 msgr2=0x5642193c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 -- 172.21.15.7:0/2977326494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 msgr2=0x5642193c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193c25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 -- 172.21.15.7:0/2977326494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642193c2df0 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47b6d9d640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47affff640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193c44f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x56421938d3a0 tx=0x7f47a0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2b79c321e1532c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f479dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f479dffb640 1 -- 172.21.15.7:0/2977326494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a002cca0 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f479dffb640 1 -- 172.21.15.7:0/2977326494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f47a00048c0 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a809b790 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f479dffb640 1 -- 172.21.15.7:0/2977326494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47a0064050 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47bf776740 10 client.17172 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564219417970 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.983+0000 7f479dffb640 1 -- 172.21.15.7:0/2977326494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a809b790 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f479dffb640 1 -- 172.21.15.7:0/2977326494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f47a0034b80 con 0x5642193a0260 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 10 client.17172 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 10 client.17172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 20 client.17172 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564219418340 0x56421941a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 4 client.17172 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642193079f0 con 0x564219418340 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47b659c640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564219418340 0x56421941a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47b659c640 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564219418340 0x56421941a760 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65c37cd9f36755a5 server_cookie=873f7ebc16372c9e in_seq=0 out_seq=0 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f479dffb640 10 client.17172 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f479dffb640 1 -- 172.21.15.7:0/2977326494 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642193079f0 con 0x564219418340 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f479dffb640 10 client.17172 handle_command_reply: tid=0 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 1 client.17172 shutdown 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 msgr2=0x5642193c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.984+0000 7f47bf776740 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193c44f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x56421938d3a0 tx=0x7f47a0003eb0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 shutdown_connections 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564219418340 0x56421941a760 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642193aa2e0 0x5642193c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642193a0260 0x5642193c44f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 --2- 172.21.15.7:0/2977326494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56421939b0f0 0x5642193c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f479dffb640 0 client.17172 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 >> 172.21.15.7:0/2977326494 conn(0x564219391860 msgr2=0x564219395350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 shutdown_connections 2024-03-19T00:46:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 1 -- 172.21.15.7:0/2977326494 wait complete. 2024-03-19T00:46:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bf776740 20 client.17172 trim_cache size 0 max 16384 2024-03-19T00:46:01.994 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f47a40a2800 msgr2=0x7f47a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f47a40a2800 0x7f47a40a4cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f47b0005fd0 tx=0x7f47b0009500 comp rx=0 tx=0).stop 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 msgr2=0x7f47b8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 0x7f47b8197a50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9c0 tx=0x7f47a800ee80 comp rx=0 tx=0).stop 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 shutdown_connections 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f47a40a2800 0x7f47a40a4cc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47b81033d0 0x7f47b819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8102a10 0x7f47b8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 --2- 172.21.15.7:0/1984445584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47b8101820 0x7f47b8197a50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 >> 172.21.15.7:0/1984445584 conn(0x7f47b80fcd50 msgr2=0x7f47b8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 shutdown_connections 2024-03-19T00:46:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:01.985+0000 7f47bcab5640 1 -- 172.21.15.7:0/1984445584 wait complete. 2024-03-19T00:46:01.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:46:02.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:01 smithi204 ceph-mon[66572]: pgmap v593: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 -- 172.21.15.7:0/1476032964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c101930 msgr2=0x7f9c1c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 --2- 172.21.15.7:0/1476032964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c101930 0x7f9c1c101db0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009900 tx=0x7f9c0c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 -- 172.21.15.7:0/1476032964 shutdown_connections 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 --2- 172.21.15.7:0/1476032964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c1022f0 0x7f9c1c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 --2- 172.21.15.7:0/1476032964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c101930 0x7f9c1c101db0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 --2- 172.21.15.7:0/1476032964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c1c100740 0x7f9c1c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 -- 172.21.15.7:0/1476032964 >> 172.21.15.7:0/1476032964 conn(0x7f9c1c0fbc90 msgr2=0x7f9c1c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 -- 172.21.15.7:0/1476032964 shutdown_connections 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.101+0000 7f9c23281640 1 -- 172.21.15.7:0/1476032964 wait complete. 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 Processor -- start 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 -- start start 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 0x7f9c1c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c101930 0x7f9c1c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c1c1022f0 0x7f9c1c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10c270 con 0x7f9c1c100740 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10c0f0 con 0x7f9c1c101930 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c23281640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10c3f0 con 0x7f9c1c1022f0 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 0x7f9c1c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 0x7f9c1c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38036/0 (socket says 172.21.15.7:38036) 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 -- 172.21.15.7:0/3340804393 learned_addr learned my addr 172.21.15.7:0/3340804393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2081c640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c101930 0x7f9c1c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2181e640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c1c1022f0 0x7f9c1c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 -- 172.21.15.7:0/3340804393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c1c1022f0 msgr2=0x7f9c1c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c1c1022f0 0x7f9c1c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 -- 172.21.15.7:0/3340804393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c101930 msgr2=0x7f9c1c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c101930 0x7f9c1c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2101d640 1 -- 172.21.15.7:0/3340804393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c19f6e0 con 0x7f9c1c100740 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.102+0000 7f9c2081c640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c101930 0x7f9c1c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.103+0000 7f9c2101d640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 0x7f9c1c197ac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9c0 tx=0x7f9c1800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.103+0000 7f9c127fc640 1 -- 172.21.15.7:0/3340804393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800ccc0 con 0x7f9c1c100740 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.103+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c19f9d0 con 0x7f9c1c100740 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.103+0000 7f9c127fc640 1 -- 172.21.15.7:0/3340804393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c180035f0 con 0x7f9c1c100740 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.103+0000 7f9c127fc640 1 -- 172.21.15.7:0/3340804393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800ce60 con 0x7f9c1c100740 2024-03-19T00:46:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.103+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c19fea0 con 0x7f9c1c100740 2024-03-19T00:46:02.107 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:01 smithi044 ceph-mon[66366]: pgmap v593: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.105+0000 7f9c127fc640 1 -- 172.21.15.7:0/3340804393 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f9c180037a0 con 0x7f9c1c100740 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.105+0000 7f9c25e6e740 1 Processor -- start 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.105+0000 7f9c25e6e740 1 -- start start 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.105+0000 7f9c127fc640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c080a2800 0x7f9c080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55982540f4b0 con 0x5598254ab2f0 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5598253b7d70 con 0x5598254abce0 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55982540b550 con 0x5598254b0e50 2024-03-19T00:46:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2181e640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c080a2800 0x7f9c080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c127fc640 1 -- 172.21.15.7:0/3340804393 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9c180cf2f0 con 0x7f9c1c100740 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47846/0 (socket says 172.21.15.7:47846) 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2181e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 -- 172.21.15.7:0/1055122249 learned_addr learned my addr 172.21.15.7:0/1055122249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 -- 172.21.15.7:0/1055122249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 msgr2=0x5598254b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 -- 172.21.15.7:0/1055122249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 msgr2=0x5598254ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 -- 172.21.15.7:0/1055122249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598253da210 con 0x5598254abce0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2181e640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c080a2800 0x7f9c080a4cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005fd0 tx=0x7f9c14009e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c2081c640 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b07e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009900 tx=0x7f9c0c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465d10d376846669 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.106+0000 7f9c077fe640 1 -- 172.21.15.7:0/1055122249 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c045020 con 0x5598254abce0 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c077fe640 1 -- 172.21.15.7:0/1055122249 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c0c02cc90 con 0x5598254abce0 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c077fe640 1 -- 172.21.15.7:0/1055122249 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c004920 con 0x5598254abce0 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 -- 172.21.15.7:0/1055122249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 msgr2=0x5598254b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b07e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009900 tx=0x7f9c0c0095f0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 -- 172.21.15.7:0/1055122249 shutdown_connections 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b07e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/1055122249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 -- 172.21.15.7:0/1055122249 >> 172.21.15.7:0/1055122249 conn(0x5598254a6860 msgr2=0x5598254a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 -- 172.21.15.7:0/1055122249 shutdown_connections 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.107+0000 7f9c25e6e740 1 -- 172.21.15.7:0/1055122249 wait complete. 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 -- start start 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 Processor -- start 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5598254b3da0 con 0x5598254ab2f0 2024-03-19T00:46:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5598253cb310 con 0x5598254abce0 2024-03-19T00:46:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598254b3f20 con 0x5598254b0e50 2024-03-19T00:46:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c25e6e740 10 client.0 _ll_register_callbacks cb 0x5598253bf2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2081c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38046/0 (socket says 172.21.15.7:38046) 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2101d640 1 -- 172.21.15.7:0/809947396 learned_addr learned my addr 172.21.15.7:0/809947396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2181e640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2081c640 1 -- 172.21.15.7:0/809947396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 msgr2=0x5598254bcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2081c640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254bcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2081c640 1 -- 172.21.15.7:0/809947396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 msgr2=0x5598254b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2081c640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2081c640 1 -- 172.21.15.7:0/809947396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598251233f0 con 0x5598254abce0 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.108+0000 7f9c2101d640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c2081c640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b5c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5598254a77d0 tx=0x7f9c0c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477416198edebfc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c04ff9640 1 -- 172.21.15.7:0/809947396 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c050020 con 0x5598254abce0 2024-03-19T00:46:02.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0c02cc90 con 0x5598254abce0 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c04ff9640 1 -- 172.21.15.7:0/809947396 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9c0c041480 con 0x5598254abce0 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c04ff9640 1 -- 172.21.15.7:0/809947396 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c0c05b050 con 0x5598254abce0 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 10 client.27029 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598251ed7d0 con 0x5598254abce0 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c04ff9640 1 -- 172.21.15.7:0/809947396 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c0c02cc90 con 0x5598254abce0 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c04ff9640 1 -- 172.21.15.7:0/809947396 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9c0c04c460 con 0x5598254abce0 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 10 client.27029 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 10 client.27029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 20 client.27029 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254c1190 0x5598254c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 4 client.27029 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:02.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.109+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55982546d3e0 con 0x5598254c1190 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.110+0000 7f9c2101d640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254c1190 0x5598254c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.110+0000 7f9c04ff9640 10 client.27029 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.110+0000 7f9c2101d640 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254c1190 0x5598254c3580 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a793a182fc5b164 server_cookie=8f0dfa7b0139b6d6 in_seq=0 out_seq=0 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c04ff9640 1 -- 172.21.15.7:0/809947396 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55982546d3e0 con 0x5598254c1190 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c04ff9640 10 client.27029 handle_command_reply: tid=0 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 client.27029 shutdown 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 msgr2=0x5598254b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b5c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5598254a77d0 tx=0x7f9c0c0038a0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 shutdown_connections 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254c1190 0x5598254c3580 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254b0e50 0x5598254bcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c04ff9640 0 client.27029 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254abce0 0x5598254b5c60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/809947396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 >> 172.21.15.7:0/809947396 conn(0x5598254a6860 msgr2=0x5598254a5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 shutdown_connections 2024-03-19T00:46:02.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 1 -- 172.21.15.7:0/809947396 wait complete. 2024-03-19T00:46:02.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.111+0000 7f9c25e6e740 20 client.27029 trim_cache size 0 max 16384 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 Processor -- start 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 -- start start 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254abce0 0x5598254a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1acae0 con 0x5598254ab2f0 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1aa3b0 con 0x5598254a6860 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.124+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a8200 con 0x5598254abce0 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2081c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2081c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254bcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38054/0 (socket says 172.21.15.7:38054) 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2081c640 1 -- 172.21.15.7:0/3873952321 learned_addr learned my addr 172.21.15.7:0/3873952321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2181e640 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254abce0 0x5598254a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 -- 172.21.15.7:0/3873952321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254abce0 msgr2=0x5598254a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254abce0 0x5598254a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 -- 172.21.15.7:0/3873952321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 msgr2=0x5598254bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 -- 172.21.15.7:0/3873952321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982547c880 con 0x5598254a6860 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2081c640 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c2101d640 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254a5310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c1809b860 tx=0x7f9c18005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8909d257a3afb8f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c06ffd640 1 -- 172.21.15.7:0/3873952321 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18084e60 con 0x5598254a6860 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c06ffd640 1 -- 172.21.15.7:0/3873952321 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c1809bfd0 con 0x5598254a6860 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.125+0000 7f9c06ffd640 1 -- 172.21.15.7:0/3873952321 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c180ccb40 con 0x5598254a6860 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 -- 172.21.15.7:0/3873952321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 msgr2=0x5598254a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254a5310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c1809b860 tx=0x7f9c18005f10 comp rx=0 tx=0).stop 2024-03-19T00:46:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 -- 172.21.15.7:0/3873952321 shutdown_connections 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254abce0 0x5598254a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254ab2f0 0x5598254bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/3873952321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254a5310 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 -- 172.21.15.7:0/3873952321 >> 172.21.15.7:0/3873952321 conn(0x5598254acac0 msgr2=0x5598254b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 -- 172.21.15.7:0/3873952321 shutdown_connections 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 -- 172.21.15.7:0/3873952321 wait complete. 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 -- start start 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.126+0000 7f9c25e6e740 1 Processor -- start 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254ab2f0 0x5598254ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 0x5598254bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a8870 con 0x5598254abce0 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a9ce0 con 0x5598254a6860 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a8f40 con 0x5598254ab2f0 2024-03-19T00:46:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 10 client.0 _ll_register_callbacks cb 0x7f9c18097450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 0x5598254bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2081c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254ab2f0 0x5598254ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2101d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254ba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47886/0 (socket says 172.21.15.7:47886) 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 0x5598254bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38058/0 (socket says 172.21.15.7:38058) 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2101d640 1 -- 172.21.15.7:0/2981060641 learned_addr learned my addr 172.21.15.7:0/2981060641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 -- 172.21.15.7:0/2981060641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254ab2f0 msgr2=0x5598254ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254ab2f0 0x5598254ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 -- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 msgr2=0x5598254ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254ba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 -- 172.21.15.7:0/2981060641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598254bbe00 con 0x5598254abce0 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c2181e640 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 0x5598254bb810 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9c14013630 tx=0x7f9c1401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2830268845b71d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c04ff9640 1 -- 172.21.15.7:0/2981060641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14021410 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c04ff9640 1 -- 172.21.15.7:0/2981060641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9c140215b0 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.127+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c1809bfd0 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c04ff9640 1 -- 172.21.15.7:0/2981060641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c14039050 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 10 client.17190 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559825527480 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c04ff9640 1 -- 172.21.15.7:0/2981060641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c1809bfd0 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c04ff9640 1 -- 172.21.15.7:0/2981060641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9c14020050 con 0x5598254abce0 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 10 client.17190 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 10 client.17190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 20 client.17190 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254b0e50 0x559825529e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 4 client.17190 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55982541c9f0 con 0x5598254b0e50 2024-03-19T00:46:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c2101d640 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254b0e50 0x559825529e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c04ff9640 10 client.17190 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.128+0000 7f9c2101d640 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254b0e50 0x559825529e50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b2efaa425d7610f server_cookie=bb971f3c8eadca99 in_seq=0 out_seq=0 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c04ff9640 1 -- 172.21.15.7:0/2981060641 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55982541c9f0 con 0x5598254b0e50 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c04ff9640 10 client.17190 handle_command_reply: tid=0 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 client.17190 shutdown 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 msgr2=0x5598254bb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 0x5598254bb810 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9c14013630 tx=0x7f9c1401d000 comp rx=0 tx=0).stop 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 shutdown_connections 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x5598254b0e50 0x559825529e50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598254abce0 0x5598254bb810 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598254ab2f0 0x5598254ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 --2- 172.21.15.7:0/2981060641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598254a6860 0x5598254ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c04ff9640 0 client.17190 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 >> 172.21.15.7:0/2981060641 conn(0x5598254acac0 msgr2=0x5598254a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 shutdown_connections 2024-03-19T00:46:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 1 -- 172.21.15.7:0/2981060641 wait complete. 2024-03-19T00:46:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c25e6e740 20 client.17190 trim_cache size 0 max 16384 2024-03-19T00:46:02.137 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:46:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c080a2800 msgr2=0x7f9c080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c23281640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c080a2800 0x7f9c080a4cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005fd0 tx=0x7f9c14009e80 comp rx=0 tx=0).stop 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 msgr2=0x7f9c1c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.129+0000 7f9c23281640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 0x7f9c1c197ac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9c0 tx=0x7f9c1800edc0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 shutdown_connections 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9c080a2800 0x7f9c080a4cc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c1c1022f0 0x7f9c1c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c1c101930 0x7f9c1c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 --2- 172.21.15.7:0/3340804393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100740 0x7f9c1c197ac0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 >> 172.21.15.7:0/3340804393 conn(0x7f9c1c0fbc90 msgr2=0x7f9c1c0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 shutdown_connections 2024-03-19T00:46:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.130+0000 7f9c23281640 1 -- 172.21.15.7:0/3340804393 wait complete. 2024-03-19T00:46:02.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:46:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 -- 172.21.15.7:0/2560905949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224102330 msgr2=0x7f4224108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 --2- 172.21.15.7:0/2560905949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224102330 0x7f4224108bc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 -- 172.21.15.7:0/2560905949 shutdown_connections 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 --2- 172.21.15.7:0/2560905949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224102330 0x7f4224108bc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 --2- 172.21.15.7:0/2560905949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224101970 0x7f4224101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 --2- 172.21.15.7:0/2560905949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 0x7f4224100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 -- 172.21.15.7:0/2560905949 >> 172.21.15.7:0/2560905949 conn(0x7f42240fbcd0 msgr2=0x7f42240fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 -- 172.21.15.7:0/2560905949 shutdown_connections 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.244+0000 7f4228cb5640 1 -- 172.21.15.7:0/2560905949 wait complete. 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 Processor -- start 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 -- start start 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 0x7f42241979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 0x7f4224197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224102330 0x7f422419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f422410d350 con 0x7f4224101970 2024-03-19T00:46:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f422410d1d0 con 0x7f4224100780 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4228cb5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f422410d4d0 con 0x7f4224102330 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 0x7f4224197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f422259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 0x7f42241979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 0x7f4224197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38080/0 (socket says 172.21.15.7:38080) 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4221d9b640 1 -- 172.21.15.7:0/845386547 learned_addr learned my addr 172.21.15.7:0/845386547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.245+0000 7f4222d9d640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224102330 0x7f422419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4221d9b640 1 -- 172.21.15.7:0/845386547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224102330 msgr2=0x7f422419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4221d9b640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224102330 0x7f422419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4221d9b640 1 -- 172.21.15.7:0/845386547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 msgr2=0x7f42241979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4221d9b640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 0x7f42241979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4221d9b640 1 -- 172.21.15.7:0/845386547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422419f5f0 con 0x7f4224101970 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f422259c640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 0x7f42241979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4221d9b640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 0x7f4224197f10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f421800e9f0 tx=0x7f421800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f42137fe640 1 -- 172.21.15.7:0/845386547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421800cd80 con 0x7f4224101970 2024-03-19T00:46:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f42137fe640 1 -- 172.21.15.7:0/845386547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42180054d0 con 0x7f4224101970 2024-03-19T00:46:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422419f8e0 con 0x7f4224101970 2024-03-19T00:46:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f42137fe640 1 -- 172.21.15.7:0/845386547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218010490 con 0x7f4224101970 2024-03-19T00:46:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.246+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422419fde0 con 0x7f4224101970 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 Processor -- start 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f42137fe640 1 -- 172.21.15.7:0/845386547 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f4218005020 con 0x7f4224101970 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 -- start start 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5dce0 0x5626c1f627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626c1ec14b0 con 0x5626c1f62e50 2024-03-19T00:46:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626c1e69d70 con 0x5626c1f5dce0 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f422b976740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5626c1ebd550 con 0x5626c1f5d2f0 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.249+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f422259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5dce0 0x5626c1f627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38084/0 (socket says 172.21.15.7:38084) 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 -- 172.21.15.7:0/1479123423 learned_addr learned my addr 172.21.15.7:0/1479123423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4222d9d640 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f42137fe640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f420c0a2800 0x7f420c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 -- 172.21.15.7:0/1479123423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5d2f0 msgr2=0x5626c1f5d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 -- 172.21.15.7:0/1479123423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5dce0 msgr2=0x5626c1f627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5dce0 0x5626c1f627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 -- 172.21.15.7:0/1479123423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c1e8c210 con 0x5626c1f62e50 2024-03-19T00:46:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f422259c640 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5dce0 0x5626c1f627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4222d9d640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f420c0a2800 0x7f420c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f42137fe640 1 -- 172.21.15.7:0/845386547 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f42180cf120 con 0x7f4224101970 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4221d9b640 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f652e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4218084ee0 tx=0x7f42180665e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80849bc3820c6062 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4210ff9640 1 -- 172.21.15.7:0/1479123423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421809b530 con 0x5626c1f62e50 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4222d9d640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f420c0a2800 0x7f420c0a4cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f421c006010 tx=0x7f421c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4210ff9640 1 -- 172.21.15.7:0/1479123423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4218084d20 con 0x5626c1f62e50 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.250+0000 7f4210ff9640 1 -- 172.21.15.7:0/1479123423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180bb9e0 con 0x5626c1f62e50 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 -- 172.21.15.7:0/1479123423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 msgr2=0x5626c1f652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f652e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4218084ee0 tx=0x7f42180665e0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 -- 172.21.15.7:0/1479123423 shutdown_connections 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f652e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5dce0 0x5626c1f627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 --2- 172.21.15.7:0/1479123423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 -- 172.21.15.7:0/1479123423 >> 172.21.15.7:0/1479123423 conn(0x5626c1f58860 msgr2=0x5626c1f58c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 -- 172.21.15.7:0/1479123423 shutdown_connections 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 -- 172.21.15.7:0/1479123423 wait complete. 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.251+0000 7f422b976740 1 -- start start 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 Processor -- start 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626c1f65da0 con 0x5626c1f62e50 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626c1e7d310 con 0x5626c1f5d2f0 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5626c1f65f20 con 0x5626c1f5dce0 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422b976740 10 client.0 _ll_register_callbacks cb 0x5626c1e712d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38096/0 (socket says 172.21.15.7:38096) 2024-03-19T00:46:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 -- 172.21.15.7:0/1721610856 learned_addr learned my addr 172.21.15.7:0/1721610856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 -- 172.21.15.7:0/1721610856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 msgr2=0x5626c1f67d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f67d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 -- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 msgr2=0x5626c1f677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 -- 172.21.15.7:0/1721610856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c1bd53f0 con 0x5626c1f62e50 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f422259c640 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4222d9d640 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c0324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5536f25ebfea0a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.252+0000 7f4209ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f4209ffb640 1 -- 172.21.15.7:0/1721610856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c052060 con 0x5626c1f62e50 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f4209ffb640 1 -- 172.21.15.7:0/1721610856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f421c003b70 con 0x5626c1f62e50 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5626c1f1f3e0 con 0x5626c1f62e50 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f4209ffb640 1 -- 172.21.15.7:0/1721610856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f421c039040 con 0x5626c1f62e50 2024-03-19T00:46:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 10 client.17202 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626c1c9f7d0 con 0x5626c1f62e50 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f4209ffb640 1 -- 172.21.15.7:0/1721610856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5626c1f1f3e0 con 0x5626c1f62e50 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f4209ffb640 1 -- 172.21.15.7:0/1721610856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f421c03f040 con 0x5626c1f62e50 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 10 client.17202 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 20 client.17202 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1f73660 0x5626c1f75b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 4 client.17202 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c1ece9f0 con 0x5626c1f73660 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.253+0000 7f422259c640 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1f73660 0x5626c1f75b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f4209ffb640 10 client.17202 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422259c640 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1f73660 0x5626c1f75b00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9fa5c22dfbe37e server_cookie=5b5cd7ca6b446459 in_seq=0 out_seq=0 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f4209ffb640 1 -- 172.21.15.7:0/1721610856 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5626c1ece9f0 con 0x5626c1f73660 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f4209ffb640 10 client.17202 handle_command_reply: tid=0 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422b976740 1 client.17202 shutdown 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 msgr2=0x5626c1f6ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422b976740 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c0324f0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 shutdown_connections 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422b976740 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1f73660 0x5626c1f75b00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.254+0000 7f422b976740 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f422b976740 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f67d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f422b976740 1 --2- 172.21.15.7:0/1721610856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f4209ffb640 0 client.17202 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 >> 172.21.15.7:0/1721610856 conn(0x5626c1f58860 msgr2=0x5626c1f57ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 shutdown_connections 2024-03-19T00:46:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f422b976740 1 -- 172.21.15.7:0/1721610856 wait complete. 2024-03-19T00:46:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.255+0000 7f422b976740 20 client.17202 trim_cache size 0 max 16384 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 Processor -- start 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 -- start start 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f5f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42241acb00 con 0x5626c1f62e50 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42241aa3d0 con 0x5626c1f5d2f0 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422b976740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42241a8220 con 0x5626c1f5dce0 2024-03-19T00:46:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f5f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f422259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38110/0 (socket says 172.21.15.7:38110) 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f5f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:53490/0 (socket says 172.21.15.7:53490) 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 -- 172.21.15.7:0/2594733896 learned_addr learned my addr 172.21.15.7:0/2594733896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 -- 172.21.15.7:0/2594733896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 msgr2=0x5626c1f5f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f5f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 -- 172.21.15.7:0/2594733896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 msgr2=0x5626c1f5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 -- 172.21.15.7:0/2594733896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c1f5e450 con 0x5626c1f62e50 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4221d9b640 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f5f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.268+0000 7f4222d9d640 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f421c02c630 tx=0x7f421c032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba428b01e46100d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f420bfff640 1 -- 172.21.15.7:0/2594733896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c052070 con 0x5626c1f62e50 2024-03-19T00:46:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f420bfff640 1 -- 172.21.15.7:0/2594733896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f421c046030 con 0x5626c1f62e50 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f420bfff640 1 -- 172.21.15.7:0/2594733896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c003f50 con 0x5626c1f62e50 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 -- 172.21.15.7:0/2594733896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 msgr2=0x5626c1f6ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f421c02c630 tx=0x7f421c032990 comp rx=0 tx=0).stop 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 -- 172.21.15.7:0/2594733896 shutdown_connections 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6ed10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f5f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 --2- 172.21.15.7:0/2594733896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 -- 172.21.15.7:0/2594733896 >> 172.21.15.7:0/2594733896 conn(0x5626c1f58860 msgr2=0x5626c1f58c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 -- 172.21.15.7:0/2594733896 shutdown_connections 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 -- 172.21.15.7:0/2594733896 wait complete. 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 -- start start 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.269+0000 7f422b976740 1 Processor -- start 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f6c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42241a8890 con 0x5626c1f62e50 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42241a9d00 con 0x5626c1f5d2f0 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42241a8f60 con 0x5626c1f5dce0 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 10 client.0 _ll_register_callbacks cb 0x7f4218005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f6c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f6c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47948/0 (socket says 172.21.15.7:47948) 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4221d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422259c640 1 -- 172.21.15.7:0/3559268898 learned_addr learned my addr 172.21.15.7:0/3559268898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 -- 172.21.15.7:0/3559268898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 msgr2=0x5626c1f6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 -- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 msgr2=0x5626c1f6c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f6c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 -- 172.21.15.7:0/3559268898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c1f6dc20 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4222d9d640 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6d630 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c042f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d820718c937f264b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4209ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4209ffb640 1 -- 172.21.15.7:0/3559268898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c052070 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f4209ffb640 1 -- 172.21.15.7:0/3559268898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f421c003e90 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.270+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c046030 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f4209ffb640 1 -- 172.21.15.7:0/3559268898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f421c074050 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 10 client.17208 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626c1f6ef20 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f4209ffb640 1 -- 172.21.15.7:0/3559268898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f421c046030 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f4209ffb640 1 -- 172.21.15.7:0/3559268898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f421c066b70 con 0x5626c1f62e50 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 10 client.17208 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 20 client.17208 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1fd9dc0 0x5626c1fdc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 4 client.17208 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626c1ec8d40 con 0x5626c1fd9dc0 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.271+0000 7f422259c640 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1fd9dc0 0x5626c1fdc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422259c640 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1fd9dc0 0x5626c1fdc1e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c7093f93478116 server_cookie=ae88e4e280b5963 in_seq=0 out_seq=0 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4209ffb640 10 client.17208 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4209ffb640 1 -- 172.21.15.7:0/3559268898 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626c1ec8d40 con 0x5626c1fd9dc0 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4209ffb640 10 client.17208 handle_command_reply: tid=0 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 client.17208 shutdown 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 msgr2=0x5626c1f6d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6d630 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c042f70 comp rx=0 tx=0).stop 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 shutdown_connections 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5626c1fd9dc0 0x5626c1fdc1e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626c1f62e50 0x5626c1f6d630 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626c1f5dce0 0x5626c1f6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4209ffb640 0 client.17208 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 --2- 172.21.15.7:0/3559268898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626c1f5d2f0 0x5626c1f6c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 >> 172.21.15.7:0/3559268898 conn(0x5626c1f58860 msgr2=0x5626c1f662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 shutdown_connections 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 1 -- 172.21.15.7:0/3559268898 wait complete. 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f422b976740 20 client.17208 trim_cache size 0 max 16384 2024-03-19T00:46:02.280 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f420c0a2800 msgr2=0x7f420c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4228cb5640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f420c0a2800 0x7f420c0a4cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f421c006010 tx=0x7f421c02e040 comp rx=0 tx=0).stop 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 msgr2=0x7f4224197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4228cb5640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 0x7f4224197f10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f421800e9f0 tx=0x7f421800edf0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.272+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 shutdown_connections 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f420c0a2800 0x7f420c0a4cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4224102330 0x7f422419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4224101970 0x7f4224197f10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 --2- 172.21.15.7:0/845386547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4224100780 0x7f42241979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 >> 172.21.15.7:0/845386547 conn(0x7f42240fbcd0 msgr2=0x7f4224104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 shutdown_connections 2024-03-19T00:46:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:02.273+0000 7f4228cb5640 1 -- 172.21.15.7:0/845386547 wait complete. 2024-03-19T00:46:02.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:02 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2977871658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:46:03.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:02 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2977871658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:46:03.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:02 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2977871658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:46:03.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:46:03.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:46:03.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:46:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 -- 172.21.15.7:0/2263603754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9658101a10 msgr2=0x7f9658108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2263603754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9658101a10 0x7f9658108330 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ca0 tx=0x7f965002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 -- 172.21.15.7:0/2263603754 shutdown_connections 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2263603754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9658101a10 0x7f9658108330 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2263603754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9658101050 0x7f96581014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2263603754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96580ffe70 0x7f9658100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 -- 172.21.15.7:0/2263603754 >> 172.21.15.7:0/2263603754 conn(0x7f96580fb430 msgr2=0x7f96580fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 -- 172.21.15.7:0/2263603754 shutdown_connections 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.403+0000 7f965f2ff640 1 -- 172.21.15.7:0/2263603754 wait complete. 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 Processor -- start 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 -- start start 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 0x7f9658197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9658101050 0x7f9658197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9658101a10 0x7f965819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f965810b170 con 0x7f96580ffe70 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f965810aff0 con 0x7f9658101a10 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965f2ff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f965810b2f0 con 0x7f9658101050 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 0x7f9658197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 0x7f9658197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38126/0 (socket says 172.21.15.7:38126) 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 -- 172.21.15.7:0/2903067383 learned_addr learned my addr 172.21.15.7:0/2903067383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d89c640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9658101a10 0x7f965819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965c89a640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9658101050 0x7f9658197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 -- 172.21.15.7:0/2903067383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9658101050 msgr2=0x7f9658197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9658101050 0x7f9658197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 -- 172.21.15.7:0/2903067383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9658101a10 msgr2=0x7f965819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9658101a10 0x7f965819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.404+0000 7f965d09b640 1 -- 172.21.15.7:0/2903067383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965819f640 con 0x7f96580ffe70 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.405+0000 7f965d09b640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 0x7f9658197a20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f965400e9c0 tx=0x7f965400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.405+0000 7f964e7fc640 1 -- 172.21.15.7:0/2903067383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965400cd80 con 0x7f96580ffe70 2024-03-19T00:46:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.405+0000 7f964e7fc640 1 -- 172.21.15.7:0/2903067383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96540035f0 con 0x7f96580ffe70 2024-03-19T00:46:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.405+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965819f930 con 0x7f96580ffe70 2024-03-19T00:46:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.405+0000 7f964e7fc640 1 -- 172.21.15.7:0/2903067383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654010690 con 0x7f96580ffe70 2024-03-19T00:46:03.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.405+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965819fed0 con 0x7f96580ffe70 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.407+0000 7f964e7fc640 1 -- 172.21.15.7:0/2903067383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f96540037a0 con 0x7f96580ffe70 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.407+0000 7f9661fc8740 1 Processor -- start 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.407+0000 7f9661fc8740 1 -- start start 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a2027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a207890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a207f00 0x563f6a20a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f964e7fc640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96440a2880 0x7f96440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f9661fc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563f6a167160 con 0x563f6a2023a0 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f9661fc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563f6a10fe10 con 0x563f6a207f00 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f9661fc8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563f6a163200 con 0x563f6a202d90 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d89c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a2027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d89c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a2027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38134/0 (socket says 172.21.15.7:38134) 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d89c640 1 -- 172.21.15.7:0/1054420467 learned_addr learned my addr 172.21.15.7:0/1054420467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d09b640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a207890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965c89a640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a207f00 0x563f6a20a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d89c640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96440a2880 0x7f96440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f964e7fc640 1 -- 172.21.15.7:0/2903067383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f96540cf130 con 0x7f96580ffe70 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d09b640 1 -- 172.21.15.7:0/1054420467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a207f00 msgr2=0x563f6a20a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d09b640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a207f00 0x563f6a20a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d09b640 1 -- 172.21.15.7:0/1054420467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 msgr2=0x563f6a2027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d09b640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a2027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d09b640 1 -- 172.21.15.7:0/1054420467 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f69f437e0 con 0x563f6a202d90 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965d89c640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a2027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.408+0000 7f965c89a640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a207f00 0x563f6a20a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f965d09b640 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a207890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9654097b70 tx=0x7f965401b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=decfb6dd32a1a991 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f965d89c640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96440a2880 0x7f96440a4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9650032380 tx=0x7f96500324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f96437fe640 1 -- 172.21.15.7:0/1054420467 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965409b250 con 0x563f6a202d90 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f96437fe640 1 -- 172.21.15.7:0/1054420467 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9654003b30 con 0x563f6a202d90 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f96437fe640 1 -- 172.21.15.7:0/1054420467 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965409b070 con 0x563f6a202d90 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 -- 172.21.15.7:0/1054420467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 msgr2=0x563f6a207890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a207890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9654097b70 tx=0x7f965401b4f0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 -- 172.21.15.7:0/1054420467 shutdown_connections 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a207f00 0x563f6a20a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a207890 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 --2- 172.21.15.7:0/1054420467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a2027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 -- 172.21.15.7:0/1054420467 >> 172.21.15.7:0/1054420467 conn(0x563f6a1fd910 msgr2=0x563f6a1fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.409+0000 7f9661fc8740 1 -- 172.21.15.7:0/1054420467 shutdown_connections 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 -- 172.21.15.7:0/1054420467 wait complete. 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 -- start start 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 Processor -- start 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a20c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a20cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a213c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563f6a20aeb0 con 0x563f6a2023a0 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563f6a1231a0 con 0x563f6a202d90 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563f6a20b030 con 0x563f6a207f00 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f9661fc8740 10 client.0 _ll_register_callbacks cb 0x563f69e7ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f965d09b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a20c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f965d09b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a20c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38148/0 (socket says 172.21.15.7:38148) 2024-03-19T00:46:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f965d09b640 1 -- 172.21.15.7:0/398061341 learned_addr learned my addr 172.21.15.7:0/398061341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.410+0000 7f965c89a640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a20cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965d89c640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a213c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965c89a640 1 -- 172.21.15.7:0/398061341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 msgr2=0x563f6a213c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965c89a640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a213c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965c89a640 1 -- 172.21.15.7:0/398061341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 msgr2=0x563f6a20c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965c89a640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a20c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965c89a640 1 -- 172.21.15.7:0/398061341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f69fa3f70 con 0x563f6a202d90 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965d09b640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a20c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f965c89a640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a20cc80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9658198f30 tx=0x7f964800eb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a984d655fae5e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9640ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9640ff9640 1 -- 172.21.15.7:0/398061341 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964801e060 con 0x563f6a202d90 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9640ff9640 1 -- 172.21.15.7:0/398061341 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9648004260 con 0x563f6a202d90 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f6a1746a0 con 0x563f6a202d90 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9640ff9640 1 -- 172.21.15.7:0/398061341 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9648025050 con 0x563f6a202d90 2024-03-19T00:46:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9661fc8740 10 client.27065 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.411+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f69e3b170 con 0x563f6a202d90 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9640ff9640 1 -- 172.21.15.7:0/398061341 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563f6a1746a0 con 0x563f6a202d90 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9640ff9640 1 -- 172.21.15.7:0/398061341 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f964800b040 con 0x563f6a202d90 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9661fc8740 10 client.27065 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9661fc8740 10 client.27065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9661fc8740 20 client.27065 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9661fc8740 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a2185b0 0x563f6a21aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9661fc8740 4 client.27065 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f6a16e9f0 con 0x563f6a2185b0 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f965d09b640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a2185b0 0x563f6a21aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f965d09b640 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a2185b0 0x563f6a21aa30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6642b195b04f817 server_cookie=7118f0787444e896 in_seq=0 out_seq=0 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.412+0000 7f9640ff9640 10 client.27065 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9640ff9640 1 -- 172.21.15.7:0/398061341 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563f6a16e9f0 con 0x563f6a2185b0 2024-03-19T00:46:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9640ff9640 10 client.27065 handle_command_reply: tid=0 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 client.27065 shutdown 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 msgr2=0x563f6a20cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a20cc80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9658198f30 tx=0x7f964800eb40 comp rx=0 tx=0).stop 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 shutdown_connections 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a213c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a20cc80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a2185b0 0x563f6a21aa30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 --2- 172.21.15.7:0/398061341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a20c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 >> 172.21.15.7:0/398061341 conn(0x563f6a1fd910 msgr2=0x563f6a1fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9640ff9640 0 client.27065 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 shutdown_connections 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 1 -- 172.21.15.7:0/398061341 wait complete. 2024-03-19T00:46:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.413+0000 7f9661fc8740 20 client.27065 trim_cache size 0 max 16384 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 Processor -- start 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 -- start start 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a2023a0 0x563f6a20bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a213c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 0x563f6a1fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96581acbd0 con 0x563f6a207f00 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96581aa4a0 con 0x563f6a2023a0 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f9661fc8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96581a82f0 con 0x563f6a202d90 2024-03-19T00:46:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 0x563f6a1fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d09b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a2023a0 0x563f6a20bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 0x563f6a1fc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38158/0 (socket says 172.21.15.7:38158) 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 -- 172.21.15.7:0/3076025975 learned_addr learned my addr 172.21.15.7:0/3076025975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965c89a640 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a213c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 -- 172.21.15.7:0/3076025975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 msgr2=0x563f6a213c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a213c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 -- 172.21.15.7:0/3076025975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a2023a0 msgr2=0x563f6a20bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a2023a0 0x563f6a20bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.431+0000 7f965d89c640 1 -- 172.21.15.7:0/3076025975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a203170 con 0x563f6a207f00 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f965d89c640 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 0x563f6a1fc2a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ca0 tx=0x7f965002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4dc57bc68ca833 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9642ffd640 1 -- 172.21.15.7:0/3076025975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650039040 con 0x563f6a207f00 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9642ffd640 1 -- 172.21.15.7:0/3076025975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9650054070 con 0x563f6a207f00 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9642ffd640 1 -- 172.21.15.7:0/3076025975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965004f7e0 con 0x563f6a207f00 2024-03-19T00:46:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 -- 172.21.15.7:0/3076025975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 msgr2=0x563f6a1fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 0x563f6a1fc2a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ca0 tx=0x7f965002e040 comp rx=0 tx=0).stop 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 -- 172.21.15.7:0/3076025975 shutdown_connections 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a207f00 0x563f6a1fc2a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a202d90 0x563f6a213c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 --2- 172.21.15.7:0/3076025975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a2023a0 0x563f6a20bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 -- 172.21.15.7:0/3076025975 >> 172.21.15.7:0/3076025975 conn(0x563f6a1fd910 msgr2=0x563f6a20b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 -- 172.21.15.7:0/3076025975 shutdown_connections 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.432+0000 7f9661fc8740 1 -- 172.21.15.7:0/3076025975 wait complete. 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 -- start start 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 Processor -- start 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a211570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a211ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a212ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96581a8960 con 0x563f6a2023a0 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96581a9dd0 con 0x563f6a202d90 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96581a9030 con 0x563f6a207f00 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f9661fc8740 10 client.0 _ll_register_callbacks cb 0x7f964800add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a211570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a211570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38170/0 (socket says 172.21.15.7:38170) 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 -- 172.21.15.7:0/4027226871 learned_addr learned my addr 172.21.15.7:0/4027226871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d89c640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a212ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965c89a640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a211ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 -- 172.21.15.7:0/4027226871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 msgr2=0x563f6a212ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a212ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 -- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 msgr2=0x563f6a211ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a211ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.433+0000 7f965d09b640 1 -- 172.21.15.7:0/4027226871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6a2130b0 con 0x563f6a2023a0 2024-03-19T00:46:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f965d09b640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a211570 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f965400c6a0 tx=0x7f96540cf3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb00824750f62e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9640ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9640ff9640 1 -- 172.21.15.7:0/4027226871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965409b070 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9640ff9640 1 -- 172.21.15.7:0/4027226871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f96540cc8f0 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9650054070 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9640ff9640 1 -- 172.21.15.7:0/4027226871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f965409be10 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 10 client.17226 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f6a27ebe0 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9640ff9640 1 -- 172.21.15.7:0/4027226871 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9650054070 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9640ff9640 1 -- 172.21.15.7:0/4027226871 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f96540ab300 con 0x563f6a2023a0 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 10 client.17226 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 20 client.17226 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a27f110 0x563f6a281500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 4 client.17226 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:46:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.434+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f6a0ecbe0 con 0x563f6a27f110 2024-03-19T00:46:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.435+0000 7f965c89a640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a27f110 0x563f6a281500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.435+0000 7f9640ff9640 10 client.17226 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.435+0000 7f965c89a640 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a27f110 0x563f6a281500 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e48c47a9dbe7db server_cookie=bd187afb30050e5b in_seq=0 out_seq=0 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9640ff9640 1 -- 172.21.15.7:0/4027226871 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f6a0ecbe0 con 0x563f6a27f110 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9640ff9640 10 client.17226 handle_command_reply: tid=0 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 client.17226 shutdown 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 msgr2=0x563f6a211570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a211570 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f965400c6a0 tx=0x7f96540cf3b0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 shutdown_connections 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f6a207f00 0x563f6a212ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563f6a202d90 0x563f6a211ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563f6a27f110 0x563f6a281500 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 --2- 172.21.15.7:0/4027226871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f6a2023a0 0x563f6a211570 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 >> 172.21.15.7:0/4027226871 conn(0x563f6a1fd910 msgr2=0x563f6a200b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9640ff9640 0 client.17226 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 shutdown_connections 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 1 -- 172.21.15.7:0/4027226871 wait complete. 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.436+0000 7f9661fc8740 20 client.17226 trim_cache size 0 max 16384 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "86723147-85f0-4159-98e1-78443f503882", 2024-03-19T00:46:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:46:03.444 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:46:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96440a2880 msgr2=0x7f96440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96440a2880 0x7f96440a4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9650032380 tx=0x7f96500324f0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 msgr2=0x7f9658197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 0x7f9658197a20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f965400e9c0 tx=0x7f965400ee80 comp rx=0 tx=0).stop 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 shutdown_connections 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f96440a2880 0x7f96440a4d40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9658101a10 0x7f965819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9658101050 0x7f9658197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 --2- 172.21.15.7:0/2903067383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96580ffe70 0x7f9658197a20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 >> 172.21.15.7:0/2903067383 conn(0x7f96580fb430 msgr2=0x7f9658106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 shutdown_connections 2024-03-19T00:46:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.437+0000 7f965f2ff640 1 -- 172.21.15.7:0/2903067383 wait complete. 2024-03-19T00:46:03.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:46:03.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:46:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 -- 172.21.15.7:0/1114227722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea01033e0 msgr2=0x7f1ea0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 --2- 172.21.15.7:0/1114227722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea01033e0 0x7f1ea0109c70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ca0 tx=0x7f1e9802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 -- 172.21.15.7:0/1114227722 shutdown_connections 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 --2- 172.21.15.7:0/1114227722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea01033e0 0x7f1ea0109c70 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 --2- 172.21.15.7:0/1114227722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea0102a20 0x7f1ea0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 --2- 172.21.15.7:0/1114227722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea0101830 0x7f1ea0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 -- 172.21.15.7:0/1114227722 >> 172.21.15.7:0/1114227722 conn(0x7f1ea00fcda0 msgr2=0x7f1ea00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 -- 172.21.15.7:0/1114227722 shutdown_connections 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.552+0000 7f1ea8109640 1 -- 172.21.15.7:0/1114227722 wait complete. 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 Processor -- start 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 -- start start 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea0101830 0x7f1ea0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 0x7f1ea0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea01033e0 0x7f1ea019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea010d360 con 0x7f1ea0102a20 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea010d1e0 con 0x7f1ea0101830 2024-03-19T00:46:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea8109640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea010d4e0 con 0x7f1ea01033e0 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 0x7f1ea0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea5ea5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea0101830 0x7f1ea0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 0x7f1ea0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38198/0 (socket says 172.21.15.7:38198) 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 -- 172.21.15.7:0/2271133022 learned_addr learned my addr 172.21.15.7:0/2271133022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea01033e0 0x7f1ea019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 -- 172.21.15.7:0/2271133022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea01033e0 msgr2=0x7f1ea019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea01033e0 0x7f1ea019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 -- 172.21.15.7:0/2271133022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea0101830 msgr2=0x7f1ea0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea0101830 0x7f1ea0197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.553+0000 7f1ea56a4640 1 -- 172.21.15.7:0/2271133022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea019f650 con 0x7f1ea0102a20 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.554+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 0x7f1ea0197f70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.554+0000 7f1e96ffd640 1 -- 172.21.15.7:0/2271133022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000cd80 con 0x7f1ea0102a20 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.554+0000 7f1e96ffd640 1 -- 172.21.15.7:0/2271133022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e900108e0 con 0x7f1ea0102a20 2024-03-19T00:46:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.554+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea019f940 con 0x7f1ea0102a20 2024-03-19T00:46:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.554+0000 7f1e96ffd640 1 -- 172.21.15.7:0/2271133022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90018920 con 0x7f1ea0102a20 2024-03-19T00:46:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.554+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea019fee0 con 0x7f1ea0102a20 2024-03-19T00:46:03.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.556+0000 7f1e96ffd640 1 -- 172.21.15.7:0/2271133022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1e90010a90 con 0x7f1ea0102a20 2024-03-19T00:46:03.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.556+0000 7f1eaadd2740 1 Processor -- start 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.556+0000 7f1eaadd2740 1 -- start start 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c971280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c976350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c978e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563d8c8d1500 con 0x563d8c970e60 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d8c879be0 con 0x563d8c9769c0 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563d8c8cd5a0 con 0x563d8c971850 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea66a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c971280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea66a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c971280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38200/0 (socket says 172.21.15.7:38200) 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea66a6640 1 -- 172.21.15.7:0/3376187715 learned_addr learned my addr 172.21.15.7:0/3376187715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c976350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1e96ffd640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e8c0a2880 0x7f1e8c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c978e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e8c0a2880 0x7f1e8c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3376187715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 msgr2=0x563d8c978e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c978e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1e96ffd640 1 -- 172.21.15.7:0/2271133022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1e900cf050 con 0x7f1ea0102a20 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3376187715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 msgr2=0x563d8c971280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c971280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3376187715 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8c89b000 con 0x563d8c971850 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c971280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c976350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009bd0 tx=0x7f1e9c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec9262724ab48fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.557+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e8c0a2880 0x7f1e8c0a4d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x563d8c9720b0 tx=0x7f1e9802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1e8bfff640 1 -- 172.21.15.7:0/3376187715 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c004230 con 0x563d8c971850 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1e8bfff640 1 -- 172.21.15.7:0/3376187715 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e9c009e40 con 0x563d8c971850 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1e8bfff640 1 -- 172.21.15.7:0/3376187715 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c019790 con 0x563d8c971850 2024-03-19T00:46:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3376187715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 msgr2=0x563d8c976350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c976350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009bd0 tx=0x7f1e9c011810 comp rx=0 tx=0).stop 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3376187715 shutdown_connections 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c978e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c976350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3376187715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c971280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3376187715 >> 172.21.15.7:0/3376187715 conn(0x563d8c967fc0 msgr2=0x563d8c9683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3376187715 shutdown_connections 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.558+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3376187715 wait complete. 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 -- start start 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 Processor -- start 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c970e60 0x563d8c97b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c971850 0x563d8c97b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 0x563d8c982890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563d8c979970 con 0x563d8c9769c0 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d8c88d160 con 0x563d8c971850 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563d8c979af0 con 0x563d8c970e60 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1eaadd2740 10 client.0 _ll_register_callbacks cb 0x563d8c8c9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1ea66a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 0x563d8c982890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1ea56a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c971850 0x563d8c97b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1ea66a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 0x563d8c982890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38204/0 (socket says 172.21.15.7:38204) 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1ea66a6640 1 -- 172.21.15.7:0/3528994701 learned_addr learned my addr 172.21.15.7:0/3528994701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.559+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c970e60 0x563d8c97b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1ea66a6640 1 -- 172.21.15.7:0/3528994701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c970e60 msgr2=0x563d8c97b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c970e60 0x563d8c97b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1ea66a6640 1 -- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c971850 msgr2=0x563d8c97b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c971850 0x563d8c97b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1ea66a6640 1 -- 172.21.15.7:0/3528994701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8c6af7d0 con 0x563d8c9769c0 2024-03-19T00:46:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 0x563d8c982890 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1e98037040 tx=0x7f1e980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aec225eddd94c33 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1e897fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1e897fa640 1 -- 172.21.15.7:0/3528994701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98052060 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1e897fa640 1 -- 172.21.15.7:0/3528994701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1e98004450 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d8c8dea40 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1e897fa640 1 -- 172.21.15.7:0/3528994701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e98059050 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 10 client.17238 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d8c5e53f0 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1e897fa640 1 -- 172.21.15.7:0/3528994701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563d8c8dea40 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1e897fa640 1 -- 172.21.15.7:0/3528994701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e9803f070 con 0x563d8c9769c0 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 10 client.17238 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 20 client.17238 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.560+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9871e0 0x563d8c989660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.561+0000 7f1eaadd2740 4 client.17238 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.561+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d8c8d8d90 con 0x563d8c9871e0 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.561+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9871e0 0x563d8c989660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.561+0000 7f1e897fa640 10 client.17238 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.561+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9871e0 0x563d8c989660 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed0611969ee7be8 server_cookie=c7ea212718a7e7d3 in_seq=0 out_seq=0 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1e897fa640 1 -- 172.21.15.7:0/3528994701 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563d8c8d8d90 con 0x563d8c9871e0 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1e897fa640 10 client.17238 handle_command_reply: tid=0 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 client.17238 shutdown 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 msgr2=0x563d8c982890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 0x563d8c982890 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1e98037040 tx=0x7f1e980095c0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 shutdown_connections 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9871e0 0x563d8c989660 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c9769c0 0x563d8c982890 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c971850 0x563d8c97b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1e897fa640 0 client.17238 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3528994701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c970e60 0x563d8c97b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.562+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 >> 172.21.15.7:0/3528994701 conn(0x563d8c967fc0 msgr2=0x563d8c9671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.563+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 shutdown_connections 2024-03-19T00:46:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.563+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3528994701 wait complete. 2024-03-19T00:46:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.563+0000 7f1eaadd2740 20 client.17238 trim_cache size 0 max 16384 2024-03-19T00:46:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 Processor -- start 2024-03-19T00:46:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 -- start start 2024-03-19T00:46:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c96b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c982850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c98df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01aa930 con 0x563d8c970e60 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01a8200 con 0x563d8c9769c0 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01a6050 con 0x563d8c971850 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c96b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c96b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38214/0 (socket says 172.21.15.7:38214) 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/1732705743 learned_addr learned my addr 172.21.15.7:0/1732705743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c98df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c982850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/1732705743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 msgr2=0x563d8c982850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c982850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/1732705743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 msgr2=0x563d8c98df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c98df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/1732705743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8c5a50f0 con 0x563d8c970e60 2024-03-19T00:46:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.580+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c98df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c96b930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009b20 tx=0x7f1e9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40eecf3cf94727b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1e8b7fe640 1 -- 172.21.15.7:0/1732705743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c032070 con 0x563d8c970e60 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1e8b7fe640 1 -- 172.21.15.7:0/1732705743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e9c003b00 con 0x563d8c970e60 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1e8b7fe640 1 -- 172.21.15.7:0/1732705743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c005040 con 0x563d8c970e60 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 -- 172.21.15.7:0/1732705743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 msgr2=0x563d8c96b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c96b930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009b20 tx=0x7f1e9c011810 comp rx=0 tx=0).stop 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 -- 172.21.15.7:0/1732705743 shutdown_connections 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c98df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c982850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/1732705743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c96b930 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 -- 172.21.15.7:0/1732705743 >> 172.21.15.7:0/1732705743 conn(0x563d8c967fc0 msgr2=0x563d8c96b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 -- 172.21.15.7:0/1732705743 shutdown_connections 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.581+0000 7f1eaadd2740 1 -- 172.21.15.7:0/1732705743 wait complete. 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 -- start start 2024-03-19T00:46:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 Processor -- start 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c9805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c980b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c981af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01a66c0 con 0x563d8c970e60 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01a7b30 con 0x563d8c9769c0 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01a6d90 con 0x563d8c971850 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1eaadd2740 10 client.0 _ll_register_callbacks cb 0x7f1e9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c9805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c9805f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38222/0 (socket says 172.21.15.7:38222) 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3500881386 learned_addr learned my addr 172.21.15.7:0/3500881386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea66a6640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c981af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c980b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3500881386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 msgr2=0x563d8c980b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c980b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 msgr2=0x563d8c981af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c981af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.582+0000 7f1ea5ea5640 1 -- 172.21.15.7:0/3500881386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8c9820e0 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1ea5ea5640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c9805f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c01b040 tx=0x7f1e9c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83582c94164f32c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1e897fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1e897fa640 1 -- 172.21.15.7:0/3500881386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c03b060 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1e897fa640 1 -- 172.21.15.7:0/3500881386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1e9c0052c0 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e9c003b00 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1e897fa640 1 -- 172.21.15.7:0/3500881386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e9c042050 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 10 client.17244 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d8c9edb30 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1e897fa640 1 -- 172.21.15.7:0/3500881386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e9c003b00 con 0x563d8c970e60 2024-03-19T00:46:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1e897fa640 1 -- 172.21.15.7:0/3500881386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e9c025420 con 0x563d8c970e60 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 10 client.17244 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 20 client.17244 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9ee280 0x563d8c9f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 4 client.17244 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d8c856540 con 0x563d8c9ee280 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.583+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9ee280 0x563d8c9f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1ea56a4640 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9ee280 0x563d8c9f06a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a168dc1b549e32 server_cookie=514225dc56802130 in_seq=0 out_seq=0 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1e897fa640 10 client.17244 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1e897fa640 1 -- 172.21.15.7:0/3500881386 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563d8c856540 con 0x563d8c9ee280 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1e897fa640 10 client.17244 handle_command_reply: tid=0 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 client.17244 shutdown 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 msgr2=0x563d8c9805f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c9805f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c01b040 tx=0x7f1e9c005f10 comp rx=0 tx=0).stop 2024-03-19T00:46:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 shutdown_connections 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d8c9769c0 0x563d8c981af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d8c971850 0x563d8c980b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x563d8c9ee280 0x563d8c9f06a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 --2- 172.21.15.7:0/3500881386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d8c970e60 0x563d8c9805f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1e897fa640 0 client.17244 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 >> 172.21.15.7:0/3500881386 conn(0x563d8c967fc0 msgr2=0x563d8c96bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 shutdown_connections 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 1 -- 172.21.15.7:0/3500881386 wait complete. 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.584+0000 7f1eaadd2740 20 client.17244 trim_cache size 0 max 16384 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:46:03.592 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "86723147-85f0-4159-98e1-78443f503882": { 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "86723147-85f0-4159-98e1-78443f503882", 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:46:03.593 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e8c0a2880 msgr2=0x7f1e8c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e8c0a2880 0x7f1e8c0a4d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x563d8c9720b0 tx=0x7f1e9802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 msgr2=0x7f1ea0197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 0x7f1ea0197f70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 shutdown_connections 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1e8c0a2880 0x7f1e8c0a4d40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea01033e0 0x7f1ea019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea0102a20 0x7f1ea0197f70 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 --2- 172.21.15.7:0/2271133022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea0101830 0x7f1ea0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:03.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 >> 172.21.15.7:0/2271133022 conn(0x7f1ea00fcda0 msgr2=0x7f1ea0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 shutdown_connections 2024-03-19T00:46:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:03.585+0000 7f1ea8109640 1 -- 172.21.15.7:0/2271133022 wait complete. 2024-03-19T00:46:03.596 INFO:tasks.cephfs.filesystem:scrub status for tag:86723147-85f0-4159-98e1-78443f503882 - {'path': '/', 'tag': '86723147-85f0-4159-98e1-78443f503882', 'options': 'recursive,force'} 2024-03-19T00:46:03.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:03 smithi007 ceph-mon[62854]: pgmap v594: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-03-19T00:46:04.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:03 smithi204 ceph-mon[66572]: pgmap v594: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-03-19T00:46:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:03 smithi044 ceph-mon[66366]: pgmap v594: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-03-19T00:46:04.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:04 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:46:04.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:04 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:04.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:04 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:46:04.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:04 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:46:04.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:04 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:04.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:04 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:04 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:46:05.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:04 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:04 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:46:05.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:04 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:46:05.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:04 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:04 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:04 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:46:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:04 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:04 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:46:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:04 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:46:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:04 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:04 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:05.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:05 smithi007 ceph-mon[62854]: pgmap v595: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:46:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:05 smithi204 ceph-mon[66572]: pgmap v595: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:46:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:05 smithi044 ceph-mon[66366]: pgmap v595: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-03-19T00:46:07.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:07 smithi007 ceph-mon[62854]: pgmap v596: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:07 smithi204 ceph-mon[66572]: pgmap v596: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:07 smithi044 ceph-mon[66366]: pgmap v596: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:09.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:09 smithi007 ceph-mon[62854]: pgmap v597: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:09 smithi204 ceph-mon[66572]: pgmap v597: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:09 smithi044 ceph-mon[66366]: pgmap v597: 129 pgs: 129 active+clean; 2.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:11.908 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:11 smithi044 ceph-mon[66366]: pgmap v598: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:46:11.908 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:11 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:46:11.908 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:11 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:46:11.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:11 smithi007 ceph-mon[62854]: pgmap v598: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:46:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:11 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:46:11.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:11 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:46:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:11 smithi204 ceph-mon[66572]: pgmap v598: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:46:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:11 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:46:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:11 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:46:13.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:13 smithi007 ceph-mon[62854]: pgmap v599: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:13 smithi204 ceph-mon[66572]: pgmap v599: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:13 smithi044 ceph-mon[66366]: pgmap v599: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:15.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:15 smithi007 ceph-mon[62854]: pgmap v600: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:16.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:15 smithi204 ceph-mon[66572]: pgmap v600: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:15 smithi044 ceph-mon[66366]: pgmap v600: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:17.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:17 smithi007 ceph-mon[62854]: pgmap v601: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:18.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:17 smithi204 ceph-mon[66572]: pgmap v601: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:17 smithi044 ceph-mon[66366]: pgmap v601: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:19.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:19 smithi007 ceph-mon[62854]: pgmap v602: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:20.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:19 smithi204 ceph-mon[66572]: pgmap v602: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:19 smithi044 ceph-mon[66366]: pgmap v602: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:21.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:21 smithi007 ceph-mon[62854]: pgmap v603: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:46:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:21 smithi204 ceph-mon[66572]: pgmap v603: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:46:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:21 smithi044 ceph-mon[66366]: pgmap v603: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:46:23.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:23 smithi007 ceph-mon[62854]: pgmap v604: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:23 smithi204 ceph-mon[66572]: pgmap v604: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:23 smithi044 ceph-mon[66366]: pgmap v604: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:25.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:25 smithi007 ceph-mon[62854]: pgmap v605: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:46:26.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:25 smithi204 ceph-mon[66572]: pgmap v605: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:46:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:25 smithi044 ceph-mon[66366]: pgmap v605: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:46:27.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:27 smithi007 ceph-mon[62854]: pgmap v606: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:28.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:27 smithi204 ceph-mon[66572]: pgmap v606: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:27 smithi044 ceph-mon[66366]: pgmap v606: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:29 smithi007 ceph-mon[62854]: pgmap v607: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:30.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:29 smithi204 ceph-mon[66572]: pgmap v607: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:29 smithi044 ceph-mon[66366]: pgmap v607: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:30.135 INFO:tasks.workunit.client.0.smithi007.stdout:375/600: open fd = 3 2024-03-19T00:46:30.135 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.135 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.135 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.135 INFO:tasks.workunit.client.0.smithi007.stdout:376/600: open fd = 3 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:377/600: open fd = 3 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:378/600: open fd = 3 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:379/600: open fd = 3 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:380/600: open fd = 3 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:381/600: open fd = 3 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:382/600: open fd = 3 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:383/600: open fd = 3 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.137 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:384/600: open fd = 3 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:385/600: open fd = 3 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:386/600: open fd = 3 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:387/600: open fd = 3 2024-03-19T00:46:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:388/600: open fd = 3 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:389/600: open fd = 3 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:390/600: open fd = 3 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.139 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:391/600: open fd = 3 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:392/600: open fd = 3 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:393/600: open fd = 3 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:394/600: open fd = 3 2024-03-19T00:46:30.140 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:395/600: open fd = 3 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:396/600: open fd = 3 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:397/600: open fd = 3 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:398/600: open fd = 3 2024-03-19T00:46:30.141 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:399/600: open fd = 3 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:400/600: open fd = 3 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:401/600: open fd = 3 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:402/600: open fd = 3 2024-03-19T00:46:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:403/600: open fd = 3 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:404/600: open fd = 3 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:405/600: open fd = 3 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:406/600: open fd = 3 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:407/600: open fd = 3 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:408/600: open fd = 3 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:409/600: open fd = 3 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.144 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:410/600: open fd = 3 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:411/600: open fd = 3 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:412/600: open fd = 3 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:413/600: open fd = 3 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.145 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:414/600: open fd = 3 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:415/600: open fd = 3 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:416/600: open fd = 3 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:417/600: open fd = 3 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:418/600: open fd = 3 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:419/600: open fd = 3 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:420/600: open fd = 3 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:421/600: open fd = 3 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:422/600: open fd = 3 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:423/600: open fd = 3 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:424/600: open fd = 3 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:425/600: open fd = 3 2024-03-19T00:46:30.148 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:426/600: open fd = 3 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:427/600: open fd = 3 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:428/600: open fd = 3 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:429/600: open fd = 3 2024-03-19T00:46:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:430/600: open fd = 3 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:431/600: open fd = 3 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:432/600: open fd = 3 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.150 INFO:tasks.workunit.client.0.smithi007.stdout:433/600: open fd = 3 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:434/600: open fd = 3 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:435/600: open fd = 3 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:436/600: open fd = 3 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:46:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:46:31.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:31 smithi007 ceph-mon[62854]: pgmap v608: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:46:32.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:31 smithi204 ceph-mon[66572]: pgmap v608: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:46:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:31 smithi044 ceph-mon[66366]: pgmap v608: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:46:33.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:46:33.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:46:33.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 -- 172.21.15.7:0/777919997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01033e0 msgr2=0x7f9ce0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 --2- 172.21.15.7:0/777919997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01033e0 0x7f9ce0109c70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009be0 tx=0x7f9cd002bef0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 -- 172.21.15.7:0/777919997 shutdown_connections 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 --2- 172.21.15.7:0/777919997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01033e0 0x7f9ce0109c70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 --2- 172.21.15.7:0/777919997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce0102a20 0x7f9ce0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 --2- 172.21.15.7:0/777919997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0101830 0x7f9ce0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 -- 172.21.15.7:0/777919997 >> 172.21.15.7:0/777919997 conn(0x7f9ce00fcda0 msgr2=0x7f9ce00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 -- 172.21.15.7:0/777919997 shutdown_connections 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 -- 172.21.15.7:0/777919997 wait complete. 2024-03-19T00:46:33.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 Processor -- start 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 -- start start 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce0101830 0x7f9ce0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.718+0000 7f9ce792f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 0x7f9ce0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce792f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce01033e0 0x7f9ce019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce792f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce010d340 con 0x7f9ce0102a20 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce792f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce010d1c0 con 0x7f9ce01033e0 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce792f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce010d4c0 con 0x7f9ce0101830 2024-03-19T00:46:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 0x7f9ce0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 0x7f9ce0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58824/0 (socket says 172.21.15.7:58824) 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 -- 172.21.15.7:0/899991211 learned_addr learned my addr 172.21.15.7:0/899991211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce0101830 0x7f9ce0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce01033e0 0x7f9ce019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 -- 172.21.15.7:0/899991211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce0101830 msgr2=0x7f9ce0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce0101830 0x7f9ce0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 -- 172.21.15.7:0/899991211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce01033e0 msgr2=0x7f9ce019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce01033e0 0x7f9ce019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 -- 172.21.15.7:0/899991211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce019f630 con 0x7f9ce0102a20 2024-03-19T00:46:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 0x7f9ce0197f50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9f0 tx=0x7f9cd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9cde7fc640 1 -- 172.21.15.7:0/899991211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400cd80 con 0x7f9ce0102a20 2024-03-19T00:46:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce019f920 con 0x7f9ce0102a20 2024-03-19T00:46:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9cde7fc640 1 -- 172.21.15.7:0/899991211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd40108e0 con 0x7f9ce0102a20 2024-03-19T00:46:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9cde7fc640 1 -- 172.21.15.7:0/899991211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4018920 con 0x7f9ce0102a20 2024-03-19T00:46:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.719+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce019fe70 con 0x7f9ce0102a20 2024-03-19T00:46:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.722+0000 7f9cde7fc640 1 -- 172.21.15.7:0/899991211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f9cd4010a90 con 0x7f9ce0102a20 2024-03-19T00:46:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.722+0000 7f9cea5f0740 1 Processor -- start 2024-03-19T00:46:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 -- start start 2024-03-19T00:46:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356eeae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356eeff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356ef2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556356e4f500 con 0x556356eeb440 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556356df7be0 con 0x556356eeaa50 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556356e4b5a0 con 0x556356ef05b0 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cde7fc640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9cc80a2800 0x7f9cc80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356eeff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356eeff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58826/0 (socket says 172.21.15.7:58826) 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 -- 172.21.15.7:0/2363449606 learned_addr learned my addr 172.21.15.7:0/2363449606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356ef2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356eeae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9cde7fc640 1 -- 172.21.15.7:0/899991211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9cd40cf110 con 0x7f9ce0102a20 2024-03-19T00:46:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9cc80a2800 0x7f9cc80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 -- 172.21.15.7:0/2363449606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 msgr2=0x556356ef2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356ef2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 -- 172.21.15.7:0/2363449606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 msgr2=0x556356eeae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356eeae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.723+0000 7f9ce56cb640 1 -- 172.21.15.7:0/2363449606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556356e19000 con 0x556356eeb440 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356eeff40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009c80 tx=0x7f9cd8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef89de0acaa34a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9cc80a2800 0x7f9cc80a4cc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0004af0 tx=0x7f9cd0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cc77fe640 1 -- 172.21.15.7:0/2363449606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8005220 con 0x556356eeb440 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cc77fe640 1 -- 172.21.15.7:0/2363449606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd8009e30 con 0x556356eeb440 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cc77fe640 1 -- 172.21.15.7:0/2363449606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8019740 con 0x556356eeb440 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2363449606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 msgr2=0x556356eeff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356eeff40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009c80 tx=0x7f9cd8011810 comp rx=0 tx=0).stop 2024-03-19T00:46:33.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2363449606 shutdown_connections 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356ef2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356eeff40 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.724+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2363449606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356eeae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2363449606 >> 172.21.15.7:0/2363449606 conn(0x556356ee5fc0 msgr2=0x556356ee63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2363449606 shutdown_connections 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2363449606 wait complete. 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- start start 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 Processor -- start 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeaa50 0x556356ef4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356ef5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356ef05b0 0x556356efc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556356ef3560 con 0x556356eeb440 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556356e0b160 con 0x556356ef05b0 2024-03-19T00:46:33.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556356ef36e0 con 0x556356eeaa50 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9cea5f0740 10 client.0 _ll_register_callbacks cb 0x556356e47a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9ce4eca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356ef5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9ce4eca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356ef5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58834/0 (socket says 172.21.15.7:58834) 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.725+0000 7f9ce56cb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeaa50 0x556356ef4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 -- 172.21.15.7:0/2198652674 learned_addr learned my addr 172.21.15.7:0/2198652674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356ef05b0 0x556356efc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 -- 172.21.15.7:0/2198652674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeaa50 msgr2=0x556356ef4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeaa50 0x556356ef4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 -- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356ef05b0 msgr2=0x556356efc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356ef05b0 0x556356efc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 -- 172.21.15.7:0/2198652674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556356c2d7d0 con 0x556356eeb440 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356ef5380 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400afd0 tx=0x7f9cd4025b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179664d54712c010 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/2198652674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd409f840 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/2198652674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9cd40a3a00 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd8009e30 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/2198652674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9cd40a3e90 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cea5f0740 10 client.17256 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556356b633f0 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.726+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/2198652674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd8009e30 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/2198652674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9cd409abf0 con 0x556356eeb440 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cea5f0740 10 client.17256 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cea5f0740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cea5f0740 20 client.17256 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f00d80 0x556356f031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cea5f0740 4 client.17256 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:33.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556356e5ca40 con 0x556356f00d80 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f00d80 0x556356f031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9cc4ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.727+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f00d80 0x556356f031d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd9d5658f2eb6fc server_cookie=51f74e4bbe8ee0a5 in_seq=0 out_seq=0 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/2198652674 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556356e5ca40 con 0x556356f00d80 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cc4ff9640 10 client.17256 handle_command_reply: tid=0 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 client.17256 shutdown 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 msgr2=0x556356ef5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356ef5380 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400afd0 tx=0x7f9cd4025b40 comp rx=0 tx=0).stop 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 shutdown_connections 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356ef05b0 0x556356efc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f00d80 0x556356f031d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356ef5380 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/2198652674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeaa50 0x556356ef4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cc4ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 >> 172.21.15.7:0/2198652674 conn(0x556356ee5fc0 msgr2=0x556356ee51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 shutdown_connections 2024-03-19T00:46:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 1 -- 172.21.15.7:0/2198652674 wait complete. 2024-03-19T00:46:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.728+0000 7f9cea5f0740 20 client.17256 trim_cache size 0 max 16384 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 Processor -- start 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 -- start start 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356ee4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeb440 0x556356ee4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 0x556356ee53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01acbc0 con 0x556356ef05b0 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01aa490 con 0x556356eeaa50 2024-03-19T00:46:33.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01a82e0 con 0x556356eeb440 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce5ecc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 0x556356ee53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce5ecc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 0x556356ee53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58842/0 (socket says 172.21.15.7:58842) 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356ee4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce5ecc640 1 -- 172.21.15.7:0/634407664 learned_addr learned my addr 172.21.15.7:0/634407664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeb440 0x556356ee4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 -- 172.21.15.7:0/634407664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeb440 msgr2=0x556356ee4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeb440 0x556356ee4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 -- 172.21.15.7:0/634407664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 msgr2=0x556356ee53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 0x556356ee53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 -- 172.21.15.7:0/634407664 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556356b230f0 con 0x556356eeaa50 2024-03-19T00:46:33.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 0x556356ee53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.746+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356ee4950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x556356ee6f30 tx=0x7f9cd8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce780fc11ee889 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cc6ffd640 1 -- 172.21.15.7:0/634407664 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8009e30 con 0x556356eeaa50 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cc6ffd640 1 -- 172.21.15.7:0/634407664 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd80054e0 con 0x556356eeaa50 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cc6ffd640 1 -- 172.21.15.7:0/634407664 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd80048a0 con 0x556356eeaa50 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 -- 172.21.15.7:0/634407664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 msgr2=0x556356ee4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356ee4950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x556356ee6f30 tx=0x7f9cd8005fd0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 -- 172.21.15.7:0/634407664 shutdown_connections 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ef05b0 0x556356ee53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356eeb440 0x556356ee4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/634407664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356ee4950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 -- 172.21.15.7:0/634407664 >> 172.21.15.7:0/634407664 conn(0x556356ee5fc0 msgr2=0x556356ef4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 -- 172.21.15.7:0/634407664 shutdown_connections 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 -- 172.21.15.7:0/634407664 wait complete. 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.747+0000 7f9cea5f0740 1 -- start start 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 Processor -- start 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356f62af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356f63030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356f6a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01a8950 con 0x556356eeb440 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01a9dc0 con 0x556356eeaa50 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01a9020 con 0x556356ef05b0 2024-03-19T00:46:33.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9cea5f0740 10 client.0 _ll_register_callbacks cb 0x7f9cd4097a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce4eca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356f63030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce56cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356f62af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce4eca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356f63030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58852/0 (socket says 172.21.15.7:58852) 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce4eca640 1 -- 172.21.15.7:0/1201024229 learned_addr learned my addr 172.21.15.7:0/1201024229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce5ecc640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356f6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce56cb640 1 -- 172.21.15.7:0/1201024229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 msgr2=0x556356f6a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356f6a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce56cb640 1 -- 172.21.15.7:0/1201024229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 msgr2=0x556356f63030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356f63030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce56cb640 1 -- 172.21.15.7:0/1201024229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556356f6a5f0 con 0x556356eeaa50 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.748+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356f63030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9ce56cb640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356f62af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x556356eff940 tx=0x7f9cd8005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc9311806483a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/1201024229 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8011d10 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd80054e0 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/1201024229 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9cd8009e30 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/1201024229 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9cd802b420 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 10 client.27107 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556356f6b900 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/1201024229 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd80054e0 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/1201024229 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9cd8004db0 con 0x556356eeaa50 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 10 client.27107 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 10 client.27107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 20 client.27107 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:33.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f6c060 0x556356f6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 4 client.27107 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.749+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556356e56d90 con 0x556356f6c060 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f6c060 0x556356f6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9ce4eca640 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f6c060 0x556356f6e480 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabb5f451bed8b72 server_cookie=5aeee117863d7b2a in_seq=0 out_seq=0 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cc4ff9640 10 client.27107 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cc4ff9640 1 -- 172.21.15.7:0/1201024229 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556356e56d90 con 0x556356f6c060 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cc4ff9640 10 client.27107 handle_command_reply: tid=0 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 client.27107 shutdown 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 msgr2=0x556356f62af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356f62af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x556356eff940 tx=0x7f9cd8005d30 comp rx=0 tx=0).stop 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 shutdown_connections 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556356ef05b0 0x556356f6a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556356f6c060 0x556356f6e480 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356eeb440 0x556356f63030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cc4ff9640 0 client.27107 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 --2- 172.21.15.7:0/1201024229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556356eeaa50 0x556356f62af0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 >> 172.21.15.7:0/1201024229 conn(0x556356ee5fc0 msgr2=0x556356ee9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 shutdown_connections 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 1 -- 172.21.15.7:0/1201024229 wait complete. 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.750+0000 7f9cea5f0740 20 client.27107 trim_cache size 0 max 16384 2024-03-19T00:46:33.760 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:46:33.761 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:46:33.761 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:46:33.761 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9cc80a2800 msgr2=0x7f9cc80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9cc80a2800 0x7f9cc80a4cc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0004af0 tx=0x7f9cd0005fd0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 msgr2=0x7f9ce0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 0x7f9ce0197f50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9f0 tx=0x7f9cd400eeb0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 shutdown_connections 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f9cc80a2800 0x7f9cc80a4cc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce01033e0 0x7f9ce019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce0102a20 0x7f9ce0197f50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 --2- 172.21.15.7:0/899991211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ce0101830 0x7f9ce0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 >> 172.21.15.7:0/899991211 conn(0x7f9ce00fcda0 msgr2=0x7f9ce0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 shutdown_connections 2024-03-19T00:46:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.751+0000 7f9ce792f640 1 -- 172.21.15.7:0/899991211 wait complete. 2024-03-19T00:46:33.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:46:33.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:46:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 -- 172.21.15.7:0/1292022938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec100750 msgr2=0x7fb6ec100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1292022938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec100750 0x7fb6ec100b70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009900 tx=0x7fb6dc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 -- 172.21.15.7:0/1292022938 shutdown_connections 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1292022938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec102300 0x7fb6ec108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1292022938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6ec101940 0x7fb6ec101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1292022938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec100750 0x7fb6ec100b70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 -- 172.21.15.7:0/1292022938 >> 172.21.15.7:0/1292022938 conn(0x7fb6ec0fbce0 msgr2=0x7fb6ec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 -- 172.21.15.7:0/1292022938 shutdown_connections 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 -- 172.21.15.7:0/1292022938 wait complete. 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.866+0000 7fb6f1577640 1 Processor -- start 2024-03-19T00:46:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 -- start start 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100750 0x7fb6ec197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 0x7fb6ec198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6ec102300 0x7fb6ec19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d3a0 con 0x7fb6ec101940 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d220 con 0x7fb6ec102300 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d520 con 0x7fb6ec100750 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 0x7fb6ec198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 0x7fb6ec198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58866/0 (socket says 172.21.15.7:58866) 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 -- 172.21.15.7:0/1230157727 learned_addr learned my addr 172.21.15.7:0/1230157727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6eaffd640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100750 0x7fb6ec197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6eb7fe640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6ec102300 0x7fb6ec19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 -- 172.21.15.7:0/1230157727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100750 msgr2=0x7fb6ec197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100750 0x7fb6ec197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 -- 172.21.15.7:0/1230157727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6ec102300 msgr2=0x7fb6ec19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6ec102300 0x7fb6ec19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 -- 172.21.15.7:0/1230157727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec19f6e0 con 0x7fb6ec101940 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6ea7fc640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 0x7fb6ec198000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000e9f0 tx=0x7fb6e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000ccc0 con 0x7fb6ec101940 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6e00108e0 con 0x7fb6ec101940 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec19f9d0 con 0x7fb6ec101940 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00187c0 con 0x7fb6ec101940 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.867+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec19ff20 con 0x7fb6ec101940 2024-03-19T00:46:33.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.868+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec107bb0 con 0x7fb6ec101940 2024-03-19T00:46:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.869+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb6e0010a90 con 0x7fb6ec101940 2024-03-19T00:46:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.870+0000 7fb6d3fff640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb6d40a2800 0x7fb6d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.870+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb6e00cf600 con 0x7fb6ec101940 2024-03-19T00:46:33.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.870+0000 7fb6eaffd640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb6d40a2800 0x7fb6d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:33.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.871+0000 7fb6eaffd640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb6d40a2800 0x7fb6d40a4cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005ab0 tx=0x7fb6dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:33.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:33.874+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb6e009c340 con 0x7fb6ec101940 2024-03-19T00:46:33.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:33 smithi007 ceph-mon[62854]: pgmap v609: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:34.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:33 smithi204 ceph-mon[66572]: pgmap v609: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:34.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.095+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6ec02d310 con 0x7fb6ec101940 2024-03-19T00:46:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.096+0000 7fb6d3fff640 1 -- 172.21.15.7:0/1230157727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fb6e009bc40 con 0x7fb6ec101940 2024-03-19T00:46:34.097 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:46:34.097 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:46:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:46:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb6d40a2800 msgr2=0x7fb6d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb6d40a2800 0x7fb6d40a4cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005ab0 tx=0x7fb6dc005fd0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 msgr2=0x7fb6ec198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 0x7fb6ec198000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000e9f0 tx=0x7fb6e000edf0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 shutdown_connections 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb6d40a2800 0x7fb6d40a4cc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6ec102300 0x7fb6ec19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6ec101940 0x7fb6ec198000 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 --2- 172.21.15.7:0/1230157727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6ec100750 0x7fb6ec197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 >> 172.21.15.7:0/1230157727 conn(0x7fb6ec0fbce0 msgr2=0x7fb6ec104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 shutdown_connections 2024-03-19T00:46:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.098+0000 7fb6f1577640 1 -- 172.21.15.7:0/1230157727 wait complete. 2024-03-19T00:46:34.105 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:33 smithi044 ceph-mon[66366]: pgmap v609: 129 pgs: 129 active+clean; 2.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:34.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:46:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 -- 172.21.15.7:0/1679428035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc101930 msgr2=0x7f14dc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 --2- 172.21.15.7:0/1679428035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc101930 0x7f14dc101db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 -- 172.21.15.7:0/1679428035 shutdown_connections 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 --2- 172.21.15.7:0/1679428035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14dc1022f0 0x7f14dc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 --2- 172.21.15.7:0/1679428035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc101930 0x7f14dc101db0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 --2- 172.21.15.7:0/1679428035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc100740 0x7f14dc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 -- 172.21.15.7:0/1679428035 >> 172.21.15.7:0/1679428035 conn(0x7f14dc0fbc90 msgr2=0x7f14dc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 -- 172.21.15.7:0/1679428035 shutdown_connections 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.214+0000 7f14e13c4640 1 -- 172.21.15.7:0/1679428035 wait complete. 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 Processor -- start 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 -- start start 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 0x7f14dc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc101930 0x7f14dc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14dc1022f0 0x7f14dc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc10c270 con 0x7f14dc100740 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc10c0f0 con 0x7f14dc101930 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14e13c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc10c3f0 con 0x7f14dc1022f0 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 0x7f14dc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 0x7f14dc197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58894/0 (socket says 172.21.15.7:58894) 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 -- 172.21.15.7:0/2481345871 learned_addr learned my addr 172.21.15.7:0/2481345871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14d27fc640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc101930 0x7f14dc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14db7fe640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14dc1022f0 0x7f14dc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 -- 172.21.15.7:0/2481345871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14dc1022f0 msgr2=0x7f14dc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14dc1022f0 0x7f14dc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 -- 172.21.15.7:0/2481345871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc101930 msgr2=0x7f14dc197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc101930 0x7f14dc197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.215+0000 7f14daffd640 1 -- 172.21.15.7:0/2481345871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc19f670 con 0x7f14dc100740 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.216+0000 7f14daffd640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 0x7f14dc197a50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00e9c0 tx=0x7f14cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.216+0000 7f14d8ff9640 1 -- 172.21.15.7:0/2481345871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00cd80 con 0x7f14dc100740 2024-03-19T00:46:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.216+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc19f960 con 0x7f14dc100740 2024-03-19T00:46:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.216+0000 7f14d8ff9640 1 -- 172.21.15.7:0/2481345871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14cc0035f0 con 0x7f14dc100740 2024-03-19T00:46:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.216+0000 7f14d8ff9640 1 -- 172.21.15.7:0/2481345871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc010690 con 0x7f14dc100740 2024-03-19T00:46:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.216+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc19fed0 con 0x7f14dc100740 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.218+0000 7f14d8ff9640 1 -- 172.21.15.7:0/2481345871 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f14cc0037a0 con 0x7f14dc100740 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.218+0000 7f14e3fac740 1 Processor -- start 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.218+0000 7f14e3fac740 1 -- start start 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d57a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d57f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14e3fac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea5d4de4b0 con 0x55ea5d57ace0 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14e3fac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ea5d486d70 con 0x55ea5d57fe50 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14e3fac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea5d4da550 con 0x55ea5d57a2f0 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14daffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d57f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d57a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d57a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:51386/0 (socket says 172.21.15.7:51386) 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 -- 172.21.15.7:0/216810605 learned_addr learned my addr 172.21.15.7:0/216810605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14d27fc640 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14d8ff9640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14c80a2800 0x7f14c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 -- 172.21.15.7:0/216810605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 msgr2=0x55ea5d5822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 -- 172.21.15.7:0/216810605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 msgr2=0x55ea5d57f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d57f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 -- 172.21.15.7:0/216810605 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea5d4a9210 con 0x55ea5d57a2f0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14c80a2800 0x7f14c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14d8ff9640 1 -- 172.21.15.7:0/2481345871 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14cc0cf1b0 con 0x7f14dc100740 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14db7fe640 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d57a710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f14d4011590 tx=0x7f14d4011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571b4ec2a733ed51 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.219+0000 7f14d17fa640 1 -- 172.21.15.7:0/216810605 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4017050 con 0x55ea5d57a2f0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14d17fa640 1 -- 172.21.15.7:0/216810605 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d400fcf0 con 0x55ea5d57a2f0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14d17fa640 1 -- 172.21.15.7:0/216810605 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d401b9f0 con 0x55ea5d57a2f0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 -- 172.21.15.7:0/216810605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 msgr2=0x55ea5d57a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14db7fe640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14c80a2800 0x7f14c80a4cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f14d4022000 tx=0x7f14d4023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d57a710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f14d4011590 tx=0x7f14d4011a50 comp rx=0 tx=0).stop 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 -- 172.21.15.7:0/216810605 shutdown_connections 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d57f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 --2- 172.21.15.7:0/216810605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d57a710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 -- 172.21.15.7:0/216810605 >> 172.21.15.7:0/216810605 conn(0x55ea5d575860 msgr2=0x55ea5d575c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 -- 172.21.15.7:0/216810605 shutdown_connections 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.220+0000 7f14e3fac740 1 -- 172.21.15.7:0/216810605 wait complete. 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 -- start start 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 Processor -- start 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57ace0 0x55ea5d584d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 0x55ea5d58bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea5d582e00 con 0x55ea5d57fe50 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ea5d49a310 con 0x55ea5d57ace0 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea5d582f80 con 0x55ea5d57a2f0 2024-03-19T00:46:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14e3fac740 10 client.0 _ll_register_callbacks cb 0x55ea5d48e2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 0x55ea5d58bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 0x55ea5d58bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58910/0 (socket says 172.21.15.7:58910) 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 -- 172.21.15.7:0/1101000163 learned_addr learned my addr 172.21.15.7:0/1101000163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14d27fc640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57ace0 0x55ea5d584d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14daffd640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 -- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 msgr2=0x55ea5d5847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 -- 172.21.15.7:0/1101000163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57ace0 msgr2=0x55ea5d584d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57ace0 0x55ea5d584d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 -- 172.21.15.7:0/1101000163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea5d1f23f0 con 0x55ea5d57fe50 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14db7fe640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 0x55ea5d58bd60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000f80 tx=0x7f14d401b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4619ef4b08f3db2f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.221+0000 7f14c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14c27fc640 1 -- 172.21.15.7:0/1101000163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4035060 con 0x55ea5d57fe50 2024-03-19T00:46:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14c27fc640 1 -- 172.21.15.7:0/1101000163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14d4004bf0 con 0x55ea5d57fe50 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d400fcf0 con 0x55ea5d57fe50 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14c27fc640 1 -- 172.21.15.7:0/1101000163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14d403c050 con 0x55ea5d57fe50 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 10 client.17280 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea5d2bc7d0 con 0x55ea5d57fe50 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14c27fc640 1 -- 172.21.15.7:0/1101000163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d400fcf0 con 0x55ea5d57fe50 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14c27fc640 1 -- 172.21.15.7:0/1101000163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14d401a070 con 0x55ea5d57fe50 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 10 client.17280 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 10 client.17280 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 20 client.17280 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d590410 0x55ea5d592870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 4 client.17280 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.222+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea5d53c3e0 con 0x55ea5d590410 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.223+0000 7f14daffd640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d590410 0x55ea5d592870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.223+0000 7f14c27fc640 10 client.17280 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.223+0000 7f14daffd640 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d590410 0x55ea5d592870 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b012b2cc770174 server_cookie=fa8f05421d0c5fdf in_seq=0 out_seq=0 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.223+0000 7f14c27fc640 1 -- 172.21.15.7:0/1101000163 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ea5d53c3e0 con 0x55ea5d590410 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.223+0000 7f14c27fc640 10 client.17280 handle_command_reply: tid=0 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 client.17280 shutdown 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 msgr2=0x55ea5d58bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 0x55ea5d58bd60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000f80 tx=0x7f14d401b9a0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 shutdown_connections 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d590410 0x55ea5d592870 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57fe50 0x55ea5d58bd60 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57ace0 0x55ea5d584d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1101000163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14c27fc640 0 client.17280 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:34.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 >> 172.21.15.7:0/1101000163 conn(0x55ea5d575860 msgr2=0x55ea5d574ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 shutdown_connections 2024-03-19T00:46:34.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 1 -- 172.21.15.7:0/1101000163 wait complete. 2024-03-19T00:46:34.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.224+0000 7f14e3fac740 20 client.17280 trim_cache size 0 max 16384 2024-03-19T00:46:34.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 Processor -- start 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 -- start start 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d574a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d597130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d597670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1acb40 con 0x55ea5d57ace0 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1aa410 con 0x55ea5d57fe50 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.237+0000 7f14e3fac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1a8260 con 0x55ea5d57a2f0 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d597130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d597130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58918/0 (socket says 172.21.15.7:58918) 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d597670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 -- 172.21.15.7:0/1941723880 learned_addr learned my addr 172.21.15.7:0/1941723880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14daffd640 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d574a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 -- 172.21.15.7:0/1941723880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 msgr2=0x55ea5d574a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d574a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 -- 172.21.15.7:0/1941723880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 msgr2=0x55ea5d597670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d597670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 -- 172.21.15.7:0/1941723880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea5d54b880 con 0x55ea5d57ace0 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14db7fe640 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d597670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d27fc640 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d597130 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d645525569433bc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d0ff9640 1 -- 172.21.15.7:0/1941723880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c402cb10 con 0x55ea5d57ace0 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d0ff9640 1 -- 172.21.15.7:0/1941723880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c4005040 con 0x55ea5d57ace0 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.238+0000 7f14d0ff9640 1 -- 172.21.15.7:0/1941723880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4004900 con 0x55ea5d57ace0 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- 172.21.15.7:0/1941723880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 msgr2=0x55ea5d597130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d597130 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402c8e0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- 172.21.15.7:0/1941723880 shutdown_connections 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d597670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57ace0 0x55ea5d597130 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- 172.21.15.7:0/1941723880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57a2f0 0x55ea5d574a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- 172.21.15.7:0/1941723880 >> 172.21.15.7:0/1941723880 conn(0x55ea5d575860 msgr2=0x55ea5d574310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- 172.21.15.7:0/1941723880 shutdown_connections 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- 172.21.15.7:0/1941723880 wait complete. 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- start start 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 Processor -- start 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57ace0 0x55ea5d5f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1a88d0 con 0x55ea5d57a2f0 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1a9d40 con 0x55ea5d57fe50 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.239+0000 7f14e3fac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1a8fa0 con 0x55ea5d57ace0 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14e3fac740 10 client.0 _ll_register_callbacks cb 0x7f14cc0d2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58922/0 (socket says 172.21.15.7:58922) 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 -- 172.21.15.7:0/297041765 learned_addr learned my addr 172.21.15.7:0/297041765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14db7fe640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14d27fc640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57ace0 0x55ea5d5f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 -- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57ace0 msgr2=0x55ea5d5f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57ace0 0x55ea5d5f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 -- 172.21.15.7:0/297041765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 msgr2=0x55ea5d5f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 -- 172.21.15.7:0/297041765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea5d5f99f0 con 0x55ea5d57a2f0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14daffd640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5f1e80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f14cc097bf0 tx=0x7f14cc0d2bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924ae8811ea399e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14c27fc640 1 -- 172.21.15.7:0/297041765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc003b30 con 0x55ea5d57a2f0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14c27fc640 1 -- 172.21.15.7:0/297041765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14cc0d2880 con 0x55ea5d57a2f0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c4005040 con 0x55ea5d57a2f0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14c27fc640 1 -- 172.21.15.7:0/297041765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14cc09be90 con 0x55ea5d57a2f0 2024-03-19T00:46:34.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14e3fac740 10 client.17286 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea5d5fadc0 con 0x55ea5d57a2f0 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.240+0000 7f14c27fc640 1 -- 172.21.15.7:0/297041765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c4005040 con 0x55ea5d57a2f0 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14c27fc640 1 -- 172.21.15.7:0/297041765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14cc0d2ce0 con 0x55ea5d57a2f0 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 10 client.17286 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 10 client.17286 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 20 client.17286 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d5fb740 0x55ea5d5fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 4 client.17286 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea5d4eb9f0 con 0x55ea5d5fb740 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14d27fc640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d5fb740 0x55ea5d5fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14c27fc640 10 client.17286 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14d27fc640 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d5fb740 0x55ea5d5fdb60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c461ccced0395b0 server_cookie=7e92a8ce22c84145 in_seq=0 out_seq=0 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14c27fc640 1 -- 172.21.15.7:0/297041765 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea5d4eb9f0 con 0x55ea5d5fb740 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14c27fc640 10 client.17286 handle_command_reply: tid=0 2024-03-19T00:46:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 client.17286 shutdown 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 msgr2=0x55ea5d5f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5f1e80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f14cc097bf0 tx=0x7f14cc0d2bf0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 shutdown_connections 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55ea5d5fb740 0x55ea5d5fdb60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea5d57fe50 0x55ea5d5f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea5d57ace0 0x55ea5d5f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14c27fc640 0 client.17286 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 --2- 172.21.15.7:0/297041765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea5d57a2f0 0x55ea5d5f1e80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.241+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 >> 172.21.15.7:0/297041765 conn(0x55ea5d575860 msgr2=0x55ea5d574ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 shutdown_connections 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e3fac740 1 -- 172.21.15.7:0/297041765 wait complete. 2024-03-19T00:46:34.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e3fac740 20 client.17286 trim_cache size 0 max 16384 2024-03-19T00:46:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:46:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14c80a2800 msgr2=0x7f14c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14c80a2800 0x7f14c80a4cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f14d4022000 tx=0x7f14d4023000 comp rx=0 tx=0).stop 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 msgr2=0x7f14dc197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 0x7f14dc197a50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00e9c0 tx=0x7f14cc00ee80 comp rx=0 tx=0).stop 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 shutdown_connections 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f14c80a2800 0x7f14c80a4cc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14dc1022f0 0x7f14dc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc101930 0x7f14dc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 --2- 172.21.15.7:0/2481345871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc100740 0x7f14dc197a50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 >> 172.21.15.7:0/2481345871 conn(0x7f14dc0fbc90 msgr2=0x7f14dc0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 shutdown_connections 2024-03-19T00:46:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.242+0000 7f14e13c4640 1 -- 172.21.15.7:0/2481345871 wait complete. 2024-03-19T00:46:34.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:46:34.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 -- 172.21.15.7:0/1700392342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201022f0 msgr2=0x7fae20108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 --2- 172.21.15.7:0/1700392342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201022f0 0x7fae20108b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae18009860 tx=0x7fae1802bb70 comp rx=0 tx=0).stop 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 -- 172.21.15.7:0/1700392342 shutdown_connections 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 --2- 172.21.15.7:0/1700392342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201022f0 0x7fae20108b80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 --2- 172.21.15.7:0/1700392342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20101930 0x7fae20101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 --2- 172.21.15.7:0/1700392342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae20100740 0x7fae20100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 -- 172.21.15.7:0/1700392342 >> 172.21.15.7:0/1700392342 conn(0x7fae200fbc90 msgr2=0x7fae200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 -- 172.21.15.7:0/1700392342 shutdown_connections 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.357+0000 7fae28485640 1 -- 172.21.15.7:0/1700392342 wait complete. 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 Processor -- start 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 -- start start 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 0x7fae20197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae20101930 0x7fae20197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 0x7fae2019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae2010d390 con 0x7fae201022f0 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae2010d210 con 0x7fae20100740 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae28485640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae2010d510 con 0x7fae20101930 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 0x7fae2019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 0x7fae20197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 0x7fae2019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58948/0 (socket says 172.21.15.7:58948) 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 0x7fae20197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57232/0 (socket says 172.21.15.7:57232) 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26a22640 1 -- 172.21.15.7:0/1070765338 learned_addr learned my addr 172.21.15.7:0/1070765338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae25a20640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae20101930 0x7fae20197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 -- 172.21.15.7:0/1070765338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae20101930 msgr2=0x7fae20197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae20101930 0x7fae20197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 -- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 msgr2=0x7fae2019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 0x7fae2019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.358+0000 7fae26221640 1 -- 172.21.15.7:0/1070765338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2019f5a0 con 0x7fae20100740 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae26a22640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 0x7fae2019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae26221640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 0x7fae20197980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae10005fd0 tx=0x7fae1000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae1f7fe640 1 -- 172.21.15.7:0/1070765338 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10003970 con 0x7fae20100740 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2019f890 con 0x7fae20100740 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae1f7fe640 1 -- 172.21.15.7:0/1070765338 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10004020 con 0x7fae20100740 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae1f7fe640 1 -- 172.21.15.7:0/1070765338 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10010720 con 0x7fae20100740 2024-03-19T00:46:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.359+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2019fd90 con 0x7fae20100740 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.361+0000 7fae1f7fe640 1 -- 172.21.15.7:0/1070765338 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fae100041d0 con 0x7fae20100740 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.361+0000 7fae2b07a740 1 Processor -- start 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 -- start start 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bde2f0 0x558206bde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 0x558206be37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206be62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558206b424b0 con 0x558206bdece0 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558206aead70 con 0x558206be3e50 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae2b07a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558206b3e550 con 0x558206bde2f0 2024-03-19T00:46:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae1f7fe640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae080a2800 0x7fae080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae26221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 0x558206be37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae25a20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206be62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae26221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 0x558206be37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58962/0 (socket says 172.21.15.7:58962) 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae25a20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206be62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57234/0 (socket says 172.21.15.7:57234) 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae26a22640 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bde2f0 0x558206bde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae26221640 1 -- 172.21.15.7:0/974690065 learned_addr learned my addr 172.21.15.7:0/974690065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae26a22640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae080a2800 0x7fae080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae1f7fe640 1 -- 172.21.15.7:0/1070765338 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae100cf600 con 0x7fae20100740 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.362+0000 7fae26a22640 1 -- 172.21.15.7:0/974690065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 msgr2=0x558206be62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26a22640 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206be62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26a22640 1 -- 172.21.15.7:0/974690065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 msgr2=0x558206be37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26a22640 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 0x558206be37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26a22640 1 -- 172.21.15.7:0/974690065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558206b0d210 con 0x558206bde2f0 2024-03-19T00:46:34.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26221640 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 0x558206be37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26a22640 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bde2f0 0x558206bde710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae180349e0 tx=0x7fae18004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2176e0d784e0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae26a22640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae080a2800 0x7fae080a4cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fae180095f0 tx=0x7fae18048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae1cbfe640 1 -- 172.21.15.7:0/974690065 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18039040 con 0x558206bde2f0 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae1cbfe640 1 -- 172.21.15.7:0/974690065 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18051070 con 0x558206bde2f0 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.363+0000 7fae1cbfe640 1 -- 172.21.15.7:0/974690065 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1804c430 con 0x558206bde2f0 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- 172.21.15.7:0/974690065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bde2f0 msgr2=0x558206bde710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bde2f0 0x558206bde710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae180349e0 tx=0x7fae18004310 comp rx=0 tx=0).stop 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- 172.21.15.7:0/974690065 shutdown_connections 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206be62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bdece0 0x558206be37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- 172.21.15.7:0/974690065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bde2f0 0x558206bde710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- 172.21.15.7:0/974690065 >> 172.21.15.7:0/974690065 conn(0x558206bd9860 msgr2=0x558206bd9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- 172.21.15.7:0/974690065 shutdown_connections 2024-03-19T00:46:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- 172.21.15.7:0/974690065 wait complete. 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- start start 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 Processor -- start 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206be88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206be8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206befdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558206be6da0 con 0x558206bde2f0 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558206afe310 con 0x558206be3e50 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558206be6f20 con 0x558206bdece0 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae2b07a740 10 client.0 _ll_register_callbacks cb 0x558206af22d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae26221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206be88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae26221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206be88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58970/0 (socket says 172.21.15.7:58970) 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.364+0000 7fae26221640 1 -- 172.21.15.7:0/2203636855 learned_addr learned my addr 172.21.15.7:0/2203636855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae25a20640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206be8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26a22640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206befdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26221640 1 -- 172.21.15.7:0/2203636855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 msgr2=0x558206be8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26221640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206be8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26221640 1 -- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 msgr2=0x558206befdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26221640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206befdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26221640 1 -- 172.21.15.7:0/2203636855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582068563f0 con 0x558206bde2f0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae26221640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206be88c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fae10004800 tx=0x7fae100046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e8a617004364b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae05ffb640 1 -- 172.21.15.7:0/2203636855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10086040 con 0x558206bde2f0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae05ffb640 1 -- 172.21.15.7:0/2203636855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae1007e0d0 con 0x558206bde2f0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18051070 con 0x558206bde2f0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae05ffb640 1 -- 172.21.15.7:0/2203636855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae100a4380 con 0x558206bde2f0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae2b07a740 10 client.17298 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582069207d0 con 0x558206bde2f0 2024-03-19T00:46:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.365+0000 7fae05ffb640 1 -- 172.21.15.7:0/2203636855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18051070 con 0x558206bde2f0 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae05ffb640 1 -- 172.21.15.7:0/2203636855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fae1009b030 con 0x558206bde2f0 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae2b07a740 10 client.17298 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae2b07a740 10 client.17298 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae2b07a740 20 client.17298 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae2b07a740 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206bf4600 0x558206bf6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae2b07a740 4 client.17298 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558206ba03e0 con 0x558206bf4600 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae25a20640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206bf4600 0x558206bf6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae25a20640 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206bf4600 0x558206bf6aa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f583cec06aea131e server_cookie=60f2da7acc018299 in_seq=0 out_seq=0 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.366+0000 7fae05ffb640 10 client.17298 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae05ffb640 1 -- 172.21.15.7:0/2203636855 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558206ba03e0 con 0x558206bf4600 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae05ffb640 10 client.17298 handle_command_reply: tid=0 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 client.17298 shutdown 2024-03-19T00:46:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 msgr2=0x558206be88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206be88c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fae10004800 tx=0x7fae100046a0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 shutdown_connections 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206bf4600 0x558206bf6aa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206befdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae05ffb640 0 client.17298 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206be8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 --2- 172.21.15.7:0/2203636855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206be88c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 >> 172.21.15.7:0/2203636855 conn(0x558206bd9860 msgr2=0x558206bd8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 shutdown_connections 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 1 -- 172.21.15.7:0/2203636855 wait complete. 2024-03-19T00:46:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.367+0000 7fae2b07a740 20 client.17298 trim_cache size 0 max 16384 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 Processor -- start 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 -- start start 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206bde2f0 0x558206be8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206befd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 0x558206bd8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae201aca70 con 0x558206be3e50 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae201aa340 con 0x558206bde2f0 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae2b07a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae201a8190 con 0x558206bdece0 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 0x558206bd8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 0x558206bd8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58976/0 (socket says 172.21.15.7:58976) 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 -- 172.21.15.7:0/1050215898 learned_addr learned my addr 172.21.15.7:0/1050215898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26221640 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206bde2f0 0x558206be8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae25a20640 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206befd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 -- 172.21.15.7:0/1050215898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 msgr2=0x558206befd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206befd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 -- 172.21.15.7:0/1050215898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206bde2f0 msgr2=0x558206be8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206bde2f0 0x558206be8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 -- 172.21.15.7:0/1050215898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558206bddf30 con 0x558206be3e50 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae26a22640 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 0x558206bd8310 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fae180349e0 tx=0x7fae1802c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e4c09880b0806d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.381+0000 7fae07fff640 1 -- 172.21.15.7:0/1050215898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1805b070 con 0x558206be3e50 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae07fff640 1 -- 172.21.15.7:0/1050215898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18051040 con 0x558206be3e50 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae07fff640 1 -- 172.21.15.7:0/1050215898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18039040 con 0x558206be3e50 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 -- 172.21.15.7:0/1050215898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 msgr2=0x558206bd8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 0x558206bd8310 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fae180349e0 tx=0x7fae1802c1f0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 -- 172.21.15.7:0/1050215898 shutdown_connections 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206be3e50 0x558206bd8310 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206befd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 --2- 172.21.15.7:0/1050215898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206bde2f0 0x558206be8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 -- 172.21.15.7:0/1050215898 >> 172.21.15.7:0/1050215898 conn(0x558206bd9860 msgr2=0x558206be84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 -- 172.21.15.7:0/1050215898 shutdown_connections 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 -- 172.21.15.7:0/1050215898 wait complete. 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 -- start start 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.382+0000 7fae2b07a740 1 Processor -- start 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206bed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206bedca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206beecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae201a8800 con 0x558206bde2f0 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae201a9c70 con 0x558206be3e50 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae201a8ed0 con 0x558206bdece0 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 10 client.0 _ll_register_callbacks cb 0x7fae10004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206bed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26a22640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206beecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206bed760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58992/0 (socket says 172.21.15.7:58992) 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae25a20640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206bedca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 -- 172.21.15.7:0/3150781717 learned_addr learned my addr 172.21.15.7:0/3150781717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 -- 172.21.15.7:0/3150781717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 msgr2=0x558206bedca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206bedca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 -- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 msgr2=0x558206beecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206beecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 -- 172.21.15.7:0/3150781717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558206baf880 con 0x558206bde2f0 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26a22640 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206beecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae26221640 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206bed760 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fae1009b4d0 tx=0x7fae1009b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4d88f2998112ba server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae05ffb640 1 -- 172.21.15.7:0/3150781717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10086030 con 0x558206bde2f0 2024-03-19T00:46:34.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18051040 con 0x558206bde2f0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.383+0000 7fae05ffb640 1 -- 172.21.15.7:0/3150781717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae100ba2d0 con 0x558206bde2f0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae05ffb640 1 -- 172.21.15.7:0/3150781717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae1009b540 con 0x558206bde2f0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 10 client.17304 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558206c5aac0 con 0x558206bde2f0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae05ffb640 1 -- 172.21.15.7:0/3150781717 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18051040 con 0x558206bde2f0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae05ffb640 1 -- 172.21.15.7:0/3150781717 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fae10083020 con 0x558206bde2f0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 10 client.17304 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 10 client.17304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 20 client.17304 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206c5b4c0 0x558206c5d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 4 client.17304 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558206b4f9f0 con 0x558206c5b4c0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae25a20640 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206c5b4c0 0x558206c5d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae25a20640 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206c5b4c0 0x558206c5d8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a52e799c87acf35d server_cookie=314371f0c0cdf87f in_seq=0 out_seq=0 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.384+0000 7fae05ffb640 10 client.17304 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae05ffb640 1 -- 172.21.15.7:0/3150781717 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558206b4f9f0 con 0x558206c5b4c0 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae05ffb640 10 client.17304 handle_command_reply: tid=0 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 client.17304 shutdown 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 msgr2=0x558206bed760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206bed760 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fae1009b4d0 tx=0x7fae1009b260 comp rx=0 tx=0).stop 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 shutdown_connections 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558206c5b4c0 0x558206c5d8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558206be3e50 0x558206beecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558206bdece0 0x558206bedca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 --2- 172.21.15.7:0/3150781717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558206bde2f0 0x558206bed760 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae05ffb640 0 client.17304 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 >> 172.21.15.7:0/3150781717 conn(0x558206bd9860 msgr2=0x558206be8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 shutdown_connections 2024-03-19T00:46:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 1 -- 172.21.15.7:0/3150781717 wait complete. 2024-03-19T00:46:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae2b07a740 20 client.17304 trim_cache size 0 max 16384 2024-03-19T00:46:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae080a2800 msgr2=0x7fae080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae080a2800 0x7fae080a4cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fae180095f0 tx=0x7fae18048000 comp rx=0 tx=0).stop 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 msgr2=0x7fae20197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 0x7fae20197980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae10005fd0 tx=0x7fae1000ef90 comp rx=0 tx=0).stop 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 shutdown_connections 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae080a2800 0x7fae080a4cc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae201022f0 0x7fae2019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae20101930 0x7fae20197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 --2- 172.21.15.7:0/1070765338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20100740 0x7fae20197980 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 >> 172.21.15.7:0/1070765338 conn(0x7fae200fbc90 msgr2=0x7fae20104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 shutdown_connections 2024-03-19T00:46:34.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.385+0000 7fae28485640 1 -- 172.21.15.7:0/1070765338 wait complete. 2024-03-19T00:46:34.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- 172.21.15.7:0/2726417040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae6410de00 msgr2=0x7fae641101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/2726417040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae6410de00 0x7fae641101f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fae54009cd0 tx=0x7fae5402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- 172.21.15.7:0/2726417040 shutdown_connections 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/2726417040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae6410de00 0x7fae641101f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/2726417040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae64074280 0x7fae64074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/2726417040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae64075860 0x7fae64073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- 172.21.15.7:0/2726417040 >> 172.21.15.7:0/2726417040 conn(0x7fae640fcd60 msgr2=0x7fae640ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- 172.21.15.7:0/2726417040 shutdown_connections 2024-03-19T00:46:34.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- 172.21.15.7:0/2726417040 wait complete. 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 Processor -- start 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- start start 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae64074280 0x7fae6419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 0x7fae6419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae6410de00 0x7fae641a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae64113930 con 0x7fae64075860 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae641137b0 con 0x7fae6410de00 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.501+0000 7fae6bd3b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae64113ab0 con 0x7fae64074280 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 0x7fae6419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 0x7fae6419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42570/0 (socket says 172.21.15.7:42570) 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 -- 172.21.15.7:0/1461175430 learned_addr learned my addr 172.21.15.7:0/1461175430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae69ad7640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae64074280 0x7fae6419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae6410de00 0x7fae641a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 -- 172.21.15.7:0/1461175430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae64074280 msgr2=0x7fae6419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae64074280 0x7fae6419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 -- 172.21.15.7:0/1461175430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae6410de00 msgr2=0x7fae641a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae6410de00 0x7fae641a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 -- 172.21.15.7:0/1461175430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae641a5be0 con 0x7fae64075860 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae692d6640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 0x7fae6419e500 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fae5800ea60 tx=0x7fae5800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae62ffd640 1 -- 172.21.15.7:0/1461175430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5800cde0 con 0x7fae64075860 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae62ffd640 1 -- 172.21.15.7:0/1461175430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae580035f0 con 0x7fae64075860 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae641a5ed0 con 0x7fae64075860 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae62ffd640 1 -- 172.21.15.7:0/1461175430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58010690 con 0x7fae64075860 2024-03-19T00:46:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.502+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a6390 con 0x7fae64075860 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.504+0000 7fae62ffd640 1 -- 172.21.15.7:0/1461175430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fae580037a0 con 0x7fae64075860 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 Processor -- start 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 -- start start 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594a37364b0 con 0x5594a37d22f0 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5594a36ded70 con 0x5594a37d7e50 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6ea03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594a3732550 con 0x5594a37d2ce0 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae62ffd640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae4c0a2800 0x7fae4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6a2d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6a2d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42584/0 (socket says 172.21.15.7:42584) 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6a2d8640 1 -- 172.21.15.7:0/493527943 learned_addr learned my addr 172.21.15.7:0/493527943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae69ad7640 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae4c0a2800 0x7fae4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae692d6640 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.505+0000 7fae62ffd640 1 -- 172.21.15.7:0/1461175430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae580cefd0 con 0x7fae64075860 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 -- 172.21.15.7:0/493527943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 msgr2=0x5594a37d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 -- 172.21.15.7:0/493527943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 msgr2=0x5594a37da2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37da2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 -- 172.21.15.7:0/493527943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594a3701210 con 0x5594a37d22f0 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae69ad7640 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37d2710 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fae54009430 tx=0x7fae540328e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7382dc2b5b1776 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae4c0a2800 0x7fae4c0a4cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fae54032380 tx=0x7fae54042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae4bfff640 1 -- 172.21.15.7:0/493527943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54051070 con 0x5594a37d22f0 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae4bfff640 1 -- 172.21.15.7:0/493527943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae54004d40 con 0x5594a37d22f0 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.506+0000 7fae4bfff640 1 -- 172.21.15.7:0/493527943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54039040 con 0x5594a37d22f0 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 -- 172.21.15.7:0/493527943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 msgr2=0x5594a37d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37d2710 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fae54009430 tx=0x7fae540328e0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 -- 172.21.15.7:0/493527943 shutdown_connections 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37da2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 --2- 172.21.15.7:0/493527943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37d2710 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 -- 172.21.15.7:0/493527943 >> 172.21.15.7:0/493527943 conn(0x5594a37cd860 msgr2=0x5594a37cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 -- 172.21.15.7:0/493527943 shutdown_connections 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 -- 172.21.15.7:0/493527943 wait complete. 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 -- start start 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.507+0000 7fae6ea03740 1 Processor -- start 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594a37dada0 con 0x5594a37d22f0 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5594a36f2310 con 0x5594a37d7e50 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594a37daf20 con 0x5594a37d2ce0 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 10 client.0 _ll_register_callbacks cb 0x5594a36e62d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37dc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42598/0 (socket says 172.21.15.7:42598) 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 -- 172.21.15.7:0/1775354311 learned_addr learned my addr 172.21.15.7:0/1775354311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae692d6640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 -- 172.21.15.7:0/1775354311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 msgr2=0x5594a37dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 -- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 msgr2=0x5594a37e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 -- 172.21.15.7:0/1775354311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594a344a3f0 con 0x5594a37d22f0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae69ad7640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37dc7e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0126e0 tx=0x7fae5c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e0adeecdf5a43b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae497fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae497fa640 1 -- 172.21.15.7:0/1775354311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c012110 con 0x5594a37d22f0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae497fa640 1 -- 172.21.15.7:0/1775354311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae5c004050 con 0x5594a37d22f0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.508+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae54004d40 con 0x5594a37d22f0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae497fa640 1 -- 172.21.15.7:0/1775354311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae5c0122b0 con 0x5594a37d22f0 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 10 client.17316 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594a35147d0 con 0x5594a37d22f0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae497fa640 1 -- 172.21.15.7:0/1775354311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae54004d40 con 0x5594a37d22f0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae497fa640 1 -- 172.21.15.7:0/1775354311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fae5c0038a0 con 0x5594a37d22f0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 10 client.17316 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 20 client.17316 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a37e86b0 0x5594a37eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 4 client.17316 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594a37943e0 con 0x5594a37e86b0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.509+0000 7fae692d6640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a37e86b0 0x5594a37eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae497fa640 10 client.17316 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae692d6640 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a37e86b0 0x5594a37eab50 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a5176cdeb8f511 server_cookie=11aa15c1a44ca234 in_seq=0 out_seq=0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae497fa640 1 -- 172.21.15.7:0/1775354311 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5594a37943e0 con 0x5594a37e86b0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae497fa640 10 client.17316 handle_command_reply: tid=0 2024-03-19T00:46:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 client.17316 shutdown 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 msgr2=0x5594a37dc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37dc7e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0126e0 tx=0x7fae5c012ba0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 shutdown_connections 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d2ce0 0x5594a37dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a37e86b0 0x5594a37eab50 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1775354311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d22f0 0x5594a37dc7e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 >> 172.21.15.7:0/1775354311 conn(0x5594a37cd860 msgr2=0x5594a37ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae497fa640 0 client.17316 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 shutdown_connections 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.510+0000 7fae6ea03740 1 -- 172.21.15.7:0/1775354311 wait complete. 2024-03-19T00:46:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.511+0000 7fae6ea03740 20 client.17316 trim_cache size 0 max 16384 2024-03-19T00:46:34.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 Processor -- start 2024-03-19T00:46:34.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 -- start start 2024-03-19T00:46:34.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d22f0 0x5594a37d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b31f0 con 0x5594a37d2ce0 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b0ac0 con 0x5594a37d7e50 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6ea03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae641ae910 con 0x5594a37d22f0 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42604/0 (socket says 172.21.15.7:42604) 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 -- 172.21.15.7:0/1374098101 learned_addr learned my addr 172.21.15.7:0/1374098101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae69ad7640 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d22f0 0x5594a37d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 -- 172.21.15.7:0/1374098101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d22f0 msgr2=0x5594a37d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d22f0 0x5594a37d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 -- 172.21.15.7:0/1374098101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 msgr2=0x5594a37dbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37dbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae692d6640 1 -- 172.21.15.7:0/1374098101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594a37a3880 con 0x5594a37d2ce0 2024-03-19T00:46:34.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.524+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37dbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae692d6640 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e3d20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fae58097a70 tx=0x7fae58007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bcc9f20beca5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae4b7fe640 1 -- 172.21.15.7:0/1374098101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58003b30 con 0x5594a37d2ce0 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae4b7fe640 1 -- 172.21.15.7:0/1374098101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae58097040 con 0x5594a37d2ce0 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae4b7fe640 1 -- 172.21.15.7:0/1374098101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58099910 con 0x5594a37d2ce0 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 -- 172.21.15.7:0/1374098101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 msgr2=0x5594a37e3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e3d20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fae58097a70 tx=0x7fae58007b90 comp rx=0 tx=0).stop 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 -- 172.21.15.7:0/1374098101 shutdown_connections 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d7e50 0x5594a37dbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e3d20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 --2- 172.21.15.7:0/1374098101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d22f0 0x5594a37d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 -- 172.21.15.7:0/1374098101 >> 172.21.15.7:0/1374098101 conn(0x5594a37cd860 msgr2=0x5594a37d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 -- 172.21.15.7:0/1374098101 shutdown_connections 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.525+0000 7fae6ea03740 1 -- 172.21.15.7:0/1374098101 wait complete. 2024-03-19T00:46:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 -- start start 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 Processor -- start 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d22f0 0x5594a37e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d7e50 0x5594a37e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae641aef80 con 0x5594a37d2ce0 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b03f0 con 0x5594a37d22f0 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae641af650 con 0x5594a37d7e50 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6ea03740 10 client.0 _ll_register_callbacks cb 0x7fae5c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae69ad7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d22f0 0x5594a37e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42610/0 (socket says 172.21.15.7:42610) 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 -- 172.21.15.7:0/2291090924 learned_addr learned my addr 172.21.15.7:0/2291090924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae6a2d8640 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d7e50 0x5594a37e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 -- 172.21.15.7:0/2291090924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d7e50 msgr2=0x5594a37e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d7e50 0x5594a37e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 -- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d22f0 msgr2=0x5594a37e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d22f0 0x5594a37e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.526+0000 7fae692d6640 1 -- 172.21.15.7:0/2291090924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594a37e8490 con 0x5594a37d2ce0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae692d6640 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e1fe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fae580ccae0 tx=0x7fae58003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb1203c6cb60efd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae497fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae497fa640 1 -- 172.21.15.7:0/2291090924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5809bd10 con 0x5594a37d2ce0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae58097040 con 0x5594a37d2ce0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae497fa640 1 -- 172.21.15.7:0/2291090924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae58099910 con 0x5594a37d2ce0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae497fa640 1 -- 172.21.15.7:0/2291090924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae58097c80 con 0x5594a37d2ce0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 10 client.17322 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594a384ee10 con 0x5594a37d2ce0 2024-03-19T00:46:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae497fa640 1 -- 172.21.15.7:0/2291090924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae58097040 con 0x5594a37d2ce0 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae497fa640 1 -- 172.21.15.7:0/2291090924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fae5809b870 con 0x5594a37d2ce0 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 10 client.17322 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 20 client.17322 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a384f700 0x5594a3851b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 4 client.17322 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594a37439f0 con 0x5594a384f700 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.527+0000 7fae69ad7640 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a384f700 0x5594a3851b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae69ad7640 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a384f700 0x5594a3851b20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ac14ea3e72611e server_cookie=b932b50afb4fc156 in_seq=0 out_seq=0 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae497fa640 10 client.17322 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae497fa640 1 -- 172.21.15.7:0/2291090924 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594a37439f0 con 0x5594a384f700 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae497fa640 10 client.17322 handle_command_reply: tid=0 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 client.17322 shutdown 2024-03-19T00:46:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 msgr2=0x5594a37e1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e1fe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fae580ccae0 tx=0x7fae58003d20 comp rx=0 tx=0).stop 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 shutdown_connections 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594a37d7e50 0x5594a37e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5594a384f700 0x5594a3851b20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594a37d2ce0 0x5594a37e1fe0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae497fa640 0 client.17322 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 --2- 172.21.15.7:0/2291090924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a37d22f0 0x5594a37e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 >> 172.21.15.7:0/2291090924 conn(0x5594a37cd860 msgr2=0x5594a37d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 shutdown_connections 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 1 -- 172.21.15.7:0/2291090924 wait complete. 2024-03-19T00:46:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.528+0000 7fae6ea03740 20 client.17322 trim_cache size 0 max 16384 2024-03-19T00:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:46:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae4c0a2800 msgr2=0x7fae4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae4c0a2800 0x7fae4c0a4cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fae54032380 tx=0x7fae54042000 comp rx=0 tx=0).stop 2024-03-19T00:46:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 msgr2=0x7fae6419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 0x7fae6419e500 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fae5800ea60 tx=0x7fae5800ef20 comp rx=0 tx=0).stop 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 shutdown_connections 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fae4c0a2800 0x7fae4c0a4cc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae6410de00 0x7fae641a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae64075860 0x7fae6419e500 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 --2- 172.21.15.7:0/1461175430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae64074280 0x7fae6419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 >> 172.21.15.7:0/1461175430 conn(0x7fae640fcd60 msgr2=0x7fae641011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 shutdown_connections 2024-03-19T00:46:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:34.529+0000 7fae6bd3b640 1 -- 172.21.15.7:0/1461175430 wait complete. 2024-03-19T00:46:34.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:34 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1230157727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:46:35.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:34 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1230157727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:46:35.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:34 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1230157727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:46:35.540 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:46:35.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:46:35.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 -- 172.21.15.7:0/1761477518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 msgr2=0x7fe300107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 --2- 172.21.15.7:0/1761477518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300107080 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009900 tx=0x7fe2e802bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 -- 172.21.15.7:0/1761477518 shutdown_connections 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 --2- 172.21.15.7:0/1761477518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3001075c0 0x7fe3001099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 --2- 172.21.15.7:0/1761477518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300107080 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 --2- 172.21.15.7:0/1761477518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe300102360 0x7fe300104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 -- 172.21.15.7:0/1761477518 >> 172.21.15.7:0/1761477518 conn(0x7fe3000fbcd0 msgr2=0x7fe3000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 -- 172.21.15.7:0/1761477518 shutdown_connections 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 -- 172.21.15.7:0/1761477518 wait complete. 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 Processor -- start 2024-03-19T00:46:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.659+0000 7fe305a4b640 1 -- start start 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe300102360 0x7fe300101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3001075c0 0x7fe300100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe30010d340 con 0x7fe300104c90 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe30010d1c0 con 0x7fe3001075c0 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe30010d4c0 con 0x7fe300102360 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42624/0 (socket says 172.21.15.7:42624) 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 -- 172.21.15.7:0/2038440731 learned_addr learned my addr 172.21.15.7:0/2038440731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe300102360 0x7fe300101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3001075c0 0x7fe300100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 -- 172.21.15.7:0/2038440731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe300102360 msgr2=0x7fe300101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe300102360 0x7fe300101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 -- 172.21.15.7:0/2038440731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3001075c0 msgr2=0x7fe300100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3001075c0 0x7fe300100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 -- 172.21.15.7:0/2038440731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3001010e0 con 0x7fe300104c90 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300100310 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8000c00 tx=0x7fe2e8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe304a49640 1 -- 172.21.15.7:0/2038440731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8043020 con 0x7fe300104c90 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe304a49640 1 -- 172.21.15.7:0/2038440731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2e8004350 con 0x7fe300104c90 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe304a49640 1 -- 172.21.15.7:0/2038440731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8003b80 con 0x7fe300104c90 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.660+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3001a1b90 con 0x7fe300104c90 2024-03-19T00:46:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.661+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3001a2130 con 0x7fe300104c90 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe304a49640 1 -- 172.21.15.7:0/2038440731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe2e8046020 con 0x7fe300104c90 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 Processor -- start 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 -- start start 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe304a49640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe2ec0a2880 0x7fe2ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f6b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f6e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d900ecb160 con 0x55d900f6bf00 2024-03-19T00:46:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d900e73e10 con 0x55d900f663a0 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe30863c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d900ec7200 con 0x55d900f66d90 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.663+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe2ec0a2880 0x7fe2ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe304a49640 1 -- 172.21.15.7:0/2038440731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe2e80f0ee0 con 0x7fe300104c90 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f6e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f6e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42626/0 (socket says 172.21.15.7:42626) 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33118/0 (socket says 172.21.15.7:33118) 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2639200927 learned_addr learned my addr 172.21.15.7:0/2639200927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f6b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 -- 172.21.15.7:0/2639200927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 msgr2=0x55d900f6b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f6b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 -- 172.21.15.7:0/2639200927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 msgr2=0x55d900f6e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f6e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 -- 172.21.15.7:0/2639200927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d900ca77e0 con 0x55d900f663a0 2024-03-19T00:46:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f6b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f6e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe2ec0a2880 0x7fe2ec0a4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0011d00 tx=0x7fe2f0011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f667c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009c20 tx=0x7fe2f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85771f1e095dad1e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2f5ffb640 1 -- 172.21.15.7:0/2639200927 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8004050 con 0x55d900f663a0 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2f5ffb640 1 -- 172.21.15.7:0/2639200927 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2f80054d0 con 0x55d900f663a0 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.664+0000 7fe2f5ffb640 1 -- 172.21.15.7:0/2639200927 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8004c30 con 0x55d900f663a0 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 -- 172.21.15.7:0/2639200927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 msgr2=0x55d900f667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f667c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009c20 tx=0x7fe2f8011810 comp rx=0 tx=0).stop 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 -- 172.21.15.7:0/2639200927 shutdown_connections 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f6e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f6b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 --2- 172.21.15.7:0/2639200927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f667c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 -- 172.21.15.7:0/2639200927 >> 172.21.15.7:0/2639200927 conn(0x55d900f61910 msgr2=0x55d900f61d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 -- 172.21.15.7:0/2639200927 shutdown_connections 2024-03-19T00:46:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 -- 172.21.15.7:0/2639200927 wait complete. 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.665+0000 7fe30863c740 1 -- start start 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 Processor -- start 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f663a0 0x55d900f70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f77ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d900f6eeb0 con 0x55d900f6bf00 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d900e871a0 con 0x55d900f66d90 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d900f6f030 con 0x55d900f663a0 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe30863c740 10 client.0 _ll_register_callbacks cb 0x55d900bdec60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f77ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f77ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42640/0 (socket says 172.21.15.7:42640) 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2fe7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f663a0 0x55d900f70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2635418017 learned_addr learned my addr 172.21.15.7:0/2635418017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2635418017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f663a0 msgr2=0x55d900f70760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f663a0 0x55d900f70760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 msgr2=0x55d900f70ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f70ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.666+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2635418017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d900d07f70 con 0x55d900f6bf00 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f70ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f77ce0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe2f001e900 tx=0x7fe2f0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2afd54b0112d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2deffd640 1 -- 172.21.15.7:0/2635418017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0011d30 con 0x55d900f6bf00 2024-03-19T00:46:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f80054d0 con 0x55d900f6bf00 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2deffd640 1 -- 172.21.15.7:0/2635418017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe2f0003970 con 0x55d900f6bf00 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2deffd640 1 -- 172.21.15.7:0/2635418017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2f0032050 con 0x55d900f6bf00 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe30863c740 10 client.17334 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d900b9f170 con 0x55d900f6bf00 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2deffd640 1 -- 172.21.15.7:0/2635418017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2f80054d0 con 0x55d900f6bf00 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.667+0000 7fe2deffd640 1 -- 172.21.15.7:0/2635418017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe2f0004020 con 0x55d900f6bf00 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe30863c740 10 client.17334 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe30863c740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe30863c740 20 client.17334 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe30863c740 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900f7c5a0 0x55d900f7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe30863c740 4 client.17334 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d900ed86a0 con 0x55d900f7c5a0 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900f7c5a0 0x55d900f7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe2deffd640 10 client.17334 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.668+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900f7c5a0 0x55d900f7ea00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77950dacd7173dea server_cookie=11b789ba0a885197 in_seq=0 out_seq=0 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe2deffd640 1 -- 172.21.15.7:0/2635418017 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d900ed86a0 con 0x55d900f7c5a0 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe2deffd640 10 client.17334 handle_command_reply: tid=0 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 client.17334 shutdown 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 msgr2=0x55d900f77ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f77ce0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe2f001e900 tx=0x7fe2f0009500 comp rx=0 tx=0).stop 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 shutdown_connections 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900f7c5a0 0x55d900f7ea00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f77ce0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f70ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe2deffd640 0 client.17334 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 --2- 172.21.15.7:0/2635418017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f663a0 0x55d900f70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 >> 172.21.15.7:0/2635418017 conn(0x55d900f61910 msgr2=0x55d900f60b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 shutdown_connections 2024-03-19T00:46:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 1 -- 172.21.15.7:0/2635418017 wait complete. 2024-03-19T00:46:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.669+0000 7fe30863c740 20 client.17334 trim_cache size 0 max 16384 2024-03-19T00:46:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 Processor -- start 2024-03-19T00:46:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 -- start start 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f65280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f77ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f83300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001aece0 con 0x55d900f6bf00 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001ac5b0 con 0x55d900f663a0 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe30863c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001aa400 con 0x55d900f66d90 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe2ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f83300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe2ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f83300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42656/0 (socket says 172.21.15.7:42656) 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2883361679 learned_addr learned my addr 172.21.15.7:0/2883361679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe2fe7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f77ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.689+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f65280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2883361679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 msgr2=0x55d900f77ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f77ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2883361679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 msgr2=0x55d900f65280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f65280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2ff7fe640 1 -- 172.21.15.7:0/2883361679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d900f67170 con 0x55d900f6bf00 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f83300 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000fcc0 tx=0x7fe2f00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadb9150ded847de server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2f57fa640 1 -- 172.21.15.7:0/2883361679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f003b070 con 0x55d900f6bf00 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2f57fa640 1 -- 172.21.15.7:0/2883361679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2f002e070 con 0x55d900f6bf00 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.690+0000 7fe2f57fa640 1 -- 172.21.15.7:0/2883361679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00207c0 con 0x55d900f6bf00 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 -- 172.21.15.7:0/2883361679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 msgr2=0x55d900f83300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f83300 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000fcc0 tx=0x7fe2f00049e0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 -- 172.21.15.7:0/2883361679 shutdown_connections 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f6bf00 0x55d900f83300 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f66d90 0x55d900f77ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- 172.21.15.7:0/2883361679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f663a0 0x55d900f65280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 -- 172.21.15.7:0/2883361679 >> 172.21.15.7:0/2883361679 conn(0x55d900f61910 msgr2=0x55d900f64b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 -- 172.21.15.7:0/2883361679 shutdown_connections 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 -- 172.21.15.7:0/2883361679 wait complete. 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 -- start start 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 Processor -- start 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 0x55d900f77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.691+0000 7fe30863c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f6bf00 0x55d900f75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe30863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001aaa70 con 0x55d900f663a0 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe30863c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001abee0 con 0x55d900f66d90 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe30863c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001ab140 con 0x55d900f6bf00 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe30863c740 10 client.0 _ll_register_callbacks cb 0x7fe2f001e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 0x55d900f77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 0x55d900f77710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42660/0 (socket says 172.21.15.7:42660) 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 -- 172.21.15.7:0/2574384082 learned_addr learned my addr 172.21.15.7:0/2574384082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2ff7fe640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f6bf00 0x55d900f75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 -- 172.21.15.7:0/2574384082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f6bf00 msgr2=0x55d900f75e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f6bf00 0x55d900f75e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 -- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 msgr2=0x55d900f77c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f77c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 -- 172.21.15.7:0/2574384082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d900f76660 con 0x55d900f663a0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2feffd640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 0x55d900f77710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009c20 tx=0x7fe2f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed57a3b8f420c411 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2deffd640 1 -- 172.21.15.7:0/2574384082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8032070 con 0x55d900f663a0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2deffd640 1 -- 172.21.15.7:0/2574384082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe2f802eab0 con 0x55d900f663a0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f002e070 con 0x55d900f663a0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.692+0000 7fe2deffd640 1 -- 172.21.15.7:0/2574384082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2f803e050 con 0x55d900f663a0 2024-03-19T00:46:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 10 client.17340 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d900fe35b0 con 0x55d900f663a0 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe2deffd640 1 -- 172.21.15.7:0/2574384082 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2f002e070 con 0x55d900f663a0 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe2deffd640 1 -- 172.21.15.7:0/2574384082 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe2f8005270 con 0x55d900f663a0 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 10 client.17340 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 20 client.17340 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900fe3dd0 0x55d900fe61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 4 client.17340 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d900f70550 con 0x55d900fe3dd0 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900fe3dd0 0x55d900fe61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe2deffd640 10 client.17340 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.693+0000 7fe2fe7fc640 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900fe3dd0 0x55d900fe61f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa232a10ccb4aea1 server_cookie=46d1a0b3bdd59da0 in_seq=0 out_seq=0 2024-03-19T00:46:35.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.694+0000 7fe2deffd640 1 -- 172.21.15.7:0/2574384082 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d900f70550 con 0x55d900fe3dd0 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.694+0000 7fe2deffd640 10 client.17340 handle_command_reply: tid=0 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.694+0000 7fe30863c740 1 client.17340 shutdown 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.694+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 msgr2=0x55d900f77710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.694+0000 7fe30863c740 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 0x55d900f77710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009c20 tx=0x7fe2f8009e30 comp rx=0 tx=0).stop 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.694+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 shutdown_connections 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d900f6bf00 0x55d900f75e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d900f66d90 0x55d900f77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55d900fe3dd0 0x55d900fe61f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 --2- 172.21.15.7:0/2574384082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d900f663a0 0x55d900f77710 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe2deffd640 0 client.17340 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 >> 172.21.15.7:0/2574384082 conn(0x55d900f61910 msgr2=0x55d900f65400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 shutdown_connections 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 1 -- 172.21.15.7:0/2574384082 wait complete. 2024-03-19T00:46:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe30863c740 20 client.17340 trim_cache size 0 max 16384 2024-03-19T00:46:35.702 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:46:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:46:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "6e95c388-d7d1-4fc2-846c-5e84ff99bbda", 2024-03-19T00:46:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:46:35.702 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe2ec0a2880 msgr2=0x7fe2ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe2ec0a2880 0x7fe2ec0a4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0011d00 tx=0x7fe2f0011440 comp rx=0 tx=0).stop 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 msgr2=0x7fe300100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300100310 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8000c00 tx=0x7fe2e8005dc0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 shutdown_connections 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe2ec0a2880 0x7fe2ec0a4d40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3001075c0 0x7fe300100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe300104c90 0x7fe300100310 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 --2- 172.21.15.7:0/2038440731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe300102360 0x7fe300101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 >> 172.21.15.7:0/2038440731 conn(0x7fe3000fbcd0 msgr2=0x7fe300102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.695+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 shutdown_connections 2024-03-19T00:46:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.696+0000 7fe305a4b640 1 -- 172.21.15.7:0/2038440731 wait complete. 2024-03-19T00:46:35.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:46:35.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- 172.21.15.7:0/1922702188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 msgr2=0x7f02e8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- 172.21.15.7:0/1922702188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8102ea0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009900 tx=0x7f02dc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- 172.21.15.7:0/1922702188 shutdown_connections 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- 172.21.15.7:0/1922702188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02e81033e0 0x7f02e8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- 172.21.15.7:0/1922702188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8102ea0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- 172.21.15.7:0/1922702188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- 172.21.15.7:0/1922702188 >> 172.21.15.7:0/1922702188 conn(0x7f02e80fcda0 msgr2=0x7f02e80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- 172.21.15.7:0/1922702188 shutdown_connections 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- 172.21.15.7:0/1922702188 wait complete. 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 Processor -- start 2024-03-19T00:46:35.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- start start 2024-03-19T00:46:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02e81033e0 0x7f02e819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.812+0000 7f02ed275640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d360 con 0x7f02e8102a20 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02ed275640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d1e0 con 0x7f02e8101830 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02ed275640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d4e0 con 0x7f02e81033e0 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42676/0 (socket says 172.21.15.7:42676) 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 -- 172.21.15.7:0/2060721223 learned_addr learned my addr 172.21.15.7:0/2060721223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02e81033e0 0x7f02e819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 -- 172.21.15.7:0/2060721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02e81033e0 msgr2=0x7f02e819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02e81033e0 0x7f02e819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 -- 172.21.15.7:0/2060721223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 msgr2=0x7f02e8197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 -- 172.21.15.7:0/2060721223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e819f660 con 0x7f02e8102a20 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8197f80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f02dc006010 tx=0x7f02dc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02cffff640 1 -- 172.21.15.7:0/2060721223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc043020 con 0x7f02e8102a20 2024-03-19T00:46:35.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02cffff640 1 -- 172.21.15.7:0/2060721223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02dc0043d0 con 0x7f02e8102a20 2024-03-19T00:46:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e819f8f0 con 0x7f02e8102a20 2024-03-19T00:46:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02cffff640 1 -- 172.21.15.7:0/2060721223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc003b80 con 0x7f02e8102a20 2024-03-19T00:46:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.813+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e819fe40 con 0x7f02e8102a20 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02cffff640 1 -- 172.21.15.7:0/2060721223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f02dc046020 con 0x7f02e8102a20 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 Processor -- start 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 -- start start 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d21e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d22440 0x560844d26f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 0x560844d29a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560844c86500 con 0x560844d275b0 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560844c2ebe0 con 0x560844d21a50 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02efe5a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560844c825a0 con 0x560844d22440 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 0x560844d29a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02cffff640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02d00a2800 0x7f02d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 0x560844d29a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42692/0 (socket says 172.21.15.7:42692) 2024-03-19T00:46:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02e67fc640 1 -- 172.21.15.7:0/2134099954 learned_addr learned my addr 172.21.15.7:0/2134099954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.816+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d22440 0x560844d26f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d21e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02cffff640 1 -- 172.21.15.7:0/2060721223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f02dc0f0ff0 con 0x7f02e8102a20 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02d00a2800 0x7f02d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e67fc640 1 -- 172.21.15.7:0/2134099954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d22440 msgr2=0x560844d26f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d22440 0x560844d26f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e67fc640 1 -- 172.21.15.7:0/2134099954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 msgr2=0x560844d21e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d21e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e67fc640 1 -- 172.21.15.7:0/2134099954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560844c50000 con 0x560844d275b0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d21e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d22440 0x560844d26f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 0x560844d29a40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f02dc000c00 tx=0x7f02dc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df7f3eb0b7e64f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02cd7fa640 1 -- 172.21.15.7:0/2134099954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0bd5f0 con 0x560844d275b0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02d00a2800 0x7f02d00a4cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f02e0005c40 tx=0x7f02e0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02cd7fa640 1 -- 172.21.15.7:0/2134099954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02dc0bdcf0 con 0x560844d275b0 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.817+0000 7f02cd7fa640 1 -- 172.21.15.7:0/2134099954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc003d20 con 0x560844d275b0 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- 172.21.15.7:0/2134099954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 msgr2=0x560844d29a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 0x560844d29a40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f02dc000c00 tx=0x7f02dc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- 172.21.15.7:0/2134099954 shutdown_connections 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d275b0 0x560844d29a40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d22440 0x560844d26f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2134099954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d21e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- 172.21.15.7:0/2134099954 >> 172.21.15.7:0/2134099954 conn(0x560844d1cfc0 msgr2=0x560844d1d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- 172.21.15.7:0/2134099954 shutdown_connections 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- 172.21.15.7:0/2134099954 wait complete. 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- start start 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 Processor -- start 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560844d2a320 con 0x560844d22440 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.818+0000 7f02efe5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560844c42160 con 0x560844d21a50 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02efe5a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560844d2a4a0 con 0x560844d275b0 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02efe5a740 10 client.0 _ll_register_callbacks cb 0x560844c7ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d2c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42708/0 (socket says 172.21.15.7:42708) 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 -- 172.21.15.7:0/57803928 learned_addr learned my addr 172.21.15.7:0/57803928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e77fe640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 -- 172.21.15.7:0/57803928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 msgr2=0x560844d331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 -- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 msgr2=0x560844d2bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d2bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 -- 172.21.15.7:0/57803928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560844a647d0 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02e67fc640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d2c1f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0b9fc0 tx=0x7f02dc0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd85608d48f9c51e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02caff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02caff5640 1 -- 172.21.15.7:0/57803928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0d6060 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02caff5640 1 -- 172.21.15.7:0/57803928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f02dc0c5900 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02dc0bdcf0 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02caff5640 1 -- 172.21.15.7:0/57803928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02dc0c5d90 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02efe5a740 10 client.17352 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.819+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56084499a3f0 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02caff5640 1 -- 172.21.15.7:0/57803928 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02dc0bdcf0 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02caff5640 1 -- 172.21.15.7:0/57803928 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f02dc0a6bf0 con 0x560844d22440 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02efe5a740 10 client.17352 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02efe5a740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02efe5a740 20 client.17352 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02efe5a740 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d37880 0x560844d39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02efe5a740 4 client.17352 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560844c93a40 con 0x560844d37880 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d37880 0x560844d39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d37880 0x560844d39ce0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11912a5adca322fa server_cookie=86ec83b307c500af in_seq=0 out_seq=0 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.820+0000 7f02caff5640 10 client.17352 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02caff5640 1 -- 172.21.15.7:0/57803928 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560844c93a40 con 0x560844d37880 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02caff5640 10 client.17352 handle_command_reply: tid=0 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 client.17352 shutdown 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 msgr2=0x560844d2c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d2c1f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0b9fc0 tx=0x7f02dc0056e0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 shutdown_connections 2024-03-19T00:46:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d37880 0x560844d39ce0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d2c1f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 --2- 172.21.15.7:0/57803928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d21a50 0x560844d2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02caff5640 0 client.17352 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 >> 172.21.15.7:0/57803928 conn(0x560844d1cfc0 msgr2=0x560844d1c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 shutdown_connections 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 1 -- 172.21.15.7:0/57803928 wait complete. 2024-03-19T00:46:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.821+0000 7f02efe5a740 20 client.17352 trim_cache size 0 max 16384 2024-03-19T00:46:35.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.838+0000 7f02efe5a740 1 Processor -- start 2024-03-19T00:46:35.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.838+0000 7f02efe5a740 1 -- start start 2024-03-19T00:46:35.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d21a50 0x560844d2aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d33180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d275b0 0x560844d1c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02efe5a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81acae0 con 0x560844d22440 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02efe5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81aa3b0 con 0x560844d275b0 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02efe5a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a8200 con 0x560844d21a50 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d33180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d33180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42716/0 (socket says 172.21.15.7:42716) 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 -- 172.21.15.7:0/2280850965 learned_addr learned my addr 172.21.15.7:0/2280850965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d21a50 0x560844d2aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d275b0 0x560844d1c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 -- 172.21.15.7:0/2280850965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d21a50 msgr2=0x560844d2aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d21a50 0x560844d2aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 -- 172.21.15.7:0/2280850965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d275b0 msgr2=0x560844d1c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d275b0 0x560844d1c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 -- 172.21.15.7:0/2280850965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084495a0f0 con 0x560844d22440 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d21a50 0x560844d2aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d33180 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0bd2f0 tx=0x7f02dc0bd5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de603d6f07454553 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02ccff9640 1 -- 172.21.15.7:0/2280850965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0d6040 con 0x560844d22440 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02ccff9640 1 -- 172.21.15.7:0/2280850965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02dc0bdcf0 con 0x560844d22440 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.839+0000 7f02ccff9640 1 -- 172.21.15.7:0/2280850965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0c5900 con 0x560844d22440 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- 172.21.15.7:0/2280850965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 msgr2=0x560844d33180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d33180 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0bd2f0 tx=0x7f02dc0bd5f0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- 172.21.15.7:0/2280850965 shutdown_connections 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d275b0 0x560844d1c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d22440 0x560844d33180 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2280850965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d21a50 0x560844d2aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- 172.21.15.7:0/2280850965 >> 172.21.15.7:0/2280850965 conn(0x560844d1cfc0 msgr2=0x560844d1b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- 172.21.15.7:0/2280850965 shutdown_connections 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- 172.21.15.7:0/2280850965 wait complete. 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- start start 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 Processor -- start 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 0x560844d30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d22440 0x560844d31360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d32320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a8870 con 0x560844d21a50 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a9ce0 con 0x560844d22440 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81a8f40 con 0x560844d275b0 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.840+0000 7f02efe5a740 10 client.0 _ll_register_callbacks cb 0x7f02d8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 0x560844d30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 0x560844d30e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42730/0 (socket says 172.21.15.7:42730) 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 -- 172.21.15.7:0/2095984655 learned_addr learned my addr 172.21.15.7:0/2095984655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:46:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d22440 0x560844d31360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e77fe640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d32320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 -- 172.21.15.7:0/2095984655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 msgr2=0x560844d32320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d32320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 -- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d22440 msgr2=0x560844d31360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d22440 0x560844d31360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 -- 172.21.15.7:0/2095984655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560844d32910 con 0x560844d21a50 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02e6ffd640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 0x560844d30e20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009fd0 tx=0x7f02d8012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b78b48fb54c4ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02caff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02caff5640 1 -- 172.21.15.7:0/2095984655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d80200f0 con 0x560844d21a50 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02caff5640 1 -- 172.21.15.7:0/2095984655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f02d8020290 con 0x560844d21a50 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02dc0bdcf0 con 0x560844d21a50 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02caff5640 1 -- 172.21.15.7:0/2095984655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02d802f050 con 0x560844d21a50 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02efe5a740 10 client.17358 fetch_fsmap learned FSMap version 27 2024-03-19T00:46:35.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560844d9e120 con 0x560844d21a50 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.841+0000 7f02caff5640 1 -- 172.21.15.7:0/2095984655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02dc0bdcf0 con 0x560844d21a50 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02caff5640 1 -- 172.21.15.7:0/2095984655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f02d8010430 con 0x560844d21a50 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 10 client.17358 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 20 client.17358 populate_metadata read hostname 'smithi007' 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d9eac0 0x560844da0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 4 client.17358 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560844c8dd90 con 0x560844d9eac0 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d9eac0 0x560844da0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02e67fc640 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d9eac0 0x560844da0ee0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef12e0bbc498ffd server_cookie=c462c2ccce3e4e8b in_seq=0 out_seq=0 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02caff5640 10 client.17358 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02caff5640 1 -- 172.21.15.7:0/2095984655 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560844c8dd90 con 0x560844d9eac0 2024-03-19T00:46:35.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02caff5640 10 client.17358 handle_command_reply: tid=0 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.842+0000 7f02efe5a740 1 client.17358 shutdown 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 msgr2=0x560844d30e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 0x560844d30e20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009fd0 tx=0x7f02d8012ae0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 shutdown_connections 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560844d275b0 0x560844d32320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560844d22440 0x560844d31360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560844d9eac0 0x560844da0ee0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 --2- 172.21.15.7:0/2095984655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560844d21a50 0x560844d30e20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 >> 172.21.15.7:0/2095984655 conn(0x560844d1cfc0 msgr2=0x560844d20ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02caff5640 0 client.17358 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 shutdown_connections 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 1 -- 172.21.15.7:0/2095984655 wait complete. 2024-03-19T00:46:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02efe5a740 20 client.17358 trim_cache size 0 max 16384 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "6e95c388-d7d1-4fc2-846c-5e84ff99bbda": { 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "6e95c388-d7d1-4fc2-846c-5e84ff99bbda", 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:46:35.852 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02d00a2800 msgr2=0x7f02d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02d00a2800 0x7f02d00a4cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f02e0005c40 tx=0x7f02e0009500 comp rx=0 tx=0).stop 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 msgr2=0x7f02e8197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8197f80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f02dc006010 tx=0x7f02dc005e40 comp rx=0 tx=0).stop 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 shutdown_connections 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f02d00a2800 0x7f02d00a4cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02e81033e0 0x7f02e819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8102a20 0x7f02e8197f80 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 --2- 172.21.15.7:0/2060721223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:46:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 >> 172.21.15.7:0/2060721223 conn(0x7f02e80fcda0 msgr2=0x7f02e8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:46:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 shutdown_connections 2024-03-19T00:46:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:46:35.843+0000 7f02ed275640 1 -- 172.21.15.7:0/2060721223 wait complete. 2024-03-19T00:46:35.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:35 smithi007 ceph-mon[62854]: pgmap v610: 129 pgs: 129 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:35.855 INFO:tasks.cephfs.filesystem:scrub status for tag:6e95c388-d7d1-4fc2-846c-5e84ff99bbda - {'path': '/', 'tag': '6e95c388-d7d1-4fc2-846c-5e84ff99bbda', 'options': 'recursive,force'} 2024-03-19T00:46:36.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:35 smithi204 ceph-mon[66572]: pgmap v610: 129 pgs: 129 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:35 smithi044 ceph-mon[66366]: pgmap v610: 129 pgs: 129 active+clean; 2.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:36.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:36 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:46:36.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:36 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:36.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:36 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:46:36.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:36 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:46:36.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:36 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:36.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:36 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:36 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:46:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:36 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:36 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:46:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:36 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:46:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:36 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:36 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:36 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:36 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:36 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:36 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:36 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:36 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:46:37.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:37 smithi007 ceph-mon[62854]: pgmap v611: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:38.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:37 smithi204 ceph-mon[66572]: pgmap v611: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:37 smithi044 ceph-mon[66366]: pgmap v611: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:46:39.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:39 smithi007 ceph-mon[62854]: pgmap v612: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:40.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:39 smithi204 ceph-mon[66572]: pgmap v612: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:40.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:39 smithi044 ceph-mon[66366]: pgmap v612: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:41 smithi007 ceph-mon[62854]: pgmap v613: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:41 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:46:41.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:41 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:46:42.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:41 smithi204 ceph-mon[66572]: pgmap v613: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:42.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:41 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:46:42.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:41 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:46:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:41 smithi044 ceph-mon[66366]: pgmap v613: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:46:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:41 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:46:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:41 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:46:43.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:43 smithi007 ceph-mon[62854]: pgmap v614: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:46:44.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:43 smithi204 ceph-mon[66572]: pgmap v614: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:46:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:43 smithi044 ceph-mon[66366]: pgmap v614: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:46:45.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:45 smithi007 ceph-mon[62854]: pgmap v615: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:46:46.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:45 smithi204 ceph-mon[66572]: pgmap v615: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:46:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:45 smithi044 ceph-mon[66366]: pgmap v615: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:46:47.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:47 smithi007 ceph-mon[62854]: pgmap v616: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:46:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:47 smithi204 ceph-mon[66572]: pgmap v616: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:46:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:47 smithi044 ceph-mon[66366]: pgmap v616: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:46:48.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:46:48.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:46:48.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:46:48.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:46:49.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:46:49.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:46:49.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:46:49.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:46:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:46:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:46:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:46:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:46:49.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:49 smithi007 ceph-mon[62854]: pgmap v617: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:46:50.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:49 smithi204 ceph-mon[66572]: pgmap v617: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:46:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:49 smithi044 ceph-mon[66366]: pgmap v617: 129 pgs: 129 active+clean; 2.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:46:51.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:51 smithi007 ceph-mon[62854]: pgmap v618: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:46:52.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:51 smithi204 ceph-mon[66572]: pgmap v618: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:46:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:51 smithi044 ceph-mon[66366]: pgmap v618: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:46:53.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:53 smithi007 ceph-mon[62854]: pgmap v619: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:46:54.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:53 smithi204 ceph-mon[66572]: pgmap v619: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:46:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:53 smithi044 ceph-mon[66366]: pgmap v619: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:46:56.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:55 smithi204 ceph-mon[66572]: pgmap v620: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-03-19T00:46:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:55 smithi044 ceph-mon[66366]: pgmap v620: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-03-19T00:46:56.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:55 smithi007 ceph-mon[62854]: pgmap v620: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-03-19T00:46:58.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:57 smithi204 ceph-mon[66572]: pgmap v621: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:57 smithi044 ceph-mon[66366]: pgmap v621: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:46:58.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:57 smithi007 ceph-mon[62854]: pgmap v621: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:00.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:46:59 smithi204 ceph-mon[66572]: pgmap v622: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:46:59 smithi044 ceph-mon[66366]: pgmap v622: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:46:59 smithi007 ceph-mon[62854]: pgmap v622: 129 pgs: 129 active+clean; 2.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:02.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:01 smithi204 ceph-mon[66572]: pgmap v623: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:47:02.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:01 smithi044 ceph-mon[66366]: pgmap v623: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:47:02.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:01 smithi007 ceph-mon[62854]: pgmap v623: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:47:04.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:03 smithi204 ceph-mon[66572]: pgmap v624: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:03 smithi044 ceph-mon[66366]: pgmap v624: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:04.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:03 smithi007 ceph-mon[62854]: pgmap v624: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:05.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:47:05.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:47:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 -- 172.21.15.7:0/1448226973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350102a40 msgr2=0x7f2350102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 --2- 172.21.15.7:0/1448226973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350102a40 0x7f2350102ec0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2340009900 tx=0x7f234002bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 -- 172.21.15.7:0/1448226973 shutdown_connections 2024-03-19T00:47:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 --2- 172.21.15.7:0/1448226973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 0x7f2350109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 --2- 172.21.15.7:0/1448226973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350102a40 0x7f2350102ec0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 --2- 172.21.15.7:0/1448226973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2350101850 0x7f2350101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 -- 172.21.15.7:0/1448226973 >> 172.21.15.7:0/1448226973 conn(0x7f23500fcde0 msgr2=0x7f23500ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 -- 172.21.15.7:0/1448226973 shutdown_connections 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.974+0000 7f2357f6f640 1 -- 172.21.15.7:0/1448226973 wait complete. 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 Processor -- start 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 -- start start 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 0x7f2350197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2350102a40 0x7f2350198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 0x7f235019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f235010d380 con 0x7f2350101850 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f235010d200 con 0x7f2350103400 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2357f6f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f235010d500 con 0x7f2350102a40 2024-03-19T00:47:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 0x7f2350197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 0x7f2350197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34362/0 (socket says 172.21.15.7:34362) 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 -- 172.21.15.7:0/3794981731 learned_addr learned my addr 172.21.15.7:0/3794981731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f235650c640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 0x7f235019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f235550a640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2350102a40 0x7f2350198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 -- 172.21.15.7:0/3794981731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2350102a40 msgr2=0x7f2350198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2350102a40 0x7f2350198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 -- 172.21.15.7:0/3794981731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 msgr2=0x7f235019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 0x7f235019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f2355d0b640 1 -- 172.21.15.7:0/3794981731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235019f700 con 0x7f2350101850 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.975+0000 7f235650c640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 0x7f235019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.976+0000 7f2355d0b640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 0x7f2350197ae0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e9c0 tx=0x7f234c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.976+0000 7f2346ffd640 1 -- 172.21.15.7:0/3794981731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c00cd80 con 0x7f2350101850 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.976+0000 7f2346ffd640 1 -- 172.21.15.7:0/3794981731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f234c0035f0 con 0x7f2350101850 2024-03-19T00:47:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.976+0000 7f2346ffd640 1 -- 172.21.15.7:0/3794981731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c010690 con 0x7f2350101850 2024-03-19T00:47:05.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.976+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235019f9f0 con 0x7f2350101850 2024-03-19T00:47:05.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.976+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235019fec0 con 0x7f2350101850 2024-03-19T00:47:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f2346ffd640 1 -- 172.21.15.7:0/3794981731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f234c0037a0 con 0x7f2350101850 2024-03-19T00:47:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 Processor -- start 2024-03-19T00:47:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 -- start start 2024-03-19T00:47:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 0x55fbc81f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc81fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc81ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fbc815c500 con 0x55fbc81fd5b0 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fbc8104be0 con 0x55fbc81f7a50 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235ac30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fbc81585a0 con 0x55fbc81f8440 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f2346ffd640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f233c0a2800 0x7f233c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235550a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc81ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.979+0000 7f235650c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 0x55fbc81f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235550a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc81ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34370/0 (socket says 172.21.15.7:34370) 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235550a640 1 -- 172.21.15.7:0/218579802 learned_addr learned my addr 172.21.15.7:0/218579802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 0x55fbc81f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50336/0 (socket says 172.21.15.7:50336) 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f2355d0b640 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc81fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f233c0a2800 0x7f233c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f2346ffd640 1 -- 172.21.15.7:0/3794981731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f234c0cf2c0 con 0x7f2350101850 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 -- 172.21.15.7:0/218579802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 msgr2=0x55fbc81fcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc81fcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 -- 172.21.15.7:0/218579802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 msgr2=0x55fbc81ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc81ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 -- 172.21.15.7:0/218579802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc8126000 con 0x55fbc81f7a50 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235550a640 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc81ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 0x55fbc81f7e70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f234800f510 tx=0x7f234800f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32fe184cc410470 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f235650c640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f233c0a2800 0x7f233c0a4cc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23480099e0 tx=0x7f234801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f233bfff640 1 -- 172.21.15.7:0/218579802 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348004b20 con 0x55fbc81f7a50 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f233bfff640 1 -- 172.21.15.7:0/218579802 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2348011580 con 0x55fbc81f7a50 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.980+0000 7f233bfff640 1 -- 172.21.15.7:0/218579802 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348004510 con 0x55fbc81f7a50 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 -- 172.21.15.7:0/218579802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 msgr2=0x55fbc81f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 0x55fbc81f7e70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f234800f510 tx=0x7f234800f9d0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 -- 172.21.15.7:0/218579802 shutdown_connections 2024-03-19T00:47:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc81ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc81fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 --2- 172.21.15.7:0/218579802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f7a50 0x55fbc81f7e70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 -- 172.21.15.7:0/218579802 >> 172.21.15.7:0/218579802 conn(0x55fbc81f2fc0 msgr2=0x55fbc81f33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 -- 172.21.15.7:0/218579802 shutdown_connections 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 -- 172.21.15.7:0/218579802 wait complete. 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.981+0000 7f235ac30740 1 -- start start 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 Processor -- start 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 0x55fbc8201e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc82023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc82093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fbc82004a0 con 0x55fbc81f7a50 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fbc8118160 con 0x55fbc81fd5b0 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fbc8200620 con 0x55fbc81f8440 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235ac30740 10 client.0 _ll_register_callbacks cb 0x55fbc8154a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 0x55fbc8201e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 0x55fbc8201e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34386/0 (socket says 172.21.15.7:34386) 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 -- 172.21.15.7:0/438786590 learned_addr learned my addr 172.21.15.7:0/438786590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235650c640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc82093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f235550a640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc82023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 -- 172.21.15.7:0/438786590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 msgr2=0x55fbc82023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc82023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 -- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 msgr2=0x55fbc82093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc82093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.982+0000 7f2355d0b640 1 -- 172.21.15.7:0/438786590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc7f3a7d0 con 0x55fbc81f7a50 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235650c640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc82093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f2355d0b640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 0x55fbc8201e70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c6a0 tx=0x7f234c014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19c0f4b52f323c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f23397fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f23397fa640 1 -- 172.21.15.7:0/438786590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c09c000 con 0x55fbc81f7a50 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f23397fa640 1 -- 172.21.15.7:0/438786590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f234c00aad0 con 0x55fbc81f7a50 2024-03-19T00:47:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2348011580 con 0x55fbc81f7a50 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f23397fa640 1 -- 172.21.15.7:0/438786590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f234c09b900 con 0x55fbc81f7a50 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 10 client.17370 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbc7e703f0 con 0x55fbc81f7a50 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f23397fa640 1 -- 172.21.15.7:0/438786590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2348011580 con 0x55fbc81f7a50 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f23397fa640 1 -- 172.21.15.7:0/438786590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f234c0afce0 con 0x55fbc81f7a50 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 10 client.17370 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 20 client.17370 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc820dc90 0x55fbc82100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 4 client.17370 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.983+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc8169a40 con 0x55fbc820dc90 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.984+0000 7f235550a640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc820dc90 0x55fbc82100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.984+0000 7f23397fa640 10 client.17370 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.984+0000 7f235550a640 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc820dc90 0x55fbc82100c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3e0d20caa8c1b2 server_cookie=7ac2461466068597 in_seq=0 out_seq=0 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f23397fa640 1 -- 172.21.15.7:0/438786590 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fbc8169a40 con 0x55fbc820dc90 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f23397fa640 10 client.17370 handle_command_reply: tid=0 2024-03-19T00:47:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 client.17370 shutdown 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 msgr2=0x55fbc8201e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 0x55fbc8201e70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c6a0 tx=0x7f234c014f10 comp rx=0 tx=0).stop 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 shutdown_connections 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc82093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f8440 0x55fbc82023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc820dc90 0x55fbc82100c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 --2- 172.21.15.7:0/438786590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f7a50 0x55fbc8201e70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f23397fa640 0 client.17370 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 >> 172.21.15.7:0/438786590 conn(0x55fbc81f2fc0 msgr2=0x55fbc81f21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 shutdown_connections 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 1 -- 172.21.15.7:0/438786590 wait complete. 2024-03-19T00:47:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:05.985+0000 7f235ac30740 20 client.17370 trim_cache size 0 max 16384 2024-03-19T00:47:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 Processor -- start 2024-03-19T00:47:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 -- start start 2024-03-19T00:47:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc81f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f8440 0x55fbc82093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc8202150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23501acb10 con 0x55fbc81fd5b0 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23501aa3e0 con 0x55fbc81f8440 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235ac30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23501a8230 con 0x55fbc81f7a50 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc8202150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc8202150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34398/0 (socket says 172.21.15.7:34398) 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 -- 172.21.15.7:0/2481968677 learned_addr learned my addr 172.21.15.7:0/2481968677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f2355d0b640 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc81f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235550a640 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f8440 0x55fbc82093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 -- 172.21.15.7:0/2481968677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 msgr2=0x55fbc81f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc81f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 -- 172.21.15.7:0/2481968677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f8440 msgr2=0x55fbc82093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f8440 0x55fbc82093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 -- 172.21.15.7:0/2481968677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc7e300f0 con 0x55fbc81fd5b0 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f235650c640 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc8202150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f234800fed0 tx=0x7f2348011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b5ea6e858eb8a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.003+0000 7f233b7fe640 1 -- 172.21.15.7:0/2481968677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348011640 con 0x55fbc81fd5b0 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f233b7fe640 1 -- 172.21.15.7:0/2481968677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2348011020 con 0x55fbc81fd5b0 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f233b7fe640 1 -- 172.21.15.7:0/2481968677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23480058f0 con 0x55fbc81fd5b0 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 -- 172.21.15.7:0/2481968677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 msgr2=0x55fbc8202150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc8202150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f234800fed0 tx=0x7f2348011c20 comp rx=0 tx=0).stop 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 -- 172.21.15.7:0/2481968677 shutdown_connections 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81fd5b0 0x55fbc8202150 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81f8440 0x55fbc82093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 --2- 172.21.15.7:0/2481968677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc81f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 -- 172.21.15.7:0/2481968677 >> 172.21.15.7:0/2481968677 conn(0x55fbc81f2fc0 msgr2=0x55fbc81f1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 -- 172.21.15.7:0/2481968677 shutdown_connections 2024-03-19T00:47:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 -- 172.21.15.7:0/2481968677 wait complete. 2024-03-19T00:47:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.004+0000 7f235ac30740 1 -- start start 2024-03-19T00:47:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 Processor -- start 2024-03-19T00:47:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc8206b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 0x55fbc82070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc8208020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23501a88a0 con 0x55fbc81f8440 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23501a9d10 con 0x55fbc81fd5b0 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23501a8f70 con 0x55fbc81f7a50 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235ac30740 10 client.0 _ll_register_callbacks cb 0x7f234c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235550a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 0x55fbc82070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235650c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc8208020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235550a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 0x55fbc82070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34414/0 (socket says 172.21.15.7:34414) 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f2355d0b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc8206b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235650c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc8208020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50352/0 (socket says 172.21.15.7:50352) 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235550a640 1 -- 172.21.15.7:0/1134914022 learned_addr learned my addr 172.21.15.7:0/1134914022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f2355d0b640 1 -- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 msgr2=0x55fbc8208020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f2355d0b640 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc8208020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f2355d0b640 1 -- 172.21.15.7:0/1134914022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 msgr2=0x55fbc82070b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f2355d0b640 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 0x55fbc82070b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f2355d0b640 1 -- 172.21.15.7:0/1134914022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc8208610 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.005+0000 7f235550a640 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 0x55fbc82070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f2355d0b640 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc8206b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f234c0982d0 tx=0x7f234c09b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e082fa73ee63db server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f23397fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f23397fa640 1 -- 172.21.15.7:0/1134914022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0862b0 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2348011020 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f23397fa640 1 -- 172.21.15.7:0/1134914022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f234c09ad50 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f23397fa640 1 -- 172.21.15.7:0/1134914022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f234c09c000 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 10 client.27133 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fbc8274000 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f23397fa640 1 -- 172.21.15.7:0/1134914022 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2348011020 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f23397fa640 1 -- 172.21.15.7:0/1134914022 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f234c0afce0 con 0x55fbc81f7a50 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 10 client.27133 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 20 client.27133 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc82748f0 0x55fbc8276d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 4 client.27133 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.006+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbc8163d90 con 0x55fbc82748f0 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235550a640 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc82748f0 0x55fbc8276d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f23397fa640 10 client.27133 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235550a640 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc82748f0 0x55fbc8276d10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b30d049d6b47e1f server_cookie=ecf7b98f4393958a in_seq=0 out_seq=0 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f23397fa640 1 -- 172.21.15.7:0/1134914022 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fbc8163d90 con 0x55fbc82748f0 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f23397fa640 10 client.27133 handle_command_reply: tid=0 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 client.27133 shutdown 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 msgr2=0x55fbc8206b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc8206b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f234c0982d0 tx=0x7f234c09b6d0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 shutdown_connections 2024-03-19T00:47:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fbc81fd5b0 0x55fbc8208020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55fbc82748f0 0x55fbc8276d10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fbc81f8440 0x55fbc82070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 --2- 172.21.15.7:0/1134914022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbc81f7a50 0x55fbc8206b70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f23397fa640 0 client.27133 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 >> 172.21.15.7:0/1134914022 conn(0x55fbc81f2fc0 msgr2=0x55fbc81f6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 shutdown_connections 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.007+0000 7f235ac30740 1 -- 172.21.15.7:0/1134914022 wait complete. 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f235ac30740 20 client.27133 trim_cache size 0 max 16384 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:47:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:47:06.016 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:47:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f233c0a2800 msgr2=0x7f233c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f233c0a2800 0x7f233c0a4cc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23480099e0 tx=0x7f234801e000 comp rx=0 tx=0).stop 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 msgr2=0x7f2350197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 0x7f2350197ae0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e9c0 tx=0x7f234c00ee80 comp rx=0 tx=0).stop 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 shutdown_connections 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f233c0a2800 0x7f233c0a4cc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2350103400 0x7f235019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2350102a40 0x7f2350198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 --2- 172.21.15.7:0/3794981731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2350101850 0x7f2350197ae0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 >> 172.21.15.7:0/3794981731 conn(0x7f23500fcde0 msgr2=0x7f2350105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 shutdown_connections 2024-03-19T00:47:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.008+0000 7f2357f6f640 1 -- 172.21.15.7:0/3794981731 wait complete. 2024-03-19T00:47:06.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:05 smithi204 ceph-mon[66572]: pgmap v625: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-03-19T00:47:06.018 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:05 smithi007 ceph-mon[62854]: pgmap v625: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-03-19T00:47:06.020 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:47:06.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:47:06.105 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:05 smithi044 ceph-mon[66366]: pgmap v625: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-03-19T00:47:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 -- 172.21.15.7:0/1195469754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc101940 msgr2=0x7fdbdc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 --2- 172.21.15.7:0/1195469754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc101940 0x7fdbdc101dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009860 tx=0x7fdbd002bb70 comp rx=0 tx=0).stop 2024-03-19T00:47:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 -- 172.21.15.7:0/1195469754 shutdown_connections 2024-03-19T00:47:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 --2- 172.21.15.7:0/1195469754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc102300 0x7fdbdc108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 --2- 172.21.15.7:0/1195469754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc101940 0x7fdbdc101dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 --2- 172.21.15.7:0/1195469754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc100750 0x7fdbdc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 -- 172.21.15.7:0/1195469754 >> 172.21.15.7:0/1195469754 conn(0x7fdbdc0fbce0 msgr2=0x7fdbdc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.122+0000 7fdbe312f640 1 -- 172.21.15.7:0/1195469754 shutdown_connections 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 -- 172.21.15.7:0/1195469754 wait complete. 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 Processor -- start 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 -- start start 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc100750 0x7fdbdc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc101940 0x7fdbdc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 0x7fdbdc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc10c260 con 0x7fdbdc102300 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc10c0e0 con 0x7fdbdc100750 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe312f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc10c3e0 con 0x7fdbdc101940 2024-03-19T00:47:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe16cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 0x7fdbdc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe0ecb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc100750 0x7fdbdc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe16cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 0x7fdbdc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34444/0 (socket says 172.21.15.7:34444) 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbe16cc640 1 -- 172.21.15.7:0/4058705002 learned_addr learned my addr 172.21.15.7:0/4058705002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.123+0000 7fdbdbfff640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc101940 0x7fdbdc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe16cc640 1 -- 172.21.15.7:0/4058705002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc101940 msgr2=0x7fdbdc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe16cc640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc101940 0x7fdbdc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe16cc640 1 -- 172.21.15.7:0/4058705002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc100750 msgr2=0x7fdbdc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe16cc640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc100750 0x7fdbdc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe16cc640 1 -- 172.21.15.7:0/4058705002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc19f710 con 0x7fdbdc102300 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe0ecb640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc100750 0x7fdbdc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe16cc640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 0x7fdbdc19f0a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00e9b0 tx=0x7fdbcc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00cd80 con 0x7fdbdc102300 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbcc0035f0 con 0x7fdbdc102300 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc19fa00 con 0x7fdbdc102300 2024-03-19T00:47:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc010690 con 0x7fdbdc102300 2024-03-19T00:47:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.124+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc19fed0 con 0x7fdbdc102300 2024-03-19T00:47:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.125+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc1a0100 con 0x7fdbdc102300 2024-03-19T00:47:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.126+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fdbcc0037a0 con 0x7fdbdc102300 2024-03-19T00:47:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.127+0000 7fdbd9ffb640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbc40a2800 0x7fdbc40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.127+0000 7fdbe0ecb640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbc40a2800 0x7fdbc40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.127+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdbcc0cf0f0 con 0x7fdbdc102300 2024-03-19T00:47:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.127+0000 7fdbe0ecb640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbc40a2800 0x7fdbc40a4cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc1017a0 tx=0x7fdbd4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.131+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdbcc09bdd0 con 0x7fdbdc102300 2024-03-19T00:47:06.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.354+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbdc1a0320 con 0x7fdbdc102300 2024-03-19T00:47:06.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.355+0000 7fdbd9ffb640 1 -- 172.21.15.7:0/4058705002 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fdbcc0860e0 con 0x7fdbdc102300 2024-03-19T00:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:47:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbc40a2800 msgr2=0x7fdbc40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbc40a2800 0x7fdbc40a4cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc1017a0 tx=0x7fdbd4010040 comp rx=0 tx=0).stop 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 msgr2=0x7fdbdc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 0x7fdbdc19f0a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00e9b0 tx=0x7fdbcc00ee70 comp rx=0 tx=0).stop 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 shutdown_connections 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdbc40a2800 0x7fdbc40a4cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbdc102300 0x7fdbdc19f0a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc101940 0x7fdbdc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 --2- 172.21.15.7:0/4058705002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdbdc100750 0x7fdbdc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 >> 172.21.15.7:0/4058705002 conn(0x7fdbdc0fbce0 msgr2=0x7fdbdc069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 shutdown_connections 2024-03-19T00:47:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.357+0000 7fdbe312f640 1 -- 172.21.15.7:0/4058705002 wait complete. 2024-03-19T00:47:06.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 -- 172.21.15.7:0/741332865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628102a10 msgr2=0x7fa628102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 --2- 172.21.15.7:0/741332865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628102a10 0x7fa628102e90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009900 tx=0x7fa61c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 -- 172.21.15.7:0/741332865 shutdown_connections 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 --2- 172.21.15.7:0/741332865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6281033d0 0x7fa628109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 --2- 172.21.15.7:0/741332865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628102a10 0x7fa628102e90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 --2- 172.21.15.7:0/741332865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628101820 0x7fa628101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 -- 172.21.15.7:0/741332865 >> 172.21.15.7:0/741332865 conn(0x7fa6280fcd50 msgr2=0x7fa6280ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 -- 172.21.15.7:0/741332865 shutdown_connections 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 -- 172.21.15.7:0/741332865 wait complete. 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 Processor -- start 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.474+0000 7fa62d431640 1 -- start start 2024-03-19T00:47:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 0x7fa628197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628102a10 0x7fa628197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6281033d0 0x7fa62819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa62810d350 con 0x7fa628101820 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa62810d1d0 con 0x7fa628102a10 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa62810d4d0 con 0x7fa6281033d0 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 0x7fa628197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 0x7fa628197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34474/0 (socket says 172.21.15.7:34474) 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628102a10 0x7fa628197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa626ffd640 1 -- 172.21.15.7:0/2212245895 learned_addr learned my addr 172.21.15.7:0/2212245895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6281033d0 0x7fa62819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 -- 172.21.15.7:0/2212245895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6281033d0 msgr2=0x7fa62819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6281033d0 0x7fa62819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 -- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 msgr2=0x7fa628197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 0x7fa628197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 -- 172.21.15.7:0/2212245895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819f670 con 0x7fa628102a10 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 0x7fa628197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628102a10 0x7fa628197f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa61c000c00 tx=0x7fa61c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa60ffff640 1 -- 172.21.15.7:0/2212245895 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c043020 con 0x7fa628102a10 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa60ffff640 1 -- 172.21.15.7:0/2212245895 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61c004350 con 0x7fa628102a10 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819f900 con 0x7fa628102a10 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa60ffff640 1 -- 172.21.15.7:0/2212245895 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c003b50 con 0x7fa628102a10 2024-03-19T00:47:06.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.475+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62819fe50 con 0x7fa628102a10 2024-03-19T00:47:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.477+0000 7fa60ffff640 1 -- 172.21.15.7:0/2212245895 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa61c046020 con 0x7fa628102a10 2024-03-19T00:47:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 Processor -- start 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 -- start start 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db42f0 0x559423db4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423db97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dbc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559423d184b0 con 0x559423db4ce0 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559423cc0d70 con 0x559423db9e50 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa630018740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559423d14550 con 0x559423db42f0 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa60ffff640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6100a2800 0x7fa6100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423db97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423db97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34482/0 (socket says 172.21.15.7:34482) 2024-03-19T00:47:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 -- 172.21.15.7:0/387254934 learned_addr learned my addr 172.21.15.7:0/387254934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa6267fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dbc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa6277fe640 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db42f0 0x559423db4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6100a2800 0x7fa6100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa60ffff640 1 -- 172.21.15.7:0/2212245895 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa61c0f0e80 con 0x7fa628102a10 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 -- 172.21.15.7:0/387254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db42f0 msgr2=0x559423db4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db42f0 0x559423db4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 -- 172.21.15.7:0/387254934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 msgr2=0x559423dbc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dbc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.478+0000 7fa626ffd640 1 -- 172.21.15.7:0/387254934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559423ce3210 con 0x559423db4ce0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6100a2800 0x7fa6100a4cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa620009710 tx=0x7fa620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa626ffd640 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423db97e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa618011900 tx=0x7fa618011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18a0b4cd5b21c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa60d7fa640 1 -- 172.21.15.7:0/387254934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618011330 con 0x559423db4ce0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa60d7fa640 1 -- 172.21.15.7:0/387254934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61800fe70 con 0x559423db4ce0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa60d7fa640 1 -- 172.21.15.7:0/387254934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618010e40 con 0x559423db4ce0 2024-03-19T00:47:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa630018740 1 -- 172.21.15.7:0/387254934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 msgr2=0x559423db97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.479+0000 7fa630018740 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423db97e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa618011900 tx=0x7fa618011d00 comp rx=0 tx=0).stop 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- 172.21.15.7:0/387254934 shutdown_connections 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dbc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423db97e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 --2- 172.21.15.7:0/387254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db42f0 0x559423db4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- 172.21.15.7:0/387254934 >> 172.21.15.7:0/387254934 conn(0x559423daf860 msgr2=0x559423dafc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- 172.21.15.7:0/387254934 shutdown_connections 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- 172.21.15.7:0/387254934 wait complete. 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- start start 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 Processor -- start 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423dbe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dbec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 0x559423dc5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559423dbce00 con 0x559423db9e50 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559423cd4310 con 0x559423db42f0 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559423dbcf80 con 0x559423db4ce0 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa630018740 10 client.0 _ll_register_callbacks cb 0x559423cc82d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 0x559423dc5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 0x559423dc5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34492/0 (socket says 172.21.15.7:34492) 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 -- 172.21.15.7:0/2941845072 learned_addr learned my addr 172.21.15.7:0/2941845072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423dbe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dbec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 -- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 msgr2=0x559423dbec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dbec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 -- 172.21.15.7:0/2941845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 msgr2=0x559423dbe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.480+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423dbe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa6277fe640 1 -- 172.21.15.7:0/2941845072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559423a2c3f0 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 0x559423dc5ca0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa62000fae0 tx=0x7fa620004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1500390933d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa60aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa60aff5640 1 -- 172.21.15.7:0/2941845072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6200044e0 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa60aff5640 1 -- 172.21.15.7:0/2941845072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa62001a050 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61800fe70 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa60aff5640 1 -- 172.21.15.7:0/2941845072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa620034050 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 10 client.17394 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559423af67d0 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa60aff5640 1 -- 172.21.15.7:0/2941845072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61800fe70 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa60aff5640 1 -- 172.21.15.7:0/2941845072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa620026480 con 0x559423db9e50 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 10 client.17394 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 10 client.17394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 20 client.17394 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423dca690 0x559423dccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 4 client.17394 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.481+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559423d763e0 con 0x559423dca690 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.482+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423dca690 0x559423dccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.482+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423dca690 0x559423dccb10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76bd3cee28e36a7c server_cookie=80c8245b247762f2 in_seq=0 out_seq=0 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.482+0000 7fa60aff5640 10 client.17394 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa60aff5640 1 -- 172.21.15.7:0/2941845072 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559423d763e0 con 0x559423dca690 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa60aff5640 10 client.17394 handle_command_reply: tid=0 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 client.17394 shutdown 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 msgr2=0x559423dc5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 0x559423dc5ca0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa62000fae0 tx=0x7fa620004730 comp rx=0 tx=0).stop 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 shutdown_connections 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423dca690 0x559423dccb10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db9e50 0x559423dc5ca0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dbec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa60aff5640 0 client.17394 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 --2- 172.21.15.7:0/2941845072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423dbe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 >> 172.21.15.7:0/2941845072 conn(0x559423daf860 msgr2=0x559423daeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 shutdown_connections 2024-03-19T00:47:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 1 -- 172.21.15.7:0/2941845072 wait complete. 2024-03-19T00:47:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.483+0000 7fa630018740 20 client.17394 trim_cache size 0 max 16384 2024-03-19T00:47:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 Processor -- start 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 -- start start 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423db9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423dc5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db9e50 0x559423daaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281aa980 con 0x559423db4ce0 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281a8250 con 0x559423db42f0 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa630018740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281a60a0 con 0x559423db9e50 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa6267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423dc5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423db9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa6267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423dc5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34498/0 (socket says 172.21.15.7:34498) 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423db9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50422/0 (socket says 172.21.15.7:50422) 2024-03-19T00:47:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa6267fc640 1 -- 172.21.15.7:0/4172128632 learned_addr learned my addr 172.21.15.7:0/4172128632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.496+0000 7fa6277fe640 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db9e50 0x559423daaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa6267fc640 1 -- 172.21.15.7:0/4172128632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db9e50 msgr2=0x559423daaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa6267fc640 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db9e50 0x559423daaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa6267fc640 1 -- 172.21.15.7:0/4172128632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 msgr2=0x559423db9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa6267fc640 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423db9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa6267fc640 1 -- 172.21.15.7:0/4172128632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559423d85880 con 0x559423db4ce0 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa626ffd640 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423db9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa6267fc640 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423dc5c60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa61c02c170 tx=0x7fa61c0b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f9ce5ed6d2adf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa60cff9640 1 -- 172.21.15.7:0/4172128632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0bcdc0 con 0x559423db4ce0 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa60cff9640 1 -- 172.21.15.7:0/4172128632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61c0bdb60 con 0x559423db4ce0 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.497+0000 7fa60cff9640 1 -- 172.21.15.7:0/4172128632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c003cf0 con 0x559423db4ce0 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- 172.21.15.7:0/4172128632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 msgr2=0x559423dc5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423dc5c60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa61c02c170 tx=0x7fa61c0b9040 comp rx=0 tx=0).stop 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- 172.21.15.7:0/4172128632 shutdown_connections 2024-03-19T00:47:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db9e50 0x559423daaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db4ce0 0x559423dc5c60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- 172.21.15.7:0/4172128632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db42f0 0x559423db9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- 172.21.15.7:0/4172128632 >> 172.21.15.7:0/4172128632 conn(0x559423daf860 msgr2=0x559423dafc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- 172.21.15.7:0/4172128632 shutdown_connections 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- 172.21.15.7:0/4172128632 wait complete. 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- start start 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 Processor -- start 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 0x559423dc33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dc3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dc4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281a6710 con 0x559423db42f0 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281a7b80 con 0x559423db9e50 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281a6de0 con 0x559423db4ce0 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa630018740 10 client.0 _ll_register_callbacks cb 0x7fa620004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 0x559423dc33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 0x559423dc33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34514/0 (socket says 172.21.15.7:34514) 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa626ffd640 1 -- 172.21.15.7:0/2050132316 learned_addr learned my addr 172.21.15.7:0/2050132316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa6277fe640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dc4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dc3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa626ffd640 1 -- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 msgr2=0x559423dc3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.498+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dc3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa626ffd640 1 -- 172.21.15.7:0/2050132316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 msgr2=0x559423dc4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dc4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa626ffd640 1 -- 172.21.15.7:0/2050132316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559423dc4f10 con 0x559423db42f0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa626ffd640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 0x559423dc33d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa61800f7b0 tx=0x7fa618009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7caae79b7e25b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa60aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa60aff5640 1 -- 172.21.15.7:0/2050132316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61800f9c0 con 0x559423db42f0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa60aff5640 1 -- 172.21.15.7:0/2050132316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa61800fb60 con 0x559423db42f0 2024-03-19T00:47:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61c0bdb60 con 0x559423db42f0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa60aff5640 1 -- 172.21.15.7:0/2050132316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa618037050 con 0x559423db42f0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 10 client.17400 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559423e30b10 con 0x559423db42f0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa60aff5640 1 -- 172.21.15.7:0/2050132316 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61c0bdb60 con 0x559423db42f0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa60aff5640 1 -- 172.21.15.7:0/2050132316 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa618018070 con 0x559423db42f0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 10 client.17400 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 10 client.17400 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 20 client.17400 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423e31300 0x559423e33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 4 client.17400 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.499+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559423d259f0 con 0x559423e31300 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423e31300 0x559423e33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa60aff5640 10 client.17400 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa6267fc640 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423e31300 0x559423e33720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ef01fd271b4ecb server_cookie=8c54de09fe29a861 in_seq=0 out_seq=0 2024-03-19T00:47:06.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa60aff5640 1 -- 172.21.15.7:0/2050132316 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559423d259f0 con 0x559423e31300 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa60aff5640 10 client.17400 handle_command_reply: tid=0 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 client.17400 shutdown 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 msgr2=0x559423dc33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 0x559423dc33d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa61800f7b0 tx=0x7fa618009590 comp rx=0 tx=0).stop 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 shutdown_connections 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x559423e31300 0x559423e33720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559423db9e50 0x559423dc4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559423db4ce0 0x559423dc3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa60aff5640 0 client.17400 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 --2- 172.21.15.7:0/2050132316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559423db42f0 0x559423dc33d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 >> 172.21.15.7:0/2050132316 conn(0x559423daf860 msgr2=0x559423daaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 shutdown_connections 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 1 -- 172.21.15.7:0/2050132316 wait complete. 2024-03-19T00:47:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.500+0000 7fa630018740 20 client.17400 trim_cache size 0 max 16384 2024-03-19T00:47:06.509 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6100a2800 msgr2=0x7fa6100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6100a2800 0x7fa6100a4cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa620009710 tx=0x7fa620005f90 comp rx=0 tx=0).stop 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628102a10 msgr2=0x7fa628197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628102a10 0x7fa628197f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa61c000c00 tx=0x7fa61c005dc0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 shutdown_connections 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6100a2800 0x7fa6100a4cc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6281033d0 0x7fa62819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa628102a10 0x7fa628197f90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 --2- 172.21.15.7:0/2212245895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa628101820 0x7fa628197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 >> 172.21.15.7:0/2212245895 conn(0x7fa6280fcd50 msgr2=0x7fa628105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 shutdown_connections 2024-03-19T00:47:06.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.501+0000 7fa62d431640 1 -- 172.21.15.7:0/2212245895 wait complete. 2024-03-19T00:47:06.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 -- 172.21.15.7:0/2271927169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4103400 msgr2=0x7fa2a4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/2271927169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4103400 0x7fa2a4109c90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 -- 172.21.15.7:0/2271927169 shutdown_connections 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/2271927169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4103400 0x7fa2a4109c90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/2271927169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4102a40 0x7fa2a4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/2271927169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a4101850 0x7fa2a4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 -- 172.21.15.7:0/2271927169 >> 172.21.15.7:0/2271927169 conn(0x7fa2a40fcd80 msgr2=0x7fa2a40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 -- 172.21.15.7:0/2271927169 shutdown_connections 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.616+0000 7fa2ab46c640 1 -- 172.21.15.7:0/2271927169 wait complete. 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 Processor -- start 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 -- start start 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 0x7fa2a41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a4102a40 0x7fa2a4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4103400 0x7fa2a419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410d360 con 0x7fa2a4101850 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410d1e0 con 0x7fa2a4102a40 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410d4e0 con 0x7fa2a4103400 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 0x7fa2a41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 0x7fa2a41979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34534/0 (socket says 172.21.15.7:34534) 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 -- 172.21.15.7:0/3153860713 learned_addr learned my addr 172.21.15.7:0/3153860713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a8a07640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a4102a40 0x7fa2a4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4103400 0x7fa2a419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 -- 172.21.15.7:0/3153860713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4103400 msgr2=0x7fa2a419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4103400 0x7fa2a419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 -- 172.21.15.7:0/3153860713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a4102a40 msgr2=0x7fa2a4197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a4102a40 0x7fa2a4197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 -- 172.21.15.7:0/3153860713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a419f5f0 con 0x7fa2a4101850 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2a9208640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 0x7fa2a41979d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000e9c0 tx=0x7fa2a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa29a7fc640 1 -- 172.21.15.7:0/3153860713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000cd80 con 0x7fa2a4101850 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa29a7fc640 1 -- 172.21.15.7:0/3153860713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a00035f0 con 0x7fa2a4101850 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a419f8e0 con 0x7fa2a4101850 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.617+0000 7fa29a7fc640 1 -- 172.21.15.7:0/3153860713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0010690 con 0x7fa2a4101850 2024-03-19T00:47:06.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.618+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a419fe80 con 0x7fa2a4101850 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa29a7fc640 1 -- 172.21.15.7:0/3153860713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa2a00037a0 con 0x7fa2a4101850 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 Processor -- start 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 -- start start 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa29a7fc640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa2900a2880 0x7fa2900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556dcb0134b0 con 0x556dcb0b4e50 2024-03-19T00:47:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556dcafbbd70 con 0x556dcb0af2f0 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2ae058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556dcb00f550 con 0x556dcb0afce0 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa2900a2880 0x7fa2900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.620+0000 7fa29a7fc640 1 -- 172.21.15.7:0/3153860713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa2a00cf130 con 0x7fa2a4101850 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9a09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50448/0 (socket says 172.21.15.7:50448) 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9a09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34548/0 (socket says 172.21.15.7:34548) 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 -- 172.21.15.7:0/2959610469 learned_addr learned my addr 172.21.15.7:0/2959610469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a8a07640 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 -- 172.21.15.7:0/2959610469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 msgr2=0x556dcb0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 -- 172.21.15.7:0/2959610469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 msgr2=0x556dcb0b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 -- 172.21.15.7:0/2959610469 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dcafde210 con 0x556dcb0af2f0 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa2900a2880 0x7fa2900a4d40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006010 tx=0x7fa29c009930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa2a9208640 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0af710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0097b70 tx=0x7fa2a009b250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0a995f1391804 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa28f7fe640 1 -- 172.21.15.7:0/2959610469 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a009be10 con 0x556dcb0af2f0 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa28f7fe640 1 -- 172.21.15.7:0/2959610469 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a009b710 con 0x556dcb0af2f0 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.621+0000 7fa28f7fe640 1 -- 172.21.15.7:0/2959610469 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a008ade0 con 0x556dcb0af2f0 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 -- 172.21.15.7:0/2959610469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 msgr2=0x556dcb0af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0af710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0097b70 tx=0x7fa2a009b250 comp rx=0 tx=0).stop 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 -- 172.21.15.7:0/2959610469 shutdown_connections 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 --2- 172.21.15.7:0/2959610469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0af710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 -- 172.21.15.7:0/2959610469 >> 172.21.15.7:0/2959610469 conn(0x556dcb0aa860 msgr2=0x556dcb0aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 -- 172.21.15.7:0/2959610469 shutdown_connections 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 -- 172.21.15.7:0/2959610469 wait complete. 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 -- start start 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.622+0000 7fa2ae058740 1 Processor -- start 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0af2f0 0x556dcb0b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0afce0 0x556dcb0b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556dcb0b7e00 con 0x556dcb0b4e50 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556dcafcf310 con 0x556dcb0afce0 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556dcb0b7f80 con 0x556dcb0af2f0 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2ae058740 10 client.0 _ll_register_callbacks cb 0x556dcafc32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9208640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0af2f0 0x556dcb0b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a8a07640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0afce0 0x556dcb0b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34558/0 (socket says 172.21.15.7:34558) 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1479358233 learned_addr learned my addr 172.21.15.7:0/1479358233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1479358233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0af2f0 msgr2=0x556dcb0b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0af2f0 0x556dcb0b9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0afce0 msgr2=0x556dcb0b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0afce0 0x556dcb0b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1479358233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dcad273f0 con 0x556dcb0b4e50 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0af2f0 0x556dcb0b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0c0d90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c0324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51e3527aff14a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.623+0000 7fa28cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa28cff9640 1 -- 172.21.15.7:0/1479358233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c051060 con 0x556dcb0b4e50 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa28cff9640 1 -- 172.21.15.7:0/1479358233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa29c004d40 con 0x556dcb0b4e50 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a009b710 con 0x556dcb0b4e50 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa28cff9640 1 -- 172.21.15.7:0/1479358233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa29c058050 con 0x556dcb0b4e50 2024-03-19T00:47:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 10 client.17412 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556dcadf17d0 con 0x556dcb0b4e50 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa28cff9640 1 -- 172.21.15.7:0/1479358233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a009b710 con 0x556dcb0b4e50 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa28cff9640 1 -- 172.21.15.7:0/1479358233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa29c03f030 con 0x556dcb0b4e50 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 10 client.17412 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 20 client.17412 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb0c56f0 0x556dcb0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 4 client.17412 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dcb0713e0 con 0x556dcb0c56f0 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.624+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb0c56f0 0x556dcb0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.625+0000 7fa28cff9640 10 client.17412 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.625+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb0c56f0 0x556dcb0c7b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75ed2c54743fcf22 server_cookie=6bf17487c26ebca2 in_seq=0 out_seq=0 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa28cff9640 1 -- 172.21.15.7:0/1479358233 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556dcb0713e0 con 0x556dcb0c56f0 2024-03-19T00:47:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa28cff9640 10 client.17412 handle_command_reply: tid=0 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 client.17412 shutdown 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 msgr2=0x556dcb0c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0c0d90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c0324f0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 shutdown_connections 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb0c56f0 0x556dcb0c7b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0c0d90 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0afce0 0x556dcb0b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1479358233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0af2f0 0x556dcb0b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa28cff9640 0 client.17412 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 >> 172.21.15.7:0/1479358233 conn(0x556dcb0aa860 msgr2=0x556dcb0a9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 shutdown_connections 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 1 -- 172.21.15.7:0/1479358233 wait complete. 2024-03-19T00:47:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.626+0000 7fa2ae058740 20 client.17412 trim_cache size 0 max 16384 2024-03-19T00:47:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 Processor -- start 2024-03-19T00:47:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 -- start start 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 0x556dcb0c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0b4e50 0x556dcb0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41acb30 con 0x556dcb0afce0 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41aa400 con 0x556dcb0af2f0 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2ae058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a8250 con 0x556dcb0b4e50 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2a8a07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 0x556dcb0c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2a8a07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 0x556dcb0c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34562/0 (socket says 172.21.15.7:34562) 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2a8a07640 1 -- 172.21.15.7:0/1571500220 learned_addr learned my addr 172.21.15.7:0/1571500220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.639+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0b4e50 0x556dcb0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a8a07640 1 -- 172.21.15.7:0/1571500220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0b4e50 msgr2=0x556dcb0b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a8a07640 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0b4e50 0x556dcb0b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a8a07640 1 -- 172.21.15.7:0/1571500220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 msgr2=0x556dcb0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a8a07640 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a8a07640 1 -- 172.21.15.7:0/1571500220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dcb080880 con 0x556dcb0afce0 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa2a8a07640 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 0x556dcb0c0d50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa294009e90 tx=0x7fa294005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9030986c8c4451d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa28effd640 1 -- 172.21.15.7:0/1571500220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294022070 con 0x556dcb0afce0 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa28effd640 1 -- 172.21.15.7:0/1571500220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2940035f0 con 0x556dcb0afce0 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.640+0000 7fa28effd640 1 -- 172.21.15.7:0/1571500220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29401d6c0 con 0x556dcb0afce0 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- 172.21.15.7:0/1571500220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 msgr2=0x556dcb0c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 0x556dcb0c0d50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa294009e90 tx=0x7fa294005f90 comp rx=0 tx=0).stop 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- 172.21.15.7:0/1571500220 shutdown_connections 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0b4e50 0x556dcb0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0afce0 0x556dcb0c0d50 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1571500220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- 172.21.15.7:0/1571500220 >> 172.21.15.7:0/1571500220 conn(0x556dcb0aa860 msgr2=0x556dcb0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- 172.21.15.7:0/1571500220 shutdown_connections 2024-03-19T00:47:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- 172.21.15.7:0/1571500220 wait complete. 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- start start 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 Processor -- start 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a88c0 con 0x556dcb0b4e50 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a9d30 con 0x556dcb0af2f0 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41a8f90 con 0x556dcb0afce0 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2ae058740 10 client.0 _ll_register_callbacks cb 0x7fa294005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2a9a09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2a9a09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34576/0 (socket says 172.21.15.7:34576) 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1709074075 learned_addr learned my addr 172.21.15.7:0/1709074075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.641+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a8a07640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1709074075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 msgr2=0x556dcb0bea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0bea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 msgr2=0x556dcb0be4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0be4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a9a09640 1 -- 172.21.15.7:0/1709074075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dcb0c0020 con 0x556dcb0b4e50 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2a9a09640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0bfa30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa29c033000 tx=0x7fa29c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f66ec6abf0fd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa28cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa28cff9640 1 -- 172.21.15.7:0/1709074075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c051070 con 0x556dcb0b4e50 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa28cff9640 1 -- 172.21.15.7:0/1709074075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa29c004410 con 0x556dcb0b4e50 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2940035f0 con 0x556dcb0b4e50 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa28cff9640 1 -- 172.21.15.7:0/1709074075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa29c039040 con 0x556dcb0b4e50 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2ae058740 10 client.17418 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556dcb12bad0 con 0x556dcb0b4e50 2024-03-19T00:47:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.642+0000 7fa28cff9640 1 -- 172.21.15.7:0/1709074075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2940035f0 con 0x556dcb0b4e50 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa28cff9640 1 -- 172.21.15.7:0/1709074075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa29c03f070 con 0x556dcb0b4e50 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 10 client.17418 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 10 client.17418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 20 client.17418 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb12c400 0x556dcb12e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 4 client.17418 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dcb0209f0 con 0x556dcb12c400 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb12c400 0x556dcb12e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa28cff9640 10 client.17418 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2a9208640 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb12c400 0x556dcb12e820 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e6f95cc45eb505 server_cookie=da2e7d535b9564ed in_seq=0 out_seq=0 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa28cff9640 1 -- 172.21.15.7:0/1709074075 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dcb0209f0 con 0x556dcb12c400 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa28cff9640 10 client.17418 handle_command_reply: tid=0 2024-03-19T00:47:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 client.17418 shutdown 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 msgr2=0x556dcb0bfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0bfa30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa29c033000 tx=0x7fa29c005bb0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 shutdown_connections 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x556dcb12c400 0x556dcb12e820 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dcb0b4e50 0x556dcb0bfa30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dcb0afce0 0x556dcb0bea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa28cff9640 0 client.17418 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 --2- 172.21.15.7:0/1709074075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dcb0af2f0 0x556dcb0be4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.643+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 >> 172.21.15.7:0/1709074075 conn(0x556dcb0aa860 msgr2=0x556dcb0ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 shutdown_connections 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ae058740 1 -- 172.21.15.7:0/1709074075 wait complete. 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ae058740 20 client.17418 trim_cache size 0 max 16384 2024-03-19T00:47:06.651 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:47:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa2900a2880 msgr2=0x7fa2900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa2900a2880 0x7fa2900a4d40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006010 tx=0x7fa29c009930 comp rx=0 tx=0).stop 2024-03-19T00:47:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 msgr2=0x7fa2a41979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 0x7fa2a41979d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000e9c0 tx=0x7fa2a000ee80 comp rx=0 tx=0).stop 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 shutdown_connections 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa2900a2880 0x7fa2900a4d40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4103400 0x7fa2a419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a4102a40 0x7fa2a4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 --2- 172.21.15.7:0/3153860713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a4101850 0x7fa2a41979d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 >> 172.21.15.7:0/3153860713 conn(0x7fa2a40fcd80 msgr2=0x7fa2a4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 shutdown_connections 2024-03-19T00:47:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.644+0000 7fa2ab46c640 1 -- 172.21.15.7:0/3153860713 wait complete. 2024-03-19T00:47:06.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 -- 172.21.15.7:0/4272312787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c80ff880 msgr2=0x7f74c80ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 --2- 172.21.15.7:0/4272312787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c80ff880 0x7f74c80ffd00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009900 tx=0x7f74b402bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 -- 172.21.15.7:0/4272312787 shutdown_connections 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 --2- 172.21.15.7:0/4272312787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74c8100240 0x7f74c8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 --2- 172.21.15.7:0/4272312787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c80ff880 0x7f74c80ffd00 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 --2- 172.21.15.7:0/4272312787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c8102e10 0x7f74c8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 -- 172.21.15.7:0/4272312787 >> 172.21.15.7:0/4272312787 conn(0x7f74c80fb400 msgr2=0x7f74c80fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 -- 172.21.15.7:0/4272312787 shutdown_connections 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.760+0000 7f74cea6c640 1 -- 172.21.15.7:0/4272312787 wait complete. 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 Processor -- start 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 -- start start 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c80ff880 0x7f74c8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 0x7f74c8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74c8102e10 0x7f74c819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74c810d370 con 0x7f74c8100240 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74c810d1f0 con 0x7f74c8102e10 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cea6c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c810d4f0 con 0x7f74c80ff880 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 0x7f74c8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 0x7f74c8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34600/0 (socket says 172.21.15.7:34600) 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 -- 172.21.15.7:0/1395559676 learned_addr learned my addr 172.21.15.7:0/1395559676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cc808640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c80ff880 0x7f74c8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74cd009640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74c8102e10 0x7f74c819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 -- 172.21.15.7:0/1395559676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c80ff880 msgr2=0x7f74c8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c80ff880 0x7f74c8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 -- 172.21.15.7:0/1395559676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74c8102e10 msgr2=0x7f74c819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74c8102e10 0x7f74c819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 -- 172.21.15.7:0/1395559676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c819f680 con 0x7f74c8100240 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.761+0000 7f74c7fff640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 0x7f74c8197ff0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f74b4005ab0 tx=0x7f74b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.762+0000 7f74c5ffb640 1 -- 172.21.15.7:0/1395559676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4043020 con 0x7f74c8100240 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.762+0000 7f74c5ffb640 1 -- 172.21.15.7:0/1395559676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74b4004a50 con 0x7f74c8100240 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.762+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c819f910 con 0x7f74c8100240 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.762+0000 7f74c5ffb640 1 -- 172.21.15.7:0/1395559676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4003ba0 con 0x7f74c8100240 2024-03-19T00:47:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.762+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c819fe60 con 0x7f74c8100240 2024-03-19T00:47:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c5ffb640 1 -- 172.21.15.7:0/1395559676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f74b4046020 con 0x7f74c8100240 2024-03-19T00:47:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 Processor -- start 2024-03-19T00:47:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 -- start start 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c60710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 0x556f95c657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 0x556f95c682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556f95bc44b0 con 0x556f95c65e50 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556f95b6cd70 con 0x556f95c60ce0 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74d1658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556f95bc0550 con 0x556f95c602f0 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 0x556f95c682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74cc808640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 0x556f95c657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 0x556f95c682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34612/0 (socket says 172.21.15.7:34612) 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 -- 172.21.15.7:0/8566132 learned_addr learned my addr 172.21.15.7:0/8566132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74cc808640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 0x556f95c657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50504/0 (socket says 172.21.15.7:50504) 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74cd009640 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c60710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c5ffb640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74b80a2800 0x7f74b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 -- 172.21.15.7:0/8566132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 msgr2=0x556f95c60710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c60710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 -- 172.21.15.7:0/8566132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 msgr2=0x556f95c657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 0x556f95c657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c5ffb640 1 -- 172.21.15.7:0/1395559676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f74b40f0fa0 con 0x7f74c8100240 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74c7fff640 1 -- 172.21.15.7:0/8566132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f95b8f210 con 0x556f95c65e50 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.765+0000 7f74cd009640 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c60710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74cc808640 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 0x556f95c657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74cd009640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74b80a2800 0x7f74b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74c7fff640 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 0x556f95c682e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f74b40ee740 tx=0x7f74b402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9c493f4aa91837 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74b2ffd640 1 -- 172.21.15.7:0/8566132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b40bd3b0 con 0x556f95c65e50 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74b2ffd640 1 -- 172.21.15.7:0/8566132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74b40bdce0 con 0x556f95c65e50 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74b2ffd640 1 -- 172.21.15.7:0/8566132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b40044e0 con 0x556f95c65e50 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74d1658740 1 -- 172.21.15.7:0/8566132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 msgr2=0x556f95c682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.766+0000 7f74cd009640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74b80a2800 0x7f74b80a4cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x556f95c61540 tx=0x7f74c0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 0x556f95c682e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f74b40ee740 tx=0x7f74b402bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- 172.21.15.7:0/8566132 shutdown_connections 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c65e50 0x556f95c682e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c60ce0 0x556f95c657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- 172.21.15.7:0/8566132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c60710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- 172.21.15.7:0/8566132 >> 172.21.15.7:0/8566132 conn(0x556f95c5b860 msgr2=0x556f95c5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- 172.21.15.7:0/8566132 shutdown_connections 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- 172.21.15.7:0/8566132 wait complete. 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- start start 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 Processor -- start 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c6a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 0x556f95c6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556f95c68bc0 con 0x556f95c60ce0 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556f95b80310 con 0x556f95c65e50 2024-03-19T00:47:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556f95c68d40 con 0x556f95c602f0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74d1658740 10 client.0 _ll_register_callbacks cb 0x556f95b742d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 0x556f95c6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 0x556f95c6a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34628/0 (socket says 172.21.15.7:34628) 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74c7fff640 1 -- 172.21.15.7:0/2521617963 learned_addr learned my addr 172.21.15.7:0/2521617963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.767+0000 7f74cd009640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c71a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74cc808640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c6a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74c7fff640 1 -- 172.21.15.7:0/2521617963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 msgr2=0x556f95c6a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74c7fff640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c6a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74c7fff640 1 -- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 msgr2=0x556f95c71a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74c7fff640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c71a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74c7fff640 1 -- 172.21.15.7:0/2521617963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f958d83f0 con 0x556f95c60ce0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74c7fff640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 0x556f95c6a9e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74b40eeab0 tx=0x7f74b40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d12da2d6d30c3b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74a7fff640 1 -- 172.21.15.7:0/2521617963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4003d40 con 0x556f95c60ce0 2024-03-19T00:47:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74a7fff640 1 -- 172.21.15.7:0/2521617963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f74b40bb680 con 0x556f95c60ce0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b40bdce0 con 0x556f95c60ce0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74a7fff640 1 -- 172.21.15.7:0/2521617963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f74b40bcf40 con 0x556f95c60ce0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74d1658740 10 client.17430 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f959a27d0 con 0x556f95c60ce0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.768+0000 7f74a7fff640 1 -- 172.21.15.7:0/2521617963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74b40bdce0 con 0x556f95c60ce0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74a7fff640 1 -- 172.21.15.7:0/2521617963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f74b40c58f0 con 0x556f95c60ce0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74d1658740 10 client.17430 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74d1658740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74d1658740 20 client.17430 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74d1658740 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95c75fb0 0x556f95c783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74d1658740 4 client.17430 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f95c223e0 con 0x556f95c75fb0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74cc808640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95c75fb0 0x556f95c783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74a7fff640 10 client.17430 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.769+0000 7f74cc808640 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95c75fb0 0x556f95c783d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe494ca1039c620 server_cookie=21218a0fbd587c43 in_seq=0 out_seq=0 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74a7fff640 1 -- 172.21.15.7:0/2521617963 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556f95c223e0 con 0x556f95c75fb0 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74a7fff640 10 client.17430 handle_command_reply: tid=0 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 client.17430 shutdown 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 msgr2=0x556f95c6a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 0x556f95c6a9e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74b40eeab0 tx=0x7f74b40095e0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 shutdown_connections 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95c75fb0 0x556f95c783d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c60ce0 0x556f95c6a9e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74a7fff640 0 client.17430 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 --2- 172.21.15.7:0/2521617963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c602f0 0x556f95c6a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 >> 172.21.15.7:0/2521617963 conn(0x556f95c5b860 msgr2=0x556f95c5aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 shutdown_connections 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 1 -- 172.21.15.7:0/2521617963 wait complete. 2024-03-19T00:47:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.770+0000 7f74d1658740 20 client.17430 trim_cache size 0 max 16384 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 Processor -- start 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 -- start start 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c69ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81acaf0 con 0x556f95c602f0 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81aa3c0 con 0x556f95c65e50 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.783+0000 7f74d1658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81a8210 con 0x556f95c60ce0 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c69ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c69ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34630/0 (socket says 172.21.15.7:34630) 2024-03-19T00:47:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 -- 172.21.15.7:0/2305838743 learned_addr learned my addr 172.21.15.7:0/2305838743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74c7fff640 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cd009640 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 -- 172.21.15.7:0/2305838743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 msgr2=0x556f95c719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 -- 172.21.15.7:0/2305838743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 msgr2=0x556f95c5a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c5a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 -- 172.21.15.7:0/2305838743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f95c31880 con 0x556f95c602f0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74cc808640 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c69ea0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009d10 tx=0x7f74bc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fabaa4dffc793df server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74b27fc640 1 -- 172.21.15.7:0/2305838743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc025070 con 0x556f95c602f0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74b27fc640 1 -- 172.21.15.7:0/2305838743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74bc004310 con 0x556f95c602f0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.784+0000 7f74b27fc640 1 -- 172.21.15.7:0/2305838743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc003b40 con 0x556f95c602f0 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- 172.21.15.7:0/2305838743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 msgr2=0x556f95c69ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c69ea0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009d10 tx=0x7f74bc005f90 comp rx=0 tx=0).stop 2024-03-19T00:47:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- 172.21.15.7:0/2305838743 shutdown_connections 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c5a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- 172.21.15.7:0/2305838743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c69ea0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- 172.21.15.7:0/2305838743 >> 172.21.15.7:0/2305838743 conn(0x556f95c5b860 msgr2=0x556f95c69790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- 172.21.15.7:0/2305838743 shutdown_connections 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- 172.21.15.7:0/2305838743 wait complete. 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- start start 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 Processor -- start 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c71390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c6f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81a8880 con 0x556f95c602f0 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81a9cf0 con 0x556f95c65e50 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.785+0000 7f74d1658740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81a8f50 con 0x556f95c60ce0 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74d1658740 10 client.0 _ll_register_callbacks cb 0x7f74bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c71390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c71390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34632/0 (socket says 172.21.15.7:34632) 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 -- 172.21.15.7:0/484445 learned_addr learned my addr 172.21.15.7:0/484445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cd009640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c6f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74c7fff640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 -- 172.21.15.7:0/484445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 msgr2=0x556f95c718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 -- 172.21.15.7:0/484445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 msgr2=0x556f95c6f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c6f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 -- 172.21.15.7:0/484445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f95c701b0 con 0x556f95c602f0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74cc808640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c71390 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0104e0 tx=0x7f74bc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80590de8ac085c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74a7fff640 1 -- 172.21.15.7:0/484445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc030060 con 0x556f95c602f0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74a7fff640 1 -- 172.21.15.7:0/484445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f74bc0038c0 con 0x556f95c602f0 2024-03-19T00:47:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74bc004310 con 0x556f95c602f0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74a7fff640 1 -- 172.21.15.7:0/484445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f74bc037050 con 0x556f95c602f0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74d1658740 10 client.17436 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.786+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f95cdcdb0 con 0x556f95c602f0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74a7fff640 1 -- 172.21.15.7:0/484445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74bc004310 con 0x556f95c602f0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74a7fff640 1 -- 172.21.15.7:0/484445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f74bc01e410 con 0x556f95c602f0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74d1658740 10 client.17436 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74d1658740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74d1658740 20 client.17436 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74d1658740 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95cdd760 0x556f95cdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74d1658740 4 client.17436 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f95bd19f0 con 0x556f95cdd760 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74c7fff640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95cdd760 0x556f95cdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74c7fff640 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95cdd760 0x556f95cdfb80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef2eba5c0d07e92 server_cookie=19055038f396b6cb in_seq=0 out_seq=0 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74a7fff640 10 client.17436 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74a7fff640 1 -- 172.21.15.7:0/484445 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f95bd19f0 con 0x556f95cdd760 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.787+0000 7f74a7fff640 10 client.17436 handle_command_reply: tid=0 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 client.17436 shutdown 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 msgr2=0x556f95c71390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c71390 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0104e0 tx=0x7f74bc005dc0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 shutdown_connections 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f95c65e50 0x556f95c6f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f95c60ce0 0x556f95c718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x556f95cdd760 0x556f95cdfb80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 --2- 172.21.15.7:0/484445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f95c602f0 0x556f95c71390 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74a7fff640 0 client.17436 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 >> 172.21.15.7:0/484445 conn(0x556f95c5b860 msgr2=0x556f95c5eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 shutdown_connections 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 1 -- 172.21.15.7:0/484445 wait complete. 2024-03-19T00:47:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74d1658740 20 client.17436 trim_cache size 0 max 16384 2024-03-19T00:47:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:47:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74b80a2800 msgr2=0x7f74b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74b80a2800 0x7f74b80a4cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x556f95c61540 tx=0x7f74c0004a40 comp rx=0 tx=0).stop 2024-03-19T00:47:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 msgr2=0x7f74c8197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 0x7f74c8197ff0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f74b4005ab0 tx=0x7f74b40042f0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 shutdown_connections 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74b80a2800 0x7f74b80a4cc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74c8102e10 0x7f74c819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74c8100240 0x7f74c8197ff0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 --2- 172.21.15.7:0/1395559676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c80ff880 0x7f74c8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.788+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 >> 172.21.15.7:0/1395559676 conn(0x7f74c80fb400 msgr2=0x7f74c8105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.789+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 shutdown_connections 2024-03-19T00:47:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:06.789+0000 7f74cea6c640 1 -- 172.21.15.7:0/1395559676 wait complete. 2024-03-19T00:47:06.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4058705002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:47:07.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4058705002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:47:07.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4058705002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:47:07.800 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:47:07.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:47:07.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 -- 172.21.15.7:0/3047014256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f58100740 msgr2=0x7f1f58100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/3047014256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f58100740 0x7f1f58100b60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009900 tx=0x7f1f4802bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 -- 172.21.15.7:0/3047014256 shutdown_connections 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/3047014256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f581022f0 0x7f1f58108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/3047014256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f58101930 0x7f1f58101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/3047014256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f58100740 0x7f1f58100b60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 -- 172.21.15.7:0/3047014256 >> 172.21.15.7:0/3047014256 conn(0x7f1f580fbcb0 msgr2=0x7f1f580fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 -- 172.21.15.7:0/3047014256 shutdown_connections 2024-03-19T00:47:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 -- 172.21.15.7:0/3047014256 wait complete. 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.919+0000 7f1f5cf59640 1 Processor -- start 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 -- start start 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f58100740 0x7f1f58197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f58101930 0x7f1f58197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 0x7f1f5819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c260 con 0x7f1f581022f0 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c0e0 con 0x7f1f58100740 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5cf59640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c3e0 con 0x7f1f58101930 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 0x7f1f5819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 0x7f1f5819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34658/0 (socket says 172.21.15.7:34658) 2024-03-19T00:47:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 -- 172.21.15.7:0/2847737075 learned_addr learned my addr 172.21.15.7:0/2847737075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f58100740 0x7f1f58197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f58101930 0x7f1f58197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 -- 172.21.15.7:0/2847737075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f58101930 msgr2=0x7f1f58197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f58101930 0x7f1f58197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 -- 172.21.15.7:0/2847737075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f58100740 msgr2=0x7f1f58197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f58100740 0x7f1f58197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 -- 172.21.15.7:0/2847737075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5819f6b0 con 0x7f1f581022f0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f5659c640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f58100740 0x7f1f58197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f56d9d640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 0x7f1f5819f040 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000efd0 tx=0x7f1f5000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.920+0000 7f1f4f7fe640 1 -- 172.21.15.7:0/2847737075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50003b70 con 0x7f1f581022f0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.921+0000 7f1f4f7fe640 1 -- 172.21.15.7:0/2847737075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f50003d10 con 0x7f1f581022f0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.921+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5819f9a0 con 0x7f1f581022f0 2024-03-19T00:47:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.921+0000 7f1f4f7fe640 1 -- 172.21.15.7:0/2847737075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50010980 con 0x7f1f581022f0 2024-03-19T00:47:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.921+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5819fea0 con 0x7f1f581022f0 2024-03-19T00:47:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.923+0000 7f1f4f7fe640 1 -- 172.21.15.7:0/2847737075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1f50004020 con 0x7f1f581022f0 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 Processor -- start 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 -- start start 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d98abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d98fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d990310 0x562c6d9927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c6d8eb160 con 0x562c6d98b1a0 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562c6d893e10 con 0x562c6d98a7b0 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c6d8e7200 con 0x562c6d990310 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d98fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d98abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d98fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34672/0 (socket says 172.21.15.7:34672) 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 -- 172.21.15.7:0/1346026397 learned_addr learned my addr 172.21.15.7:0/1346026397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d990310 0x562c6d9927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f4f7fe640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f440a2800 0x7f1f440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 -- 172.21.15.7:0/1346026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d990310 msgr2=0x562c6d9927a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d990310 0x562c6d9927a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 -- 172.21.15.7:0/1346026397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 msgr2=0x562c6d98abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d98abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f56d9d640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f440a2800 0x7f1f440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.924+0000 7f1f5659c640 1 -- 172.21.15.7:0/1346026397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6d6c77e0 con 0x562c6d98b1a0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d98abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f4f7fe640 1 -- 172.21.15.7:0/2847737075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1f500cf280 con 0x7f1f581022f0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f5659c640 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d98fca0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009900 tx=0x7f1f480095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d55b680972c8d19 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f56d9d640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f440a2800 0x7f1f440a4cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f5009b7f0 tx=0x7f1f50086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f4cff9640 1 -- 172.21.15.7:0/1346026397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48045020 con 0x562c6d98b1a0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f4cff9640 1 -- 172.21.15.7:0/1346026397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f4802cc90 con 0x562c6d98b1a0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.925+0000 7f1f4cff9640 1 -- 172.21.15.7:0/1346026397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f480048c0 con 0x562c6d98b1a0 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- 172.21.15.7:0/1346026397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 msgr2=0x562c6d98fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d98fca0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009900 tx=0x7f1f480095f0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- 172.21.15.7:0/1346026397 shutdown_connections 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d990310 0x562c6d9927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d98fca0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/1346026397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d98abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- 172.21.15.7:0/1346026397 >> 172.21.15.7:0/1346026397 conn(0x562c6d981910 msgr2=0x562c6d981d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- 172.21.15.7:0/1346026397 shutdown_connections 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- 172.21.15.7:0/1346026397 wait complete. 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- start start 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 Processor -- start 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 0x562c6d994bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d9950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d99c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c6d993260 con 0x562c6d98b1a0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.926+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562c6d8a71a0 con 0x562c6d990310 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c6d9933e0 con 0x562c6d98a7b0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f5fc15740 10 client.0 _ll_register_callbacks cb 0x562c6d5fec60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f5659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d9950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 0x562c6d994bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 0x562c6d994bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:42164/0 (socket says 172.21.15.7:42164) 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 -- 172.21.15.7:0/116232294 learned_addr learned my addr 172.21.15.7:0/116232294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d99c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 -- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 msgr2=0x562c6d99c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d99c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 -- 172.21.15.7:0/116232294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 msgr2=0x562c6d9950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d9950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 -- 172.21.15.7:0/116232294 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6d727f70 con 0x562c6d98a7b0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 0x562c6d994bb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f40012da0 tx=0x7f1f40009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7f4a335bc3f4e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f3dffb640 1 -- 172.21.15.7:0/116232294 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4001a040 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f3dffb640 1 -- 172.21.15.7:0/116232294 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1f40011250 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.927+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4802cc90 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f3dffb640 1 -- 172.21.15.7:0/116232294 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f4002a050 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 10 client.27205 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c6d5bf170 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f3dffb640 1 -- 172.21.15.7:0/116232294 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f4802cc90 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f3dffb640 1 -- 172.21.15.7:0/116232294 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1f40004a30 con 0x562c6d98a7b0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 10 client.27205 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 10 client.27205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 20 client.27205 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6d9a0ab0 0x562c6d9a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 4 client.27205 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c6d8f86a0 con 0x562c6d9a0ab0 2024-03-19T00:47:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.928+0000 7f1f5659c640 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6d9a0ab0 0x562c6d9a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f3dffb640 10 client.27205 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5659c640 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6d9a0ab0 0x562c6d9a2f50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce907e63a67abd9 server_cookie=c36a0644bdde363c in_seq=0 out_seq=0 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f3dffb640 1 -- 172.21.15.7:0/116232294 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562c6d8f86a0 con 0x562c6d9a0ab0 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f3dffb640 10 client.27205 handle_command_reply: tid=0 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 client.27205 shutdown 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 msgr2=0x562c6d994bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 0x562c6d994bb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f40012da0 tx=0x7f1f40009ac0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 shutdown_connections 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d99c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6d9a0ab0 0x562c6d9a2f50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98b1a0 0x562c6d9950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/116232294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98a7b0 0x562c6d994bb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f3dffb640 0 client.27205 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 >> 172.21.15.7:0/116232294 conn(0x562c6d981910 msgr2=0x562c6d980b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.929+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 shutdown_connections 2024-03-19T00:47:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.930+0000 7f1f5fc15740 1 -- 172.21.15.7:0/116232294 wait complete. 2024-03-19T00:47:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.930+0000 7f1f5fc15740 20 client.27205 trim_cache size 0 max 16384 2024-03-19T00:47:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 Processor -- start 2024-03-19T00:47:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 -- start start 2024-03-19T00:47:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 0x562c6d9940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d985280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581aa790 con 0x562c6d98a7b0 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581a8060 con 0x562c6d990310 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581a5eb0 con 0x562c6d98b1a0 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 0x562c6d9940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 0x562c6d9940a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34686/0 (socket says 172.21.15.7:34686) 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f4ffff640 1 -- 172.21.15.7:0/813602778 learned_addr learned my addr 172.21.15.7:0/813602778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.947+0000 7f1f5659c640 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d985280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 -- 172.21.15.7:0/813602778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 msgr2=0x562c6d99c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 -- 172.21.15.7:0/813602778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 msgr2=0x562c6d985280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d985280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 -- 172.21.15.7:0/813602778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6d99f3e0 con 0x562c6d98a7b0 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 0x562c6d9940a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1f40012b00 tx=0x7f1f40005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444bb3b97adb2cc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f3ffff640 1 -- 172.21.15.7:0/813602778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40037070 con 0x562c6d98a7b0 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f3ffff640 1 -- 172.21.15.7:0/813602778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f40005d30 con 0x562c6d98a7b0 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f3ffff640 1 -- 172.21.15.7:0/813602778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40003850 con 0x562c6d98a7b0 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f5fc15740 1 -- 172.21.15.7:0/813602778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 msgr2=0x562c6d9940a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 0x562c6d9940a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1f40012b00 tx=0x7f1f40005b90 comp rx=0 tx=0).stop 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f5fc15740 1 -- 172.21.15.7:0/813602778 shutdown_connections 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d990310 0x562c6d985280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.948+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/813602778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d98a7b0 0x562c6d9940a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- 172.21.15.7:0/813602778 >> 172.21.15.7:0/813602778 conn(0x562c6d981910 msgr2=0x562c6d984b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:07.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- 172.21.15.7:0/813602778 shutdown_connections 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- 172.21.15.7:0/813602778 wait complete. 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- start start 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 Processor -- start 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d999ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 0x562c6d99b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581a6520 con 0x562c6d990310 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581a7990 con 0x562c6d98a7b0 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581a6bf0 con 0x562c6d98b1a0 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5fc15740 10 client.0 _ll_register_callbacks cb 0x7f1f40012540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 0x562c6d99b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 0x562c6d99b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34700/0 (socket says 172.21.15.7:34700) 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 -- 172.21.15.7:0/974654290 learned_addr learned my addr 172.21.15.7:0/974654290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f5659c640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d999ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 -- 172.21.15.7:0/974654290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 msgr2=0x562c6d99a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 -- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 msgr2=0x562c6d999ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.949+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d999ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f4ffff640 1 -- 172.21.15.7:0/974654290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6d99b7b0 con 0x562c6d990310 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f4ffff640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 0x562c6d99b1c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1f400057c0 tx=0x7f1f40009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2e943edb55728c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f3dffb640 1 -- 172.21.15.7:0/974654290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40037070 con 0x562c6d990310 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f3dffb640 1 -- 172.21.15.7:0/974654290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1f400051a0 con 0x562c6d990310 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f40005d30 con 0x562c6d990310 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f3dffb640 1 -- 172.21.15.7:0/974654290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f40003800 con 0x562c6d990310 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 10 client.17454 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c6da07380 con 0x562c6d990310 2024-03-19T00:47:07.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f3dffb640 1 -- 172.21.15.7:0/974654290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f40005d30 con 0x562c6d990310 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f3dffb640 1 -- 172.21.15.7:0/974654290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1f40010a00 con 0x562c6d990310 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 10 client.17454 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 20 client.17454 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6da07bc0 0x562c6da09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 4 client.17454 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.950+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c6d8f29f0 con 0x562c6da07bc0 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.951+0000 7f1f5659c640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6da07bc0 0x562c6da09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.951+0000 7f1f3dffb640 10 client.17454 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.951+0000 7f1f5659c640 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6da07bc0 0x562c6da09fb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2375ebdf97ce0d9 server_cookie=c34b2476676444eb in_seq=0 out_seq=0 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f3dffb640 1 -- 172.21.15.7:0/974654290 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c6d8f29f0 con 0x562c6da07bc0 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f3dffb640 10 client.17454 handle_command_reply: tid=0 2024-03-19T00:47:07.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 client.17454 shutdown 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 msgr2=0x562c6d99b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 0x562c6d99b1c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1f400057c0 tx=0x7f1f40009310 comp rx=0 tx=0).stop 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 shutdown_connections 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c6da07bc0 0x562c6da09fb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6d990310 0x562c6d99b1c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6d98b1a0 0x562c6d99a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f3dffb640 0 client.17454 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 --2- 172.21.15.7:0/974654290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6d98a7b0 0x562c6d999ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 >> 172.21.15.7:0/974654290 conn(0x562c6d981910 msgr2=0x562c6d985400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 shutdown_connections 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 1 -- 172.21.15.7:0/974654290 wait complete. 2024-03-19T00:47:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.952+0000 7f1f5fc15740 20 client.17454 trim_cache size 0 max 16384 2024-03-19T00:47:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:47:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:47:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "e1cd9885-9483-4625-8401-94f9b009a586", 2024-03-19T00:47:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:47:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f440a2800 msgr2=0x7f1f440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f440a2800 0x7f1f440a4cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f5009b7f0 tx=0x7f1f50086040 comp rx=0 tx=0).stop 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 msgr2=0x7f1f5819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 0x7f1f5819f040 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000efd0 tx=0x7f1f5000c6a0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 shutdown_connections 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1f440a2800 0x7f1f440a4cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f581022f0 0x7f1f5819f040 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f58101930 0x7f1f58197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 --2- 172.21.15.7:0/2847737075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f58100740 0x7f1f58197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 >> 172.21.15.7:0/2847737075 conn(0x7f1f580fbcb0 msgr2=0x7f1f58069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 shutdown_connections 2024-03-19T00:47:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:07.953+0000 7f1f5cf59640 1 -- 172.21.15.7:0/2847737075 wait complete. 2024-03-19T00:47:07.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:47:07.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:47:07.988 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:07 smithi007 ceph-mon[62854]: pgmap v626: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:07 smithi204 ceph-mon[66572]: pgmap v626: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 -- 172.21.15.7:0/3232425471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 msgr2=0x7fcc30104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 --2- 172.21.15.7:0/3232425471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30104710 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009900 tx=0x7fcc2002bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 -- 172.21.15.7:0/3232425471 shutdown_connections 2024-03-19T00:47:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 --2- 172.21.15.7:0/3232425471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 0x7fcc30109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 --2- 172.21.15.7:0/3232425471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 0x7fcc30107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 --2- 172.21.15.7:0/3232425471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30104710 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 -- 172.21.15.7:0/3232425471 >> 172.21.15.7:0/3232425471 conn(0x7fcc300fbcb0 msgr2=0x7fcc300fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 -- 172.21.15.7:0/3232425471 shutdown_connections 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.068+0000 7fcc35a80640 1 -- 172.21.15.7:0/3232425471 wait complete. 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 Processor -- start 2024-03-19T00:47:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 -- start start 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 0x7fcc30198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 0x7fcc3019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3010d390 con 0x7fcc30102320 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3010d210 con 0x7fcc30107580 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc35a80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3010d510 con 0x7fcc30104c50 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34710/0 (socket says 172.21.15.7:34710) 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/1949727298 learned_addr learned my addr 172.21.15.7:0/1949727298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 0x7fcc3019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2effd640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 0x7fcc30198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/1949727298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 msgr2=0x7fcc30198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 0x7fcc30198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/1949727298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 msgr2=0x7fcc3019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 0x7fcc3019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/1949727298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3019f720 con 0x7fcc30102320 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2effd640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 0x7fcc30198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.069+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 0x7fcc3019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.070+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30197b00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcc20006010 tx=0x7fcc20005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.070+0000 7fcc2cff9640 1 -- 172.21.15.7:0/1949727298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20043020 con 0x7fcc30102320 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.070+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3019f9b0 con 0x7fcc30102320 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.070+0000 7fcc2cff9640 1 -- 172.21.15.7:0/1949727298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc200043d0 con 0x7fcc30102320 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.070+0000 7fcc2cff9640 1 -- 172.21.15.7:0/1949727298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20003a80 con 0x7fcc30102320 2024-03-19T00:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.070+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3019ff50 con 0x7fcc30102320 2024-03-19T00:47:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc2cff9640 1 -- 172.21.15.7:0/1949727298 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fcc20046020 con 0x7fcc30102320 2024-03-19T00:47:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 Processor -- start 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 -- start start 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cb2f33e500 con 0x55cb2f3da440 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cb2f2e6be0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.072+0000 7fcc3865f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cb2f33a5a0 con 0x55cb2f3df5b0 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50604/0 (socket says 172.21.15.7:50604) 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 -- 172.21.15.7:0/2186969461 learned_addr learned my addr 172.21.15.7:0/2186969461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2effd640 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2cff9640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc180a2800 0x7fcc180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 -- 172.21.15.7:0/2186969461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3df5b0 msgr2=0x55cb2f3e1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 -- 172.21.15.7:0/2186969461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 msgr2=0x55cb2f3def40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3def40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 -- 172.21.15.7:0/2186969461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2f308000 con 0x55cb2f3d9a50 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2effd640 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2cff9640 1 -- 172.21.15.7:0/1949727298 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcc200f0fe0 con 0x7fcc30102320 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc180a2800 0x7fcc180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d9e70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc28011990 tx=0x7fcc28011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7728bba7c9cef909 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc15ffb640 1 -- 172.21.15.7:0/2186969461 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28017050 con 0x55cb2f3d9a50 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc180a2800 0x7fcc180a4cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc28023000 tx=0x7fcc28024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc15ffb640 1 -- 172.21.15.7:0/2186969461 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc2801ccf0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.073+0000 7fcc15ffb640 1 -- 172.21.15.7:0/2186969461 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28010a90 con 0x55cb2f3d9a50 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 -- 172.21.15.7:0/2186969461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 msgr2=0x55cb2f3d9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d9e70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc28011990 tx=0x7fcc28011e50 comp rx=0 tx=0).stop 2024-03-19T00:47:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 -- 172.21.15.7:0/2186969461 shutdown_connections 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2186969461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d9e70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 -- 172.21.15.7:0/2186969461 >> 172.21.15.7:0/2186969461 conn(0x55cb2f3d4fc0 msgr2=0x55cb2f3d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 -- 172.21.15.7:0/2186969461 shutdown_connections 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 -- 172.21.15.7:0/2186969461 wait complete. 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.074+0000 7fcc3865f740 1 -- start start 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 Processor -- start 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cb2f3e2560 con 0x55cb2f3d9a50 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cb2f2fa160 con 0x55cb2f3df5b0 2024-03-19T00:47:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cb2f3e26e0 con 0x55cb2f3da440 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc3865f740 10 client.0 _ll_register_callbacks cb 0x55cb2f336a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3e3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34720/0 (socket says 172.21.15.7:34720) 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/2536846377 learned_addr learned my addr 172.21.15.7:0/2536846377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2effd640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/2536846377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 msgr2=0x55cb2f3e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 msgr2=0x55cb2f3eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 -- 172.21.15.7:0/2536846377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2f11c7d0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2effd640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3e3f30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcc20000c00 tx=0x7fcc20005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1e3e5c4ea91570 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.075+0000 7fcc0effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc0effd640 1 -- 172.21.15.7:0/2536846377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc200bd3f0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc0effd640 1 -- 172.21.15.7:0/2536846377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcc200f3110 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2801ccf0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc0effd640 1 -- 172.21.15.7:0/2536846377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc200bcf80 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 10 client.17466 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb2f0523f0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc0effd640 1 -- 172.21.15.7:0/2536846377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc2801ccf0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc0effd640 1 -- 172.21.15.7:0/2536846377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcc200f4bf0 con 0x55cb2f3d9a50 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 10 client.17466 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 20 client.17466 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f3efcb0 0x55cb2f3f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 4 client.17466 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb2f34ba40 con 0x55cb2f3efcb0 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.076+0000 7fcc2effd640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f3efcb0 0x55cb2f3f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.077+0000 7fcc2effd640 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f3efcb0 0x55cb2f3f20e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c08e03dc57e1ce server_cookie=ceb228bfea5d6c02 in_seq=0 out_seq=0 2024-03-19T00:47:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.077+0000 7fcc0effd640 10 client.17466 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc0effd640 1 -- 172.21.15.7:0/2536846377 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cb2f34ba40 con 0x55cb2f3efcb0 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc0effd640 10 client.17466 handle_command_reply: tid=0 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 client.17466 shutdown 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 msgr2=0x55cb2f3e3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3e3f30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcc20000c00 tx=0x7fcc20005bd0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 shutdown_connections 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f3efcb0 0x55cb2f3f20e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 --2- 172.21.15.7:0/2536846377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3e3f30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc0effd640 0 client.17466 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 >> 172.21.15.7:0/2536846377 conn(0x55cb2f3d4fc0 msgr2=0x55cb2f3d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 shutdown_connections 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 1 -- 172.21.15.7:0/2536846377 wait complete. 2024-03-19T00:47:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.078+0000 7fcc3865f740 20 client.17466 trim_cache size 0 max 16384 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 Processor -- start 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 -- start start 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301acc10 con 0x55cb2f3da440 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301aa4e0 con 0x55cb2f3df5b0 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.095+0000 7fcc3865f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301a8330 con 0x55cb2f3d9a50 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34722/0 (socket says 172.21.15.7:34722) 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 -- 172.21.15.7:0/1017720893 learned_addr learned my addr 172.21.15.7:0/1017720893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 -- 172.21.15.7:0/1017720893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3d9a50 msgr2=0x55cb2f3d8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 -- 172.21.15.7:0/1017720893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 msgr2=0x55cb2f3f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 -- 172.21.15.7:0/1017720893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2f0120f0 con 0x55cb2f3da440 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc2effd640 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb400 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x55cb2f3d5f30 tx=0x7fcc24005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11439620135c189c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc157fa640 1 -- 172.21.15.7:0/1017720893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400fc20 con 0x55cb2f3da440 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc157fa640 1 -- 172.21.15.7:0/1017720893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc2400fdc0 con 0x55cb2f3da440 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.096+0000 7fcc157fa640 1 -- 172.21.15.7:0/1017720893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc240221b0 con 0x55cb2f3da440 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- 172.21.15.7:0/1017720893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 msgr2=0x55cb2f3eb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb400 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x55cb2f3d5f30 tx=0x7fcc24005f90 comp rx=0 tx=0).stop 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- 172.21.15.7:0/1017720893 shutdown_connections 2024-03-19T00:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb400 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1017720893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3d8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- 172.21.15.7:0/1017720893 >> 172.21.15.7:0/1017720893 conn(0x55cb2f3d4fc0 msgr2=0x55cb2f3d8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- 172.21.15.7:0/1017720893 shutdown_connections 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- 172.21.15.7:0/1017720893 wait complete. 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- start start 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 Processor -- start 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.097+0000 7fcc3865f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301a89a0 con 0x55cb2f3df5b0 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc3865f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301a9e10 con 0x55cb2f3d9a50 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc3865f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc301a9070 con 0x55cb2f3da440 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc3865f740 10 client.0 _ll_register_callbacks cb 0x7fcc20009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34732/0 (socket says 172.21.15.7:34732) 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 -- 172.21.15.7:0/1034776433 learned_addr learned my addr 172.21.15.7:0/1034776433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2effd640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 -- 172.21.15.7:0/1034776433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 msgr2=0x55cb2f3eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 -- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 msgr2=0x55cb2f3eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 -- 172.21.15.7:0/1034776433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2f3e98c0 con 0x55cb2f3df5b0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc2ffff640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e9250 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcc28011960 tx=0x7fcc28004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fcedfb88e74510 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc0effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc0effd640 1 -- 172.21.15.7:0/1034776433 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28039060 con 0x55cb2f3df5b0 2024-03-19T00:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc0effd640 1 -- 172.21.15.7:0/1034776433 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcc28034480 con 0x55cb2f3df5b0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2400fdc0 con 0x55cb2f3df5b0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc0effd640 1 -- 172.21.15.7:0/1034776433 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc28017020 con 0x55cb2f3df5b0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc3865f740 10 client.17472 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.098+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb2f456a30 con 0x55cb2f3df5b0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc0effd640 1 -- 172.21.15.7:0/1034776433 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc2400fdc0 con 0x55cb2f3df5b0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc0effd640 1 -- 172.21.15.7:0/1034776433 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcc28005040 con 0x55cb2f3df5b0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 10 client.17472 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 20 client.17472 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f4572a0 0x55cb2f4596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 4 client.17472 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb2f345d90 con 0x55cb2f4572a0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f4572a0 0x55cb2f4596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc2f7fe640 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f4572a0 0x55cb2f4596c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48584bd61855beaa server_cookie=c96a235f88cb7732 in_seq=0 out_seq=0 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc0effd640 10 client.17472 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc0effd640 1 -- 172.21.15.7:0/1034776433 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cb2f345d90 con 0x55cb2f4572a0 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc0effd640 10 client.17472 handle_command_reply: tid=0 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.099+0000 7fcc3865f740 1 client.17472 shutdown 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 msgr2=0x55cb2f3e9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e9250 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcc28011960 tx=0x7fcc28004c30 comp rx=0 tx=0).stop 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 shutdown_connections 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55cb2f4572a0 0x55cb2f4596c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb2f3df5b0 0x55cb2f3e9250 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2f3da440 0x55cb2f3eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc0effd640 0 client.17472 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 --2- 172.21.15.7:0/1034776433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cb2f3d9a50 0x55cb2f3eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 >> 172.21.15.7:0/1034776433 conn(0x55cb2f3d4fc0 msgr2=0x55cb2f3d8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 shutdown_connections 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 1 -- 172.21.15.7:0/1034776433 wait complete. 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc3865f740 20 client.17472 trim_cache size 0 max 16384 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: "e1cd9885-9483-4625-8401-94f9b009a586": { 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "e1cd9885-9483-4625-8401-94f9b009a586", 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:47:08.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:47:08.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:47:08.109 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc180a2800 msgr2=0x7fcc180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc180a2800 0x7fcc180a4cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc28023000 tx=0x7fcc28024000 comp rx=0 tx=0).stop 2024-03-19T00:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 msgr2=0x7fcc30197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30197b00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcc20006010 tx=0x7fcc20005e40 comp rx=0 tx=0).stop 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 shutdown_connections 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcc180a2800 0x7fcc180a4cc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30107580 0x7fcc3019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc30104c50 0x7fcc30198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 --2- 172.21.15.7:0/1949727298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc30102320 0x7fcc30197b00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 >> 172.21.15.7:0/1949727298 conn(0x7fcc300fbcb0 msgr2=0x7fcc30102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 shutdown_connections 2024-03-19T00:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:08.100+0000 7fcc35a80640 1 -- 172.21.15.7:0/1949727298 wait complete. 2024-03-19T00:47:08.110 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:07 smithi044 ceph-mon[66366]: pgmap v626: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:08.111 INFO:tasks.cephfs.filesystem:scrub status for tag:e1cd9885-9483-4625-8401-94f9b009a586 - {'path': '/', 'tag': 'e1cd9885-9483-4625-8401-94f9b009a586', 'options': 'recursive,force'} 2024-03-19T00:47:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:08 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:47:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:08 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:08 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:47:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:08 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:47:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:08 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:08 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:08 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:08 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:08 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:08 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:08 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:08 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:08 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:47:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:08 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:08 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:47:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:08 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:47:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:08 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:09.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:08 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:09 smithi204 ceph-mon[66572]: pgmap v627: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:09 smithi044 ceph-mon[66366]: pgmap v627: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:09 smithi007 ceph-mon[62854]: pgmap v627: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:12.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:11 smithi204 ceph-mon[66572]: pgmap v628: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:11 smithi044 ceph-mon[66366]: pgmap v628: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:12.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:11 smithi007 ceph-mon[62854]: pgmap v628: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:13 smithi204 ceph-mon[66572]: pgmap v629: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2024-03-19T00:47:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:13 smithi044 ceph-mon[66366]: pgmap v629: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2024-03-19T00:47:14.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:13 smithi007 ceph-mon[62854]: pgmap v629: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2024-03-19T00:47:16.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:15 smithi204 ceph-mon[66572]: pgmap v630: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:15 smithi044 ceph-mon[66366]: pgmap v630: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:16.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:15 smithi007 ceph-mon[62854]: pgmap v630: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:17.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:16 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:47:17.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:16 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:47:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:16 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:47:17.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:16 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:47:17.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:16 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:47:17.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:16 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:47:18.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:17 smithi204 ceph-mon[66572]: pgmap v631: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:17 smithi044 ceph-mon[66366]: pgmap v631: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:18.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:17 smithi007 ceph-mon[62854]: pgmap v631: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:20.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:19 smithi204 ceph-mon[66572]: pgmap v632: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:19 smithi044 ceph-mon[66366]: pgmap v632: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:20.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:19 smithi007 ceph-mon[62854]: pgmap v632: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:22.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:21 smithi204 ceph-mon[66572]: pgmap v633: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:22.017 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:21 smithi007 ceph-mon[62854]: pgmap v633: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:21 smithi044 ceph-mon[66366]: pgmap v633: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:24.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:23 smithi204 ceph-mon[66572]: pgmap v634: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:47:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:23 smithi044 ceph-mon[66366]: pgmap v634: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:47:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:23 smithi007 ceph-mon[62854]: pgmap v634: 129 pgs: 129 active+clean; 2.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:47:26.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:25 smithi204 ceph-mon[66572]: pgmap v635: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:47:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:25 smithi044 ceph-mon[66366]: pgmap v635: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:47:26.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:25 smithi007 ceph-mon[62854]: pgmap v635: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-03-19T00:47:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:27 smithi044 ceph-mon[66366]: pgmap v636: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:28.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:27 smithi007 ceph-mon[62854]: pgmap v636: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:27 smithi204 ceph-mon[66572]: pgmap v636: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:29 smithi044 ceph-mon[66366]: pgmap v637: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:30.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:29 smithi007 ceph-mon[62854]: pgmap v637: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:29 smithi204 ceph-mon[66572]: pgmap v637: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:32.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:31 smithi044 ceph-mon[66366]: pgmap v638: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:32.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:31 smithi007 ceph-mon[62854]: pgmap v638: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:32.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:31 smithi204 ceph-mon[66572]: pgmap v638: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:437/600: open fd = 3 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:438/600: open fd = 3 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.583 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:439/600: open fd = 3 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:440/600: open fd = 3 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:441/600: open fd = 3 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:442/600: open fd = 3 2024-03-19T00:47:32.584 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:443/600: open fd = 3 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:444/600: open fd = 3 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:445/600: open fd = 3 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:446/600: open fd = 3 2024-03-19T00:47:32.585 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:447/600: open fd = 3 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:448/600: open fd = 3 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:449/600: open fd = 3 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.586 INFO:tasks.workunit.client.0.smithi007.stdout:450/600: open fd = 3 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:451/600: open fd = 3 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:452/600: open fd = 3 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:453/600: open fd = 3 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.587 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:454/600: open fd = 3 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:455/600: open fd = 3 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:456/600: open fd = 3 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:457/600: open fd = 3 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.588 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:458/600: open fd = 3 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:459/600: open fd = 3 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:460/600: open fd = 3 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:461/600: open fd = 3 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.589 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:462/600: open fd = 3 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:463/600: open fd = 3 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:464/600: open fd = 3 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:465/600: open fd = 3 2024-03-19T00:47:32.590 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:466/600: open fd = 3 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:467/600: open fd = 3 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:468/600: open fd = 3 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:469/600: open fd = 3 2024-03-19T00:47:32.591 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:470/600: open fd = 3 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:471/600: open fd = 3 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:472/600: open fd = 3 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:473/600: open fd = 3 2024-03-19T00:47:32.592 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:474/600: open fd = 3 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:475/600: open fd = 3 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:476/600: open fd = 3 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.593 INFO:tasks.workunit.client.0.smithi007.stdout:477/600: open fd = 3 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:478/600: open fd = 3 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:479/600: open fd = 3 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:480/600: open fd = 3 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.594 INFO:tasks.workunit.client.0.smithi007.stdout:481/600: open fd = 3 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:482/600: open fd = 3 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:483/600: open fd = 3 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:484/600: open fd = 3 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.595 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:485/600: open fd = 3 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:486/600: open fd = 3 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:487/600: open fd = 3 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:488/600: open fd = 3 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.596 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:489/600: open fd = 3 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:490/600: open fd = 3 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:491/600: open fd = 3 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:492/600: open fd = 3 2024-03-19T00:47:32.597 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:493/600: open fd = 3 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:494/600: open fd = 3 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:495/600: open fd = 3 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.598 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:496/600: open fd = 3 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:497/600: open fd = 3 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:498/600: open fd = 3 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:47:32.599 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:47:34.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:33 smithi044 ceph-mon[66366]: pgmap v639: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:34.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:33 smithi007 ceph-mon[62854]: pgmap v639: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:33 smithi204 ceph-mon[66572]: pgmap v639: 129 pgs: 129 active+clean; 2.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:36.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:35 smithi044 ceph-mon[66366]: pgmap v640: 129 pgs: 129 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:47:36.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:35 smithi007 ceph-mon[62854]: pgmap v640: 129 pgs: 129 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:47:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:35 smithi204 ceph-mon[66572]: pgmap v640: 129 pgs: 129 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:47:38.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:37 smithi044 ceph-mon[66366]: pgmap v641: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:38.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:47:38.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:47:38.141 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:37 smithi007 ceph-mon[62854]: pgmap v641: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:38.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 -- 172.21.15.7:0/2520909158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 msgr2=0x7fd994074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 --2- 172.21.15.7:0/2520909158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994074660 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd980009900 tx=0x7fd98002bcd0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 -- 172.21.15.7:0/2520909158 shutdown_connections 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 --2- 172.21.15.7:0/2520909158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9941096e0 0x7fd99410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 --2- 172.21.15.7:0/2520909158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994074660 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 --2- 172.21.15.7:0/2520909158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940757e0 0x7fd994073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 -- 172.21.15.7:0/2520909158 >> 172.21.15.7:0/2520909158 conn(0x7fd9940fcda0 msgr2=0x7fd9940ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 -- 172.21.15.7:0/2520909158 shutdown_connections 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.229+0000 7fd99afe6640 1 -- 172.21.15.7:0/2520909158 wait complete. 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 Processor -- start 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 -- start start 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9940757e0 0x7fd99419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9941096e0 0x7fd9941a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410f560 con 0x7fd994074200 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410f3e0 con 0x7fd9940757e0 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd99afe6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410f6e0 con 0x7fd9941096e0 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50634/0 (socket says 172.21.15.7:50634) 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 -- 172.21.15.7:0/999255873 learned_addr learned my addr 172.21.15.7:0/999255873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd993fff640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9940757e0 0x7fd99419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd999583640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9941096e0 0x7fd9941a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 -- 172.21.15.7:0/999255873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9941096e0 msgr2=0x7fd9941a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9941096e0 0x7fd9941a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 -- 172.21.15.7:0/999255873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9940757e0 msgr2=0x7fd99419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9940757e0 0x7fd99419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd998d82640 1 -- 172.21.15.7:0/999255873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9941a1900 con 0x7fd994074200 2024-03-19T00:47:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.230+0000 7fd993fff640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9940757e0 0x7fd99419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.231+0000 7fd998d82640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994199cc0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd98800e9f0 tx=0x7fd98800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.231+0000 7fd991ffb640 1 -- 172.21.15.7:0/999255873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98800ccc0 con 0x7fd994074200 2024-03-19T00:47:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.231+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9941a1bf0 con 0x7fd994074200 2024-03-19T00:47:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.231+0000 7fd991ffb640 1 -- 172.21.15.7:0/999255873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd98800ce60 con 0x7fd994074200 2024-03-19T00:47:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.231+0000 7fd991ffb640 1 -- 172.21.15.7:0/999255873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988010430 con 0x7fd994074200 2024-03-19T00:47:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.231+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a2110 con 0x7fd994074200 2024-03-19T00:47:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.233+0000 7fd991ffb640 1 -- 172.21.15.7:0/999255873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fd988005020 con 0x7fd994074200 2024-03-19T00:47:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 Processor -- start 2024-03-19T00:47:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 -- start start 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd991ffb640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9840a2800 0x7fd9840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 0x55b2b62f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b62faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd999583640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9840a2800 0x7fd9840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b2b625a500 con 0x55b2b62f5a50 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b2b6202be0 con 0x55b2b62f6440 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b2b62565a0 con 0x55b2b62fb5b0 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd991ffb640 1 -- 172.21.15.7:0/999255873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd9880cf1b0 con 0x7fd994074200 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd998d82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 0x55b2b62f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd998d82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 0x55b2b62f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50648/0 (socket says 172.21.15.7:50648) 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd998d82640 1 -- 172.21.15.7:0/3869802675 learned_addr learned my addr 172.21.15.7:0/3869802675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd993fff640 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b62faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.234+0000 7fd999583640 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd998d82640 1 -- 172.21.15.7:0/3869802675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 msgr2=0x55b2b62fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd998d82640 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd998d82640 1 -- 172.21.15.7:0/3869802675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 msgr2=0x55b2b62faf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd998d82640 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b62faf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd998d82640 1 -- 172.21.15.7:0/3869802675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b6224000 con 0x55b2b62f5a50 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd993fff640 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b62faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd999583640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9840a2800 0x7fd9840a4cc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c013b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd998d82640 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 0x55b2b62f5e70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd9880a3a60 tx=0x7fd98809b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba42d5fa0b0bff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd97effd640 1 -- 172.21.15.7:0/3869802675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98809f940 con 0x55b2b62f5a50 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd97effd640 1 -- 172.21.15.7:0/3869802675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9880a3f90 con 0x55b2b62f5a50 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.235+0000 7fd97effd640 1 -- 172.21.15.7:0/3869802675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988097e60 con 0x55b2b62f5a50 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- 172.21.15.7:0/3869802675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 msgr2=0x55b2b62f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 0x55b2b62f5e70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd9880a3a60 tx=0x7fd98809b150 comp rx=0 tx=0).stop 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- 172.21.15.7:0/3869802675 shutdown_connections 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b62faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/3869802675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f5a50 0x55b2b62f5e70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- 172.21.15.7:0/3869802675 >> 172.21.15.7:0/3869802675 conn(0x55b2b62f0fc0 msgr2=0x55b2b62f13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- 172.21.15.7:0/3869802675 shutdown_connections 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- 172.21.15.7:0/3869802675 wait complete. 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- start start 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 Processor -- start 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62f5a50 0x55b2b62ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b6300410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 0x55b2b6307450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b2b62fe560 con 0x55b2b62fb5b0 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b2b6216160 con 0x55b2b62f6440 2024-03-19T00:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.236+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b2b62fe6e0 con 0x55b2b62f5a50 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd99dbd3740 10 client.0 _ll_register_callbacks cb 0x55b2b6252a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 0x55b2b6307450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 0x55b2b6307450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50658/0 (socket says 172.21.15.7:50658) 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd993fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b6300410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 -- 172.21.15.7:0/2636754787 learned_addr learned my addr 172.21.15.7:0/2636754787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd998d82640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62f5a50 0x55b2b62ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 -- 172.21.15.7:0/2636754787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62f5a50 msgr2=0x55b2b62ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62f5a50 0x55b2b62ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 -- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 msgr2=0x55b2b6300410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b6300410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 -- 172.21.15.7:0/2636754787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b60387d0 con 0x55b2b62fb5b0 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd998d82640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62f5a50 0x55b2b62ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd999583640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 0x55b2b6307450 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00fd90 tx=0x7fd98c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f875212ec1da7594 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd973fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd973fff640 1 -- 172.21.15.7:0/2636754787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c024030 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd973fff640 1 -- 172.21.15.7:0/2636754787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd98c004540 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9880a3f90 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd973fff640 1 -- 172.21.15.7:0/2636754787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd98c031050 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.237+0000 7fd99dbd3740 10 client.17484 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2b5f6e3f0 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd973fff640 1 -- 172.21.15.7:0/2636754787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9880a3f90 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd973fff640 1 -- 172.21.15.7:0/2636754787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd98c01f070 con 0x55b2b62fb5b0 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 10 client.17484 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 20 client.17484 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b630bd80 0x55b2b630e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 4 client.17484 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b6267a40 con 0x55b2b630bd80 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.238+0000 7fd998d82640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b630bd80 0x55b2b630e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.239+0000 7fd998d82640 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b630bd80 0x55b2b630e220 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e2014b03fe7c46 server_cookie=9fc7a672d9e565ae in_seq=0 out_seq=0 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.239+0000 7fd973fff640 10 client.17484 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd973fff640 1 -- 172.21.15.7:0/2636754787 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b2b6267a40 con 0x55b2b630bd80 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd973fff640 10 client.17484 handle_command_reply: tid=0 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 client.17484 shutdown 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 msgr2=0x55b2b6307450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 0x55b2b6307450 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00fd90 tx=0x7fd98c004ab0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 shutdown_connections 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b630bd80 0x55b2b630e220 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62fb5b0 0x55b2b6307450 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f6440 0x55b2b6300410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd973fff640 0 client.17484 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/2636754787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62f5a50 0x55b2b62ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 >> 172.21.15.7:0/2636754787 conn(0x55b2b62f0fc0 msgr2=0x55b2b62f01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 shutdown_connections 2024-03-19T00:47:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 1 -- 172.21.15.7:0/2636754787 wait complete. 2024-03-19T00:47:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.240+0000 7fd99dbd3740 20 client.17484 trim_cache size 0 max 16384 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 Processor -- start 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 -- start start 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b62f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b6307410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62f4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aed30 con 0x55b2b62f6440 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ac600 con 0x55b2b62f5a50 2024-03-19T00:47:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aa450 con 0x55b2b62fb5b0 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b6307410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b6307410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50664/0 (socket says 172.21.15.7:50664) 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 -- 172.21.15.7:0/1154848442 learned_addr learned my addr 172.21.15.7:0/1154848442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd998d82640 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b62f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd999583640 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 -- 172.21.15.7:0/1154848442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 msgr2=0x55b2b62f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 -- 172.21.15.7:0/1154848442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 msgr2=0x55b2b62f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b62f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 -- 172.21.15.7:0/1154848442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b62f7f80 con 0x55b2b62f6440 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd998d82640 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b62f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.258+0000 7fd993fff640 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b6307410 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd98002c210 tx=0x7fd980004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9156884fcac6b7c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd97e7fc640 1 -- 172.21.15.7:0/1154848442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980004270 con 0x55b2b62f6440 2024-03-19T00:47:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd97e7fc640 1 -- 172.21.15.7:0/1154848442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd980044070 con 0x55b2b62f6440 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd97e7fc640 1 -- 172.21.15.7:0/1154848442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980003cc0 con 0x55b2b62f6440 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1154848442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 msgr2=0x55b2b6307410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b6307410 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd98002c210 tx=0x7fd980004040 comp rx=0 tx=0).stop 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1154848442 shutdown_connections 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b62f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b6307410 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1154848442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b62f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1154848442 >> 172.21.15.7:0/1154848442 conn(0x55b2b62f0fc0 msgr2=0x55b2b62f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1154848442 shutdown_connections 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.259+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1154848442 wait complete. 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 -- start start 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 Processor -- start 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b6304c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b63051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b6306180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aaac0 con 0x55b2b62f6440 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941abf30 con 0x55b2b62f5a50 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ab190 con 0x55b2b62fb5b0 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd99dbd3740 10 client.0 _ll_register_callbacks cb 0x7fd98c00fa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b63051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd998d82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b6304c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b63051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50676/0 (socket says 172.21.15.7:50676) 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 -- 172.21.15.7:0/1224517101 learned_addr learned my addr 172.21.15.7:0/1224517101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd999583640 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b6306180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 -- 172.21.15.7:0/1224517101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 msgr2=0x55b2b6306180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b6306180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 -- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 msgr2=0x55b2b6304c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b6304c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.260+0000 7fd993fff640 1 -- 172.21.15.7:0/1224517101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b5f2e0f0 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd993fff640 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b63051c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd980005ab0 tx=0x7fd98002cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2101e619034bf362 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd973fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd973fff640 1 -- 172.21.15.7:0/1224517101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980003c40 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd980044070 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd973fff640 1 -- 172.21.15.7:0/1224517101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd980003de0 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd973fff640 1 -- 172.21.15.7:0/1224517101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd98005c050 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 10 client.17490 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2b6372120 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd973fff640 1 -- 172.21.15.7:0/1224517101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd980044070 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd973fff640 1 -- 172.21.15.7:0/1224517101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd980034440 con 0x55b2b62f6440 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 10 client.17490 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 20 client.17490 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b6372a50 0x55b2b6374e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 4 client.17490 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.261+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2b6261d90 con 0x55b2b6372a50 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd998d82640 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b6372a50 0x55b2b6374e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd998d82640 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b6372a50 0x55b2b6374e70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e71ecb79915108 server_cookie=3b843c7ddf2eb6c5 in_seq=0 out_seq=0 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd973fff640 10 client.17490 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd973fff640 1 -- 172.21.15.7:0/1224517101 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b2b6261d90 con 0x55b2b6372a50 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd973fff640 10 client.17490 handle_command_reply: tid=0 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 client.17490 shutdown 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 msgr2=0x55b2b63051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b63051c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd980005ab0 tx=0x7fd98002cbb0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 shutdown_connections 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2b62fb5b0 0x55b2b6306180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b2b6372a50 0x55b2b6374e70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2b62f6440 0x55b2b63051c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 --2- 172.21.15.7:0/1224517101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2b62f5a50 0x55b2b6304c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd973fff640 0 client.17490 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 >> 172.21.15.7:0/1224517101 conn(0x55b2b62f0fc0 msgr2=0x55b2b62f4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 shutdown_connections 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.262+0000 7fd99dbd3740 1 -- 172.21.15.7:0/1224517101 wait complete. 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99dbd3740 20 client.17490 trim_cache size 0 max 16384 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:47:38.270 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:47:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9840a2800 msgr2=0x7fd9840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9840a2800 0x7fd9840a4cc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c013b80 comp rx=0 tx=0).stop 2024-03-19T00:47:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 msgr2=0x7fd994199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994199cc0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd98800e9f0 tx=0x7fd98800edf0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 shutdown_connections 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd9840a2800 0x7fd9840a4cc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9941096e0 0x7fd9941a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9940757e0 0x7fd99419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 --2- 172.21.15.7:0/999255873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd994074200 0x7fd994199cc0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 >> 172.21.15.7:0/999255873 conn(0x7fd9940fcda0 msgr2=0x7fd994105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 shutdown_connections 2024-03-19T00:47:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.263+0000 7fd99afe6640 1 -- 172.21.15.7:0/999255873 wait complete. 2024-03-19T00:47:38.272 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:37 smithi204 ceph-mon[66572]: pgmap v641: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:38.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:47:38.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 -- 172.21.15.7:0/1348254697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 msgr2=0x7f5434100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 --2- 172.21.15.7:0/1348254697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f5434100f50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5420009900 tx=0x7f542002bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 -- 172.21.15.7:0/1348254697 shutdown_connections 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 --2- 172.21.15.7:0/1348254697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434101490 0x7f5434101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 --2- 172.21.15.7:0/1348254697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f5434100f50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 --2- 172.21.15.7:0/1348254697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5434100190 0x7f54341005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 -- 172.21.15.7:0/1348254697 >> 172.21.15.7:0/1348254697 conn(0x7f54340fbd50 msgr2=0x7f54340fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 -- 172.21.15.7:0/1348254697 shutdown_connections 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.417+0000 7f543ae4d640 1 -- 172.21.15.7:0/1348254697 wait complete. 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 Processor -- start 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 -- start start 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434100190 0x7f5434199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f543419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5434101490 0x7f54341a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f543410f540 con 0x7f5434100af0 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f543410f3c0 con 0x7f5434100190 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f543ae4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f543410f6c0 con 0x7f5434101490 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f543419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f543419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50692/0 (socket says 172.21.15.7:50692) 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 -- 172.21.15.7:0/2992673233 learned_addr learned my addr 172.21.15.7:0/2992673233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5438be9640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434100190 0x7f5434199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f54393ea640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5434101490 0x7f54341a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 -- 172.21.15.7:0/2992673233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5434101490 msgr2=0x7f54341a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5434101490 0x7f54341a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 -- 172.21.15.7:0/2992673233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434100190 msgr2=0x7f5434199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434100190 0x7f5434199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5433fff640 1 -- 172.21.15.7:0/2992673233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54341a18b0 con 0x7f5434100af0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.418+0000 7f5438be9640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434100190 0x7f5434199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.419+0000 7f5433fff640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f543419a1b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5420006010 tx=0x7f5420005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.419+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420043020 con 0x7f5434100af0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.419+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54341a1b40 con 0x7f5434100af0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.419+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54200043d0 con 0x7f5434100af0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.419+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420003a80 con 0x7f5434100af0 2024-03-19T00:47:38.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.419+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54341a2090 con 0x7f5434100af0 2024-03-19T00:47:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.420+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543410b160 con 0x7f5434100af0 2024-03-19T00:47:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.421+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5420046020 con 0x7f5434100af0 2024-03-19T00:47:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.421+0000 7f5431ffb640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54240a2800 0x7f54240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.422+0000 7f5438be9640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54240a2800 0x7f54240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.422+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f54200f0ff0 con 0x7f5434100af0 2024-03-19T00:47:38.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.422+0000 7f5438be9640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54240a2800 0x7f54240a4cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f543419ab40 tx=0x7f5428005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.426+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f542004d020 con 0x7f5434100af0 2024-03-19T00:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.648+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5434062420 con 0x7f5434100af0 2024-03-19T00:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.649+0000 7f5431ffb640 1 -- 172.21.15.7:0/2992673233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f54200bdcd0 con 0x7f5434100af0 2024-03-19T00:47:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:47:38.650 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54240a2800 msgr2=0x7f54240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54240a2800 0x7f54240a4cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f543419ab40 tx=0x7f5428005f90 comp rx=0 tx=0).stop 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 msgr2=0x7f543419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f543419a1b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5420006010 tx=0x7f5420005e40 comp rx=0 tx=0).stop 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 shutdown_connections 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f54240a2800 0x7f54240a4cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5434101490 0x7f54341a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5434100af0 0x7f543419a1b0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 --2- 172.21.15.7:0/2992673233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5434100190 0x7f5434199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 >> 172.21.15.7:0/2992673233 conn(0x7f54340fbd50 msgr2=0x7f54340fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 shutdown_connections 2024-03-19T00:47:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.651+0000 7f543ae4d640 1 -- 172.21.15.7:0/2992673233 wait complete. 2024-03-19T00:47:38.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 -- 172.21.15.7:0/2828886380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 msgr2=0x7f3da4101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2828886380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4101da0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009900 tx=0x7f3d9802bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 -- 172.21.15.7:0/2828886380 shutdown_connections 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2828886380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da41022e0 0x7f3da4108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2828886380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4101da0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2828886380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4100730 0x7f3da4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 -- 172.21.15.7:0/2828886380 >> 172.21.15.7:0/2828886380 conn(0x7f3da40fbc80 msgr2=0x7f3da40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 -- 172.21.15.7:0/2828886380 shutdown_connections 2024-03-19T00:47:38.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 -- 172.21.15.7:0/2828886380 wait complete. 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.766+0000 7f3da92ac640 1 Processor -- start 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 -- start start 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4100730 0x7f3da4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da41022e0 0x7f3da419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da410d380 con 0x7f3da4101920 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3da410d200 con 0x7f3da4100730 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da92ac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3da410d500 con 0x7f3da41022e0 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50720/0 (socket says 172.21.15.7:50720) 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 -- 172.21.15.7:0/2746548632 learned_addr learned my addr 172.21.15.7:0/2746548632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da37fe640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da41022e0 0x7f3da419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4100730 0x7f3da4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 -- 172.21.15.7:0/2746548632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da41022e0 msgr2=0x7f3da419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da41022e0 0x7f3da419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 -- 172.21.15.7:0/2746548632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4100730 msgr2=0x7f3da4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4100730 0x7f3da4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 -- 172.21.15.7:0/2746548632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da419f640 con 0x7f3da4101920 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4197f60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d98005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.767+0000 7f3d8bfff640 1 -- 172.21.15.7:0/2746548632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98043020 con 0x7f3da4101920 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.768+0000 7f3d8bfff640 1 -- 172.21.15.7:0/2746548632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d980043d0 con 0x7f3da4101920 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.768+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da419f8d0 con 0x7f3da4101920 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.768+0000 7f3d8bfff640 1 -- 172.21.15.7:0/2746548632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98003b80 con 0x7f3da4101920 2024-03-19T00:47:38.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.768+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da419fe20 con 0x7f3da4101920 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.770+0000 7f3d8bfff640 1 -- 172.21.15.7:0/2746548632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f3d98046020 con 0x7f3da4101920 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.770+0000 7f3dabf72740 1 Processor -- start 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.770+0000 7f3dabf72740 1 -- start start 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x559550001710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x5595500067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x5595500092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3dabf72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55954ff654b0 con 0x5595500012f0 2024-03-19T00:47:38.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3dabf72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55954ff0dd70 con 0x559550006e50 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3dabf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55954ff61550 con 0x559550001ce0 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x559550001710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x559550001710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50736/0 (socket says 172.21.15.7:50736) 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da37fe640 1 -- 172.21.15.7:0/2377641897 learned_addr learned my addr 172.21.15.7:0/2377641897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x5595500067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x5595500092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3d8bfff640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3d8c0a2800 0x7f3d8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2377641897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 msgr2=0x5595500092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x5595500092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2377641897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 msgr2=0x559550001710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x559550001710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2377641897 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954ff30210 con 0x559550001ce0 2024-03-19T00:47:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da37fe640 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x559550001710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da37fe640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3d8c0a2800 0x7f3d8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3d8bfff640 1 -- 172.21.15.7:0/2746548632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3d980f0ff0 con 0x7f3da4101920 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x5595500067e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009bf0 tx=0x7f3d94011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc63842de16b9452 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.771+0000 7f3d897fa640 1 -- 172.21.15.7:0/2377641897 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94009da0 con 0x559550001ce0 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3da37fe640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3d8c0a2800 0x7f3d8c0a4cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x559550002540 tx=0x7f3d9c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3d897fa640 1 -- 172.21.15.7:0/2377641897 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d94005020 con 0x559550001ce0 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3d897fa640 1 -- 172.21.15.7:0/2377641897 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94004a00 con 0x559550001ce0 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 -- 172.21.15.7:0/2377641897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 msgr2=0x5595500067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x5595500067e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009bf0 tx=0x7f3d94011810 comp rx=0 tx=0).stop 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 -- 172.21.15.7:0/2377641897 shutdown_connections 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x5595500092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x5595500067e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2377641897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x559550001710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 -- 172.21.15.7:0/2377641897 >> 172.21.15.7:0/2377641897 conn(0x55954fffc860 msgr2=0x55954fffcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 -- 172.21.15.7:0/2377641897 shutdown_connections 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.772+0000 7f3dabf72740 1 -- 172.21.15.7:0/2377641897 wait complete. 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 -- start start 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 Processor -- start 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x55955000b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x55955000bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550012d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559550009e00 con 0x5595500012f0 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55954ff21310 con 0x559550006e50 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559550009f80 con 0x559550001ce0 2024-03-19T00:47:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3dabf72740 10 client.0 _ll_register_callbacks cb 0x55954ff152d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x55955000b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x55955000b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50750/0 (socket says 172.21.15.7:50750) 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2294845046 learned_addr learned my addr 172.21.15.7:0/2294845046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x55955000bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da37fe640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550012d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 msgr2=0x55955000bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x55955000bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2294845046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 msgr2=0x559550012d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550012d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2294845046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954fc793f0 con 0x5595500012f0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x55955000b820 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3d9401b040 tx=0x7f3d940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecb6c32ff2cc793 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.773+0000 7f3d86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3d86ff5640 1 -- 172.21.15.7:0/2294845046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94029020 con 0x5595500012f0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3d86ff5640 1 -- 172.21.15.7:0/2294845046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3d94004cf0 con 0x5595500012f0 2024-03-19T00:47:38.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d94005020 con 0x5595500012f0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3d86ff5640 1 -- 172.21.15.7:0/2294845046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d94034050 con 0x5595500012f0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 10 client.17508 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55954fd437d0 con 0x5595500012f0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3d86ff5640 1 -- 172.21.15.7:0/2294845046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d94005020 con 0x5595500012f0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3d86ff5640 1 -- 172.21.15.7:0/2294845046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3d94019450 con 0x5595500012f0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 10 client.17508 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 10 client.17508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 20 client.17508 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5595500173b0 0x559550019830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 4 client.17508 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55954ffc33e0 con 0x5595500173b0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.774+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5595500173b0 0x559550019830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3da27fc640 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5595500173b0 0x559550019830 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ccc85e73ba4ed15 server_cookie=aed0d0cc6337c11f in_seq=0 out_seq=0 2024-03-19T00:47:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3d86ff5640 10 client.17508 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3d86ff5640 1 -- 172.21.15.7:0/2294845046 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55954ffc33e0 con 0x5595500173b0 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3d86ff5640 10 client.17508 handle_command_reply: tid=0 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3dabf72740 1 client.17508 shutdown 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 msgr2=0x55955000b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.775+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x55955000b820 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3d9401b040 tx=0x7f3d940048f0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 shutdown_connections 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x5595500173b0 0x559550019830 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550012d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559550001ce0 0x55955000bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3d86ff5640 0 client.17508 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2294845046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595500012f0 0x55955000b820 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 >> 172.21.15.7:0/2294845046 conn(0x55954fffc860 msgr2=0x55954fffbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 shutdown_connections 2024-03-19T00:47:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 1 -- 172.21.15.7:0/2294845046 wait complete. 2024-03-19T00:47:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.776+0000 7f3dabf72740 20 client.17508 trim_cache size 0 max 16384 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 Processor -- start 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 -- start start 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x5595500001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550001ce0 0x559550012cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550006e50 0x55955000ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da41acb50 con 0x559550006e50 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3da41aa420 con 0x559550001ce0 2024-03-19T00:47:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3dabf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3da41a8270 con 0x5595500012f0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3da2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x5595500001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550006e50 0x55955000ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3da27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550001ce0 0x559550012cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3da2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x5595500001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:43168/0 (socket says 172.21.15.7:43168) 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2322006625 learned_addr learned my addr 172.21.15.7:0/2322006625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.789+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550006e50 0x55955000ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50760/0 (socket says 172.21.15.7:50760) 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2322006625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550001ce0 msgr2=0x559550012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550001ce0 0x559550012cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2322006625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550006e50 msgr2=0x55955000ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550006e50 0x55955000ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3da2ffd640 1 -- 172.21.15.7:0/2322006625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954ffd2880 con 0x5595500012f0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x5595500001d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d940198f0 tx=0x7f3d94009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33113ea21609b882 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3d88ff9640 1 -- 172.21.15.7:0/2322006625 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94029040 con 0x5595500012f0 2024-03-19T00:47:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3d88ff9640 1 -- 172.21.15.7:0/2322006625 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d94005600 con 0x5595500012f0 2024-03-19T00:47:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.790+0000 7f3d88ff9640 1 -- 172.21.15.7:0/2322006625 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94025470 con 0x5595500012f0 2024-03-19T00:47:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- 172.21.15.7:0/2322006625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 msgr2=0x5595500001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x5595500001d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d940198f0 tx=0x7f3d94009a70 comp rx=0 tx=0).stop 2024-03-19T00:47:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- 172.21.15.7:0/2322006625 shutdown_connections 2024-03-19T00:47:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550006e50 0x55955000ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550001ce0 0x559550012cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- 172.21.15.7:0/2322006625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x5595500001d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- 172.21.15.7:0/2322006625 >> 172.21.15.7:0/2322006625 conn(0x55954fffc860 msgr2=0x55954ffffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- 172.21.15.7:0/2322006625 shutdown_connections 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- 172.21.15.7:0/2322006625 wait complete. 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- start start 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 Processor -- start 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x55955000fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 0x559550010350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550011310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da41a88e0 con 0x559550001ce0 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3da41a9d50 con 0x559550006e50 2024-03-19T00:47:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3da41a8fb0 con 0x5595500012f0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3dabf72740 10 client.0 _ll_register_callbacks cb 0x7f3d9c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3da27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 0x559550010350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3da27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 0x559550010350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50776/0 (socket says 172.21.15.7:50776) 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.791+0000 7f3da27fc640 1 -- 172.21.15.7:0/1049034517 learned_addr learned my addr 172.21.15.7:0/1049034517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x55955000fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da37fe640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550011310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da27fc640 1 -- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 msgr2=0x55955000fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da27fc640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x55955000fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da27fc640 1 -- 172.21.15.7:0/1049034517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 msgr2=0x559550011310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da27fc640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550011310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da27fc640 1 -- 172.21.15.7:0/1049034517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559550011900 con 0x559550001ce0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x55955000fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3da27fc640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 0x559550010350 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3d980bd5a0 tx=0x7f3d980a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561b280849574ae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3d86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3d86ff5640 1 -- 172.21.15.7:0/1049034517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980bd150 con 0x559550001ce0 2024-03-19T00:47:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3d86ff5640 1 -- 172.21.15.7:0/1049034517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3d98003d20 con 0x559550001ce0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d94005600 con 0x559550001ce0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3d86ff5640 1 -- 172.21.15.7:0/1049034517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d980bcf70 con 0x559550001ce0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3dabf72740 10 client.17514 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559550012c70 con 0x559550001ce0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.792+0000 7f3d86ff5640 1 -- 172.21.15.7:0/1049034517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d94005600 con 0x559550001ce0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3d86ff5640 1 -- 172.21.15.7:0/1049034517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3d980b9c80 con 0x559550001ce0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 10 client.17514 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 20 client.17514 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55955007daf0 0x55955007ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 4 client.17514 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55954ff729f0 con 0x55955007daf0 2024-03-19T00:47:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55955007daf0 0x55955007ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3d86ff5640 10 client.17514 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55955007daf0 0x55955007ff10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebd03f1cba8197c2 server_cookie=74700df1ff889da2 in_seq=0 out_seq=0 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3d86ff5640 1 -- 172.21.15.7:0/1049034517 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55954ff729f0 con 0x55955007daf0 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3d86ff5640 10 client.17514 handle_command_reply: tid=0 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 1 client.17514 shutdown 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.793+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 msgr2=0x559550010350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 0x559550010350 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3d980bd5a0 tx=0x7f3d980a8040 comp rx=0 tx=0).stop 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 shutdown_connections 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55955007daf0 0x55955007ff10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559550006e50 0x559550011310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559550001ce0 0x559550010350 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3d86ff5640 0 client.17514 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 --2- 172.21.15.7:0/1049034517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595500012f0 0x55955000fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 >> 172.21.15.7:0/1049034517 conn(0x55954fffc860 msgr2=0x559550000350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 shutdown_connections 2024-03-19T00:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 1 -- 172.21.15.7:0/1049034517 wait complete. 2024-03-19T00:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3dabf72740 20 client.17514 trim_cache size 0 max 16384 2024-03-19T00:47:38.802 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3d8c0a2800 msgr2=0x7f3d8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3d8c0a2800 0x7f3d8c0a4cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x559550002540 tx=0x7f3d9c004a40 comp rx=0 tx=0).stop 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 msgr2=0x7f3da4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4197f60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d98005e40 comp rx=0 tx=0).stop 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 shutdown_connections 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f3d8c0a2800 0x7f3d8c0a4cc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3da41022e0 0x7f3da419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4101920 0x7f3da4197f60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 --2- 172.21.15.7:0/2746548632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3da4100730 0x7f3da4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 >> 172.21.15.7:0/2746548632 conn(0x7f3da40fbc80 msgr2=0x7f3da4104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 shutdown_connections 2024-03-19T00:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.794+0000 7f3da92ac640 1 -- 172.21.15.7:0/2746548632 wait complete. 2024-03-19T00:47:38.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 -- 172.21.15.7:0/1978120906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490100740 msgr2=0x7ff490100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 --2- 172.21.15.7:0/1978120906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490100740 0x7ff490100b60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff480009900 tx=0x7ff48002bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 -- 172.21.15.7:0/1978120906 shutdown_connections 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 --2- 172.21.15.7:0/1978120906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4901022f0 0x7ff490108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 --2- 172.21.15.7:0/1978120906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff490101930 0x7ff490101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 --2- 172.21.15.7:0/1978120906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490100740 0x7ff490100b60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 -- 172.21.15.7:0/1978120906 >> 172.21.15.7:0/1978120906 conn(0x7ff4900fbc90 msgr2=0x7ff4900fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 -- 172.21.15.7:0/1978120906 shutdown_connections 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 -- 172.21.15.7:0/1978120906 wait complete. 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 Processor -- start 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.909+0000 7ff496619640 1 -- start start 2024-03-19T00:47:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff496619640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff490100740 0x7ff490197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff496619640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 0x7ff490197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff496619640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4901022f0 0x7ff49019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff496619640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff49010c290 con 0x7ff490101930 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff496619640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff49010c110 con 0x7ff490100740 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff496619640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff49010c410 con 0x7ff4901022f0 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 0x7ff490197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff490100740 0x7ff490197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 0x7ff490197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50786/0 (socket says 172.21.15.7:50786) 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 -- 172.21.15.7:0/16429093 learned_addr learned my addr 172.21.15.7:0/16429093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff494bb6640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4901022f0 0x7ff49019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 -- 172.21.15.7:0/16429093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4901022f0 msgr2=0x7ff49019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4901022f0 0x7ff49019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 -- 172.21.15.7:0/16429093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff490100740 msgr2=0x7ff490197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff490100740 0x7ff490197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 -- 172.21.15.7:0/16429093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49019f6d0 con 0x7ff490101930 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48ffff640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff490100740 0x7ff490197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 0x7ff490197ff0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff48400e9f0 tx=0x7ff48400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.910+0000 7ff48d7fa640 1 -- 172.21.15.7:0/16429093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400ccc0 con 0x7ff490101930 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.911+0000 7ff48d7fa640 1 -- 172.21.15.7:0/16429093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4840108e0 con 0x7ff490101930 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.911+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49019f9c0 con 0x7ff490101930 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.911+0000 7ff48d7fa640 1 -- 172.21.15.7:0/16429093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840187c0 con 0x7ff490101930 2024-03-19T00:47:38.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.911+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49019ff10 con 0x7ff490101930 2024-03-19T00:47:38.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff48d7fa640 1 -- 172.21.15.7:0/16429093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7ff484010a90 con 0x7ff490101930 2024-03-19T00:47:38.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 Processor -- start 2024-03-19T00:47:38.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 -- start start 2024-03-19T00:47:38.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56488edc14b0 con 0x56488ee5dce0 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56488ed69d70 con 0x56488ee5d2f0 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff49920a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56488edbd550 con 0x56488ee62e50 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50798/0 (socket says 172.21.15.7:50798) 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff48ffff640 1 -- 172.21.15.7:0/1825155251 learned_addr learned my addr 172.21.15.7:0/1825155251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff494bb6640 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.913+0000 7ff48d7fa640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff4780a2800 0x7ff4780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48ffff640 1 -- 172.21.15.7:0/1825155251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 msgr2=0x56488ee652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48ffff640 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48ffff640 1 -- 172.21.15.7:0/1825155251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 msgr2=0x56488ee5d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48ffff640 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee5d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48ffff640 1 -- 172.21.15.7:0/1825155251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56488ed8c210 con 0x56488ee5dce0 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff494bb6640 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff494bb6640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff4780a2800 0x7ff4780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48d7fa640 1 -- 172.21.15.7:0/16429093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff4840cf1a0 con 0x7ff490101930 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff48ffff640 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee627e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff480009900 tx=0x7ff4800095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9a8a3a2f6a5ffa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff4767fc640 1 -- 172.21.15.7:0/1825155251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480045020 con 0x56488ee5dce0 2024-03-19T00:47:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff4767fc640 1 -- 172.21.15.7:0/1825155251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff48002cc90 con 0x56488ee5dce0 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.914+0000 7ff4767fc640 1 -- 172.21.15.7:0/1825155251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800048c0 con 0x56488ee5dce0 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- 172.21.15.7:0/1825155251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 msgr2=0x56488ee627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff494bb6640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff4780a2800 0x7ff4780a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x56488ee5e540 tx=0x7ff488004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee627e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff480009900 tx=0x7ff4800095f0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- 172.21.15.7:0/1825155251 shutdown_connections 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee627e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- 172.21.15.7:0/1825155251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- 172.21.15.7:0/1825155251 >> 172.21.15.7:0/1825155251 conn(0x56488ee58860 msgr2=0x56488ee58c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- 172.21.15.7:0/1825155251 shutdown_connections 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- 172.21.15.7:0/1825155251 wait complete. 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- start start 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 Processor -- start 2024-03-19T00:47:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee67710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee67c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee62e50 0x56488ee6ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56488ee65da0 con 0x56488ee5dce0 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56488ed7d310 con 0x56488ee62e50 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.915+0000 7ff49920a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56488ee65f20 con 0x56488ee5d2f0 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff49920a740 10 client.0 _ll_register_callbacks cb 0x56488ed712d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee67c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee67c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50802/0 (socket says 172.21.15.7:50802) 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 -- 172.21.15.7:0/2477620416 learned_addr learned my addr 172.21.15.7:0/2477620416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48ffff640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee67710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff494bb6640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee62e50 0x56488ee6ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 -- 172.21.15.7:0/2477620416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 msgr2=0x56488ee67710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee67710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 -- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee62e50 msgr2=0x56488ee6ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee62e50 0x56488ee6ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 -- 172.21.15.7:0/2477620416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488ead53f0 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee67c50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff484097c40 tx=0x7ff48400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d5fd72988f95ff server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff46f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff46f7fe640 1 -- 172.21.15.7:0/2477620416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48409b7e0 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff46f7fe640 1 -- 172.21.15.7:0/2477620416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff484097e50 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48002cc90 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff46f7fe640 1 -- 172.21.15.7:0/2477620416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff48409f930 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff49920a740 10 client.17526 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56488eb9f7d0 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.916+0000 7ff46f7fe640 1 -- 172.21.15.7:0/2477620416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff48002cc90 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff46f7fe640 1 -- 172.21.15.7:0/2477620416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff48409ad50 con 0x56488ee5dce0 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff49920a740 10 client.17526 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff49920a740 10 client.17526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:47:38.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff49920a740 20 client.17526 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff49920a740 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488ee73570 0x56488ee759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff49920a740 4 client.17526 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56488ee1f3e0 con 0x56488ee73570 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff48ffff640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488ee73570 0x56488ee759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff46f7fe640 10 client.17526 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.917+0000 7ff48ffff640 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488ee73570 0x56488ee759e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d57bcb350ec847ca server_cookie=9cbf291aa14d1f22 in_seq=0 out_seq=0 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff46f7fe640 1 -- 172.21.15.7:0/2477620416 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56488ee1f3e0 con 0x56488ee73570 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff46f7fe640 10 client.17526 handle_command_reply: tid=0 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 client.17526 shutdown 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 msgr2=0x56488ee67c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee67c50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff484097c40 tx=0x7ff48400ce60 comp rx=0 tx=0).stop 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 shutdown_connections 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488ee73570 0x56488ee759e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee62e50 0x56488ee6ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff46f7fe640 0 client.17526 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee67c50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 --2- 172.21.15.7:0/2477620416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee67710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 >> 172.21.15.7:0/2477620416 conn(0x56488ee58860 msgr2=0x56488ee57ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 shutdown_connections 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 1 -- 172.21.15.7:0/2477620416 wait complete. 2024-03-19T00:47:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.918+0000 7ff49920a740 20 client.17526 trim_cache size 0 max 16384 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.931+0000 7ff49920a740 1 Processor -- start 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.931+0000 7ff49920a740 1 -- start start 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff49920a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff49920a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5dce0 0x56488ee6ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff49920a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 0x56488ee57850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff49920a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901acbe0 con 0x56488ee62e50 2024-03-19T00:47:38.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff49920a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901aa4b0 con 0x56488ee5dce0 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff49920a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901a8300 con 0x56488ee5d2f0 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 0x56488ee57850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 0x56488ee57850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50810/0 (socket says 172.21.15.7:50810) 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 -- 172.21.15.7:0/852580471 learned_addr learned my addr 172.21.15.7:0/852580471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5dce0 0x56488ee6ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff48ffff640 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 -- 172.21.15.7:0/852580471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 msgr2=0x56488ee57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee57310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 -- 172.21.15.7:0/852580471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5dce0 msgr2=0x56488ee6ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5dce0 0x56488ee6ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 -- 172.21.15.7:0/852580471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56488ee2e880 con 0x56488ee62e50 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5dce0 0x56488ee6ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff494bb6640 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 0x56488ee57850 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff48801d5b0 tx=0x7ff488019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff016bc3709208b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff475ffb640 1 -- 172.21.15.7:0/852580471 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48802e070 con 0x56488ee62e50 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff475ffb640 1 -- 172.21.15.7:0/852580471 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff488010a20 con 0x56488ee62e50 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.932+0000 7ff475ffb640 1 -- 172.21.15.7:0/852580471 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48801cbc0 con 0x56488ee62e50 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 -- 172.21.15.7:0/852580471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 msgr2=0x56488ee57850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 0x56488ee57850 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff48801d5b0 tx=0x7ff488019000 comp rx=0 tx=0).stop 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 -- 172.21.15.7:0/852580471 shutdown_connections 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee62e50 0x56488ee57850 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5dce0 0x56488ee6ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 --2- 172.21.15.7:0/852580471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee5d2f0 0x56488ee57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 -- 172.21.15.7:0/852580471 >> 172.21.15.7:0/852580471 conn(0x56488ee58860 msgr2=0x56488ee53a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 -- 172.21.15.7:0/852580471 shutdown_connections 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 -- 172.21.15.7:0/852580471 wait complete. 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 -- start start 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.933+0000 7ff49920a740 1 Processor -- start 2024-03-19T00:47:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee6cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee6cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee6e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901a8970 con 0x56488ee5dce0 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901a9de0 con 0x56488ee5d2f0 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901a9040 con 0x56488ee62e50 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 10 client.0 _ll_register_callbacks cb 0x7ff48400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee6cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee6cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50820/0 (socket says 172.21.15.7:50820) 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 -- 172.21.15.7:0/4098845505 learned_addr learned my addr 172.21.15.7:0/4098845505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff494bb6640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee6e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48ffff640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee6cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 -- 172.21.15.7:0/4098845505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 msgr2=0x56488ee6e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee6e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 -- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 msgr2=0x56488ee6cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee6cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 -- 172.21.15.7:0/4098845505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488ee6e5f0 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff48f7fe640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee6cff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff4840cccb0 tx=0x7ff484010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5dc0a276ad363f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff46f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff46f7fe640 1 -- 172.21.15.7:0/4098845505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840a3f80 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff46f7fe640 1 -- 172.21.15.7:0/4098845505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff4840afbc0 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff488010a20 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff46f7fe640 1 -- 172.21.15.7:0/4098845505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4840afdc0 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 10 client.17532 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.934+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56488eeda070 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff46f7fe640 1 -- 172.21.15.7:0/4098845505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff488010a20 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff46f7fe640 1 -- 172.21.15.7:0/4098845505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4840976d0 con 0x56488ee5dce0 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff49920a740 10 client.17532 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff49920a740 10 client.17532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:47:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff49920a740 20 client.17532 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff49920a740 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488eedaa40 0x56488eedce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff49920a740 4 client.17532 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56488edce9f0 con 0x56488eedaa40 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff48ffff640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488eedaa40 0x56488eedce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff46f7fe640 10 client.17532 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.935+0000 7ff48ffff640 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488eedaa40 0x56488eedce60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec89a906d14f016 server_cookie=cbd09eb5ab882284 in_seq=0 out_seq=0 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff46f7fe640 1 -- 172.21.15.7:0/4098845505 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56488edce9f0 con 0x56488eedaa40 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff46f7fe640 10 client.17532 handle_command_reply: tid=0 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 client.17532 shutdown 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 msgr2=0x56488ee6cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee6cff0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff4840cccb0 tx=0x7ff484010e80 comp rx=0 tx=0).stop 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 shutdown_connections 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x56488eedaa40 0x56488eedce60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56488ee62e50 0x56488ee6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56488ee5dce0 0x56488ee6cff0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff46f7fe640 0 client.17532 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 --2- 172.21.15.7:0/4098845505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56488ee5d2f0 0x56488ee6cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 >> 172.21.15.7:0/4098845505 conn(0x56488ee58860 msgr2=0x56488ee5bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 shutdown_connections 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 1 -- 172.21.15.7:0/4098845505 wait complete. 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff49920a740 20 client.17532 trim_cache size 0 max 16384 2024-03-19T00:47:38.944 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff4780a2800 msgr2=0x7ff4780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff496619640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff4780a2800 0x7ff4780a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x56488ee5e540 tx=0x7ff488004a40 comp rx=0 tx=0).stop 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 msgr2=0x7ff490197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.936+0000 7ff496619640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 0x7ff490197ff0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff48400e9f0 tx=0x7ff48400edf0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 shutdown_connections 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff4780a2800 0x7ff4780a4cc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4901022f0 0x7ff49019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff490101930 0x7ff490197ff0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 --2- 172.21.15.7:0/16429093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff490100740 0x7ff490197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 >> 172.21.15.7:0/16429093 conn(0x7ff4900fbc90 msgr2=0x7ff4900fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 shutdown_connections 2024-03-19T00:47:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:38.937+0000 7ff496619640 1 -- 172.21.15.7:0/16429093 wait complete. 2024-03-19T00:47:38.946 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:38 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2992673233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:47:38.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:47:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 -- 172.21.15.7:0/3574870668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae01033d0 msgr2=0x7f7ae0109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 --2- 172.21.15.7:0/3574870668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae01033d0 0x7f7ae0109c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009be0 tx=0x7f7ad802bef0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 -- 172.21.15.7:0/3574870668 shutdown_connections 2024-03-19T00:47:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 --2- 172.21.15.7:0/3574870668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae01033d0 0x7f7ae0109c60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 --2- 172.21.15.7:0/3574870668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ae0102a10 0x7f7ae0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 --2- 172.21.15.7:0/3574870668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 -- 172.21.15.7:0/3574870668 >> 172.21.15.7:0/3574870668 conn(0x7f7ae00fcd50 msgr2=0x7f7ae00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 -- 172.21.15.7:0/3574870668 shutdown_connections 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.051+0000 7f7ae8176640 1 -- 172.21.15.7:0/3574870668 wait complete. 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 Processor -- start 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 -- start start 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae0102a10 0x7f7ae0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ae01033d0 0x7f7ae019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010d330 con 0x7f7ae0101820 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010d1b0 con 0x7f7ae0102a10 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae8176640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010d4b0 con 0x7f7ae01033d0 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae01979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50848/0 (socket says 172.21.15.7:50848) 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2981787533 learned_addr learned my addr 172.21.15.7:0/2981787533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5711640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae0102a10 0x7f7ae0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ae01033d0 0x7f7ae019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2981787533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ae01033d0 msgr2=0x7f7ae019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ae01033d0 0x7f7ae019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2981787533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae0102a10 msgr2=0x7f7ae0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae0102a10 0x7f7ae0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.052+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2981787533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae019f5c0 con 0x7f7ae0101820 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.053+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae01979f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e9a0 tx=0x7f7adc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.053+0000 7f7ad6ffd640 1 -- 172.21.15.7:0/2981787533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00cd80 con 0x7f7ae0101820 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.053+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae019f8b0 con 0x7f7ae0101820 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.053+0000 7f7ad6ffd640 1 -- 172.21.15.7:0/2981787533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7adc0108e0 con 0x7f7ae0101820 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.053+0000 7f7ad6ffd640 1 -- 172.21.15.7:0/2981787533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0188a0 con 0x7f7ae0101820 2024-03-19T00:47:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.053+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae019fe50 con 0x7f7ae0101820 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.055+0000 7f7ad6ffd640 1 -- 172.21.15.7:0/2981787533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7adc010a90 con 0x7f7ae0101820 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.055+0000 7f7aeae35740 1 Processor -- start 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 -- start start 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113edb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 0x560d113f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d1134d4b0 con 0x560d113ee0f0 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560d112f5d70 con 0x560d113ed700 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7aeae35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560d11349550 con 0x560d113f3260 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ad6ffd640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7acc0a2800 0x7f7acc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 0x560d113f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 0x560d113f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50856/0 (socket says 172.21.15.7:50856) 2024-03-19T00:47:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae5711640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2273539854 learned_addr learned my addr 172.21.15.7:0/2273539854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7acc0a2800 0x7f7acc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ad6ffd640 1 -- 172.21.15.7:0/2981787533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7adc0cf140 con 0x7f7ae0101820 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 -- 172.21.15.7:0/2273539854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 msgr2=0x560d113f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 -- 172.21.15.7:0/2273539854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 msgr2=0x560d113f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 0x560d113f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae6713640 1 -- 172.21.15.7:0/2273539854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d11318210 con 0x560d113ed700 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.056+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 0x560d113f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113edb20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ad802c430 tx=0x7f7ad802db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f552db39d025da8b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7acc0a2800 0x7f7acc0a4cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8036040 tx=0x7f7ad8049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7acbfff640 1 -- 172.21.15.7:0/2273539854 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8003d40 con 0x560d113ed700 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7acbfff640 1 -- 172.21.15.7:0/2273539854 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ad8051070 con 0x560d113ed700 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7acbfff640 1 -- 172.21.15.7:0/2273539854 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004280 con 0x560d113ed700 2024-03-19T00:47:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 -- 172.21.15.7:0/2273539854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 msgr2=0x560d113edb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113edb20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ad802c430 tx=0x7f7ad802db30 comp rx=0 tx=0).stop 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 -- 172.21.15.7:0/2273539854 shutdown_connections 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ee0f0 0x560d113f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2273539854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113edb20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.057+0000 7f7aeae35740 1 -- 172.21.15.7:0/2273539854 >> 172.21.15.7:0/2273539854 conn(0x560d113e4860 msgr2=0x560d113e4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 -- 172.21.15.7:0/2273539854 shutdown_connections 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 -- 172.21.15.7:0/2273539854 wait complete. 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 -- start start 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 Processor -- start 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 0x560d113f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d113f61b0 con 0x560d113ed700 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560d11309310 con 0x560d113ee0f0 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560d113f6330 con 0x560d113f3260 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7aeae35740 10 client.0 _ll_register_callbacks cb 0x560d112fd2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 0x560d113f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 0x560d113f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50866/0 (socket says 172.21.15.7:50866) 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2513711191 learned_addr learned my addr 172.21.15.7:0/2513711191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7ae5711640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.058+0000 7f7ae6713640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2513711191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 msgr2=0x560d113ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113ff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 msgr2=0x560d113f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5f12640 1 -- 172.21.15.7:0/2513711191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d110613f0 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5711640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 0x560d113f7bd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009230 tx=0x7f7adc09b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d225edbc27bc832 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ac97fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ac97fa640 1 -- 172.21.15.7:0/2513711191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc09b780 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ac97fa640 1 -- 172.21.15.7:0/2513711191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7adc0afb60 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad8051070 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ac97fa640 1 -- 172.21.15.7:0/2513711191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7adc09f8d0 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7aeae35740 10 client.17544 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d1112b7d0 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ac97fa640 1 -- 172.21.15.7:0/2513711191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ad8051070 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7ac97fa640 1 -- 172.21.15.7:0/2513711191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc097040 con 0x560d113ed700 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7aeae35740 10 client.17544 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.059+0000 7f7aeae35740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7aeae35740 20 client.17544 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:39.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11403a60 0x560d11405ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7aeae35740 4 client.17544 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d113ab3e0 con 0x560d11403a60 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7ae5711640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11403a60 0x560d11405ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7ac97fa640 10 client.17544 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.060+0000 7f7ae5711640 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11403a60 0x560d11405ed0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab332ab6aba8790d server_cookie=177e3072dd4533fe in_seq=0 out_seq=0 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7ac97fa640 1 -- 172.21.15.7:0/2513711191 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560d113ab3e0 con 0x560d11403a60 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7ac97fa640 10 client.17544 handle_command_reply: tid=0 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 client.17544 shutdown 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 msgr2=0x560d113f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 0x560d113f7bd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009230 tx=0x7f7adc09b380 comp rx=0 tx=0).stop 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 shutdown_connections 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113f3260 0x560d113ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11403a60 0x560d11405ed0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 --2- 172.21.15.7:0/2513711191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113ed700 0x560d113f7bd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 >> 172.21.15.7:0/2513711191 conn(0x560d113e4860 msgr2=0x560d113e3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:39.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7ac97fa640 0 client.17544 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:39.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 shutdown_connections 2024-03-19T00:47:39.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 1 -- 172.21.15.7:0/2513711191 wait complete. 2024-03-19T00:47:39.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.061+0000 7f7aeae35740 20 client.17544 trim_cache size 0 max 16384 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.074+0000 7f7aeae35740 1 Processor -- start 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.074+0000 7f7aeae35740 1 -- start start 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.074+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.074+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ee0f0 0x560d113ff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7aeae35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01aa7c0 con 0x560d113f3260 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7aeae35740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01a8090 con 0x560d113ed700 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7aeae35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01a5ee0 con 0x560d113ee0f0 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae6713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49116/0 (socket says 172.21.15.7:49116) 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae6713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50874/0 (socket says 172.21.15.7:50874) 2024-03-19T00:47:39.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 -- 172.21.15.7:0/196663230 learned_addr learned my addr 172.21.15.7:0/196663230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5711640 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ee0f0 0x560d113ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 -- 172.21.15.7:0/196663230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ee0f0 msgr2=0x560d113ff110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ee0f0 0x560d113ff110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 -- 172.21.15.7:0/196663230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 msgr2=0x560d113f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 -- 172.21.15.7:0/196663230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d11406550 con 0x560d113ed700 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113f77e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7adc09b480 tx=0x7f7adc009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b240cf0e30b2c814 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7acb7fe640 1 -- 172.21.15.7:0/196663230 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc09abd0 con 0x560d113ed700 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7acb7fe640 1 -- 172.21.15.7:0/196663230 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7adc09b780 con 0x560d113ed700 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.075+0000 7f7acb7fe640 1 -- 172.21.15.7:0/196663230 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0afb60 con 0x560d113ed700 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 -- 172.21.15.7:0/196663230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 msgr2=0x560d113f77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113f77e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7adc09b480 tx=0x7f7adc009400 comp rx=0 tx=0).stop 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 -- 172.21.15.7:0/196663230 shutdown_connections 2024-03-19T00:47:39.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ee0f0 0x560d113ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 --2- 172.21.15.7:0/196663230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ed700 0x560d113f77e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 -- 172.21.15.7:0/196663230 >> 172.21.15.7:0/196663230 conn(0x560d113e4860 msgr2=0x560d113e4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 -- 172.21.15.7:0/196663230 shutdown_connections 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 -- 172.21.15.7:0/196663230 wait complete. 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 -- start start 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.076+0000 7f7aeae35740 1 Processor -- start 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 0x560d113fc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01a6550 con 0x560d113f3260 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01a79c0 con 0x560d113ee0f0 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01a6c20 con 0x560d113ed700 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7aeae35740 10 client.0 _ll_register_callbacks cb 0x7f7adc09b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:39.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae6713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 0x560d113fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 0x560d113fc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:43328/0 (socket says 172.21.15.7:43328) 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae6713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50880/0 (socket says 172.21.15.7:50880) 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5711640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113fc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49132/0 (socket says 172.21.15.7:49132) 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 -- 172.21.15.7:0/1100500777 learned_addr learned my addr 172.21.15.7:0/1100500777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 -- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 msgr2=0x560d113fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 -- 172.21.15.7:0/1100500777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 msgr2=0x560d113fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 -- 172.21.15.7:0/1100500777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d113ba880 con 0x560d113ed700 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5711640 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ae5f12640 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 0x560d113fc420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7adc09aef0 tx=0x7f7adc09b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6ad30d80e3a873 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ac97fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.077+0000 7f7ac97fa640 1 -- 172.21.15.7:0/1100500777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc09abd0 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7adc09b780 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7ac97fa640 1 -- 172.21.15.7:0/1100500777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7adc010430 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7ac97fa640 1 -- 172.21.15.7:0/1100500777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7adc0a3f20 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 10 client.27307 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d113ba880 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7ac97fa640 1 -- 172.21.15.7:0/1100500777 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7adc09b780 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7ac97fa640 1 -- 172.21.15.7:0/1100500777 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc09cb40 con 0x560d113ed700 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 10 client.27307 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 10 client.27307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 20 client.27307 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11469e10 0x560d1146c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 4 client.27307 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:47:39.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d1135a9f0 con 0x560d11469e10 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.078+0000 7f7ae5711640 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11469e10 0x560d1146c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7ac97fa640 10 client.27307 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7ae5711640 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11469e10 0x560d1146c200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef1e4d13a05c7a4 server_cookie=f6d586599d7c71b2 in_seq=0 out_seq=0 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7ac97fa640 1 -- 172.21.15.7:0/1100500777 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d1135a9f0 con 0x560d11469e10 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7ac97fa640 10 client.27307 handle_command_reply: tid=0 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 client.27307 shutdown 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 msgr2=0x560d113fc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 0x560d113fc420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7adc09aef0 tx=0x7f7adc09b680 comp rx=0 tx=0).stop 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 shutdown_connections 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x560d11469e10 0x560d1146c200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d113f3260 0x560d113fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560d113ee0f0 0x560d113fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7ac97fa640 0 client.27307 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 --2- 172.21.15.7:0/1100500777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d113ed700 0x560d113fc420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 >> 172.21.15.7:0/1100500777 conn(0x560d113e4860 msgr2=0x560d113f7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:39.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 shutdown_connections 2024-03-19T00:47:39.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 1 -- 172.21.15.7:0/1100500777 wait complete. 2024-03-19T00:47:39.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.079+0000 7f7aeae35740 20 client.27307 trim_cache size 0 max 16384 2024-03-19T00:47:39.087 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7acc0a2800 msgr2=0x7f7acc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7acc0a2800 0x7f7acc0a4cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8036040 tx=0x7f7ad8049000 comp rx=0 tx=0).stop 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 msgr2=0x7f7ae01979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae01979f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e9a0 tx=0x7f7adc00ee60 comp rx=0 tx=0).stop 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 shutdown_connections 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7acc0a2800 0x7f7acc0a4cc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ae01033d0 0x7f7ae019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ae0102a10 0x7f7ae0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 --2- 172.21.15.7:0/2981787533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0101820 0x7f7ae01979f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 >> 172.21.15.7:0/2981787533 conn(0x7f7ae00fcd50 msgr2=0x7f7ae0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 shutdown_connections 2024-03-19T00:47:39.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:39.080+0000 7f7ae8176640 1 -- 172.21.15.7:0/2981787533 wait complete. 2024-03-19T00:47:39.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:38 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2992673233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:47:39.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:38 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2992673233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:47:40.092 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:47:40.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:47:40.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:47:40.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:39 smithi044 ceph-mon[66366]: pgmap v642: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:40.120 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:39 smithi007 ceph-mon[62854]: pgmap v642: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:40.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.210+0000 7f1e02636640 1 -- 172.21.15.7:0/3539604805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc104c50 msgr2=0x7f1dfc107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.210+0000 7f1e02636640 1 --2- 172.21.15.7:0/3539604805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc104c50 0x7f1dfc107040 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009900 tx=0x7f1de402bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.210+0000 7f1e02636640 1 -- 172.21.15.7:0/3539604805 shutdown_connections 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.210+0000 7f1e02636640 1 --2- 172.21.15.7:0/3539604805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc107580 0x7f1dfc109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.210+0000 7f1e02636640 1 --2- 172.21.15.7:0/3539604805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc104c50 0x7f1dfc107040 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.210+0000 7f1e02636640 1 --2- 172.21.15.7:0/3539604805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dfc102320 0x7f1dfc104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 -- 172.21.15.7:0/3539604805 >> 172.21.15.7:0/3539604805 conn(0x7f1dfc0fbcb0 msgr2=0x7f1dfc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 -- 172.21.15.7:0/3539604805 shutdown_connections 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 -- 172.21.15.7:0/3539604805 wait complete. 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 Processor -- start 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 -- start start 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc102320 0x7f1dfc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dfc104c50 0x7f1dfc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.211+0000 7f1e02636640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 0x7f1dfc19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e02636640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc10d360 con 0x7f1dfc107580 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e02636640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc10d1e0 con 0x7f1dfc102320 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e02636640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc10d4e0 con 0x7f1dfc104c50 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 0x7f1dfc19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 0x7f1dfc19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50904/0 (socket says 172.21.15.7:50904) 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 -- 172.21.15.7:0/124229551 learned_addr learned my addr 172.21.15.7:0/124229551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc102320 0x7f1dfc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dfc104c50 0x7f1dfc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 -- 172.21.15.7:0/124229551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dfc104c50 msgr2=0x7f1dfc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dfc104c50 0x7f1dfc198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 -- 172.21.15.7:0/124229551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc102320 msgr2=0x7f1dfc197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc102320 0x7f1dfc197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 -- 172.21.15.7:0/124229551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc19f720 con 0x7f1dfc107580 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc102320 0x7f1dfc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e00bd3640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 0x7f1dfc19f0b0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1df400efd0 tx=0x7f1df400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1df97fa640 1 -- 172.21.15.7:0/124229551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4003b70 con 0x7f1dfc107580 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1df97fa640 1 -- 172.21.15.7:0/124229551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df4003d10 con 0x7f1dfc107580 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc19fa10 con 0x7f1dfc107580 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1df97fa640 1 -- 172.21.15.7:0/124229551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4010980 con 0x7f1dfc107580 2024-03-19T00:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.212+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc19fee0 con 0x7f1dfc107580 2024-03-19T00:47:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1df97fa640 1 -- 172.21.15.7:0/124229551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f1df4004020 con 0x7f1dfc107580 2024-03-19T00:47:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 Processor -- start 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 -- start start 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571490903a0 0x5571490907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x557149095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x557149098390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557148ff5160 con 0x557149095f00 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557148f9de10 con 0x557149090d90 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1e0521e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557148ff1200 con 0x5571490903a0 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x557149095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x557149098390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x557149095890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49156/0 (socket says 172.21.15.7:49156) 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x557149098390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50918/0 (socket says 172.21.15.7:50918) 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1dfbfff640 1 -- 172.21.15.7:0/835114659 learned_addr learned my addr 172.21.15.7:0/835114659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.215+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571490903a0 0x5571490907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1df97fa640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1de80a2800 0x7f1de80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1dfbfff640 1 -- 172.21.15.7:0/835114659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571490903a0 msgr2=0x5571490907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571490903a0 0x5571490907c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1dfbfff640 1 -- 172.21.15.7:0/835114659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 msgr2=0x557149098390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x557149098390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1dfbfff640 1 -- 172.21.15.7:0/835114659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557148dd17e0 con 0x557149090d90 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1e00bd3640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1de80a2800 0x7f1de80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1df97fa640 1 -- 172.21.15.7:0/124229551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1df40cf220 con 0x7f1dfc107580 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x557149095890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0115c0 tx=0x7f1dec011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92f866659871ede server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1df27fc640 1 -- 172.21.15.7:0/835114659 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00f5c0 con 0x557149090d90 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1e00bd3640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1de80a2800 0x7f1de80a4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1df409b560 tx=0x7f1df4013b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1df27fc640 1 -- 172.21.15.7:0/835114659 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec00fc10 con 0x557149090d90 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.216+0000 7f1df27fc640 1 -- 172.21.15.7:0/835114659 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec019cb0 con 0x557149090d90 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 -- 172.21.15.7:0/835114659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 msgr2=0x557149095890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x557149095890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0115c0 tx=0x7f1dec011a80 comp rx=0 tx=0).stop 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 -- 172.21.15.7:0/835114659 shutdown_connections 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x557149098390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x557149095890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 --2- 172.21.15.7:0/835114659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571490903a0 0x5571490907c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 -- 172.21.15.7:0/835114659 >> 172.21.15.7:0/835114659 conn(0x55714908b910 msgr2=0x55714908bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 -- 172.21.15.7:0/835114659 shutdown_connections 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 -- 172.21.15.7:0/835114659 wait complete. 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 -- start start 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.217+0000 7f1e0521e740 1 Processor -- start 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 0x55714909ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x5571490a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557149098eb0 con 0x557149095f00 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557148fb11a0 con 0x5571490903a0 2024-03-19T00:47:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557149099030 con 0x557149090d90 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1e0521e740 10 client.0 _ll_register_callbacks cb 0x557148d08c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x5571490a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 0x55714909ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49170/0 (socket says 172.21.15.7:49170) 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/4019792300 learned_addr learned my addr 172.21.15.7:0/4019792300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 0x55714909ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:43366/0 (socket says 172.21.15.7:43366) 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/4019792300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 msgr2=0x55714909ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 0x55714909ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/4019792300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 msgr2=0x5571490a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x5571490a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/4019792300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557148e31f70 con 0x5571490903a0 2024-03-19T00:47:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 0x55714909ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909a830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1de402c840 tx=0x7f1de402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3880a3122a3ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1ddb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.218+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/4019792300 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003a10 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dec00fc10 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/4019792300 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1de4003bb0 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/4019792300 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1de4055050 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 10 client.27407 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557148cc9170 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/4019792300 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec00fc10 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/4019792300 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1de4004a10 con 0x5571490903a0 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 10 client.27407 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 20 client.27407 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:40.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5571490a65d0 0x5571490a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 4 client.27407 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571490026a0 con 0x5571490a65d0 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.219+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5571490a65d0 0x5571490a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5571490a65d0 0x5571490a8a40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b0b9e5d639d717 server_cookie=99b211e28ca27c2b in_seq=0 out_seq=0 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1ddb7fe640 10 client.27407 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/4019792300 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5571490026a0 con 0x5571490a65d0 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1ddb7fe640 10 client.27407 handle_command_reply: tid=0 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 client.27407 shutdown 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 msgr2=0x55714909a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909a830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1de402c840 tx=0x7f1de402bc30 comp rx=0 tx=0).stop 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 shutdown_connections 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x5571490a65d0 0x5571490a8a40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149095f00 0x5571490a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1ddb7fe640 0 client.27407 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149090d90 0x55714909ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 --2- 172.21.15.7:0/4019792300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909a830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.220+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 >> 172.21.15.7:0/4019792300 conn(0x55714908b910 msgr2=0x55714908ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.221+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 shutdown_connections 2024-03-19T00:47:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.221+0000 7f1e0521e740 1 -- 172.21.15.7:0/4019792300 wait complete. 2024-03-19T00:47:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.221+0000 7f1e0521e740 20 client.27407 trim_cache size 0 max 16384 2024-03-19T00:47:40.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 Processor -- start 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 -- start start 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 0x5571490a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x55714908f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1acbc0 con 0x557149090d90 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1aa490 con 0x5571490903a0 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.238+0000 7f1e0521e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1a82e0 con 0x557149095f00 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 0x5571490a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 0x5571490a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50950/0 (socket says 172.21.15.7:50950) 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 -- 172.21.15.7:0/3670857894 learned_addr learned my addr 172.21.15.7:0/3670857894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x55714908f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 -- 172.21.15.7:0/3670857894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 msgr2=0x55714908f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x55714908f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 -- 172.21.15.7:0/3670857894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 msgr2=0x55714909acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 -- 172.21.15.7:0/3670857894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55714908b4b0 con 0x557149090d90 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 0x5571490a1d70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009ea0 tx=0x7f1dec005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16074eaa1b5e1363 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1df1ffb640 1 -- 172.21.15.7:0/3670857894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00f5c0 con 0x557149090d90 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1df1ffb640 1 -- 172.21.15.7:0/3670857894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec00fc10 con 0x557149090d90 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.239+0000 7f1df1ffb640 1 -- 172.21.15.7:0/3670857894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec019b80 con 0x557149090d90 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- 172.21.15.7:0/3670857894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 msgr2=0x5571490a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 0x5571490a1d70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009ea0 tx=0x7f1dec005550 comp rx=0 tx=0).stop 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- 172.21.15.7:0/3670857894 shutdown_connections 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x55714908f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557149090d90 0x5571490a1d70 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- 172.21.15.7:0/3670857894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5571490903a0 0x55714909acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- 172.21.15.7:0/3670857894 >> 172.21.15.7:0/3670857894 conn(0x55714908b910 msgr2=0x55714908eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- 172.21.15.7:0/3670857894 shutdown_connections 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- 172.21.15.7:0/3670857894 wait complete. 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- start start 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 Processor -- start 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 0x55714909fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x55714909ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x5571490a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1a8950 con 0x5571490903a0 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1a9dc0 con 0x557149090d90 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.240+0000 7f1e0521e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1a9020 con 0x557149095f00 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1e0521e740 10 client.0 _ll_register_callbacks cb 0x7f1de4004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 0x55714909fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 0x55714909fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50954/0 (socket says 172.21.15.7:50954) 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/1467179194 learned_addr learned my addr 172.21.15.7:0/1467179194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x55714909ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x5571490a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/1467179194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 msgr2=0x5571490a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x5571490a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 msgr2=0x55714909ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x55714909ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 -- 172.21.15.7:0/1467179194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571490a15d0 con 0x5571490903a0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1dfb7fe640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 0x55714909fa90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1de402c170 tx=0x7f1de402ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d058f26dbde0e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1ddb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/1467179194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003c10 con 0x5571490903a0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/1467179194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f1de4003db0 con 0x5571490903a0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dec00fc10 con 0x5571490903a0 2024-03-19T00:47:40.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/1467179194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1de4066050 con 0x5571490903a0 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1e0521e740 10 client.17568 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.241+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55714910ceb0 con 0x5571490903a0 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/1467179194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec00fc10 con 0x5571490903a0 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/1467179194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1de40437f0 con 0x5571490903a0 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1e0521e740 10 client.17568 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1e0521e740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1e0521e740 20 client.17568 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1e0521e740 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55714910d820 0x55714910fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1e0521e740 4 client.17568 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557148ffc9f0 con 0x55714910d820 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55714910d820 0x55714910fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1dfbfff640 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55714910d820 0x55714910fc40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71846d649443c93c server_cookie=eb83cd7d8644627e in_seq=0 out_seq=0 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.242+0000 7f1ddb7fe640 10 client.17568 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.243+0000 7f1ddb7fe640 1 -- 172.21.15.7:0/1467179194 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557148ffc9f0 con 0x55714910d820 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.243+0000 7f1ddb7fe640 10 client.17568 handle_command_reply: tid=0 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.243+0000 7f1e0521e740 1 client.17568 shutdown 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.243+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 msgr2=0x55714909fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.243+0000 7f1e0521e740 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 0x55714909fa90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1de402c170 tx=0x7f1de402ccc0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.243+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 shutdown_connections 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557149095f00 0x5571490a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557149090d90 0x55714909ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55714910d820 0x55714910fc40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 --2- 172.21.15.7:0/1467179194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571490903a0 0x55714909fa90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1ddb7fe640 0 client.17568 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 >> 172.21.15.7:0/1467179194 conn(0x55714908b910 msgr2=0x55714908f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 shutdown_connections 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 1 -- 172.21.15.7:0/1467179194 wait complete. 2024-03-19T00:47:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e0521e740 20 client.17568 trim_cache size 0 max 16384 2024-03-19T00:47:40.251 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:47:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:47:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "5e9af347-af3f-4747-a793-920f18c80946", 2024-03-19T00:47:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:47:40.251 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1de80a2800 msgr2=0x7f1de80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1de80a2800 0x7f1de80a4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1df409b560 tx=0x7f1df4013b60 comp rx=0 tx=0).stop 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 msgr2=0x7f1dfc19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 0x7f1dfc19f0b0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1df400efd0 tx=0x7f1df400c6a0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 shutdown_connections 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f1de80a2800 0x7f1de80a4cc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc107580 0x7f1dfc19f0b0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dfc104c50 0x7f1dfc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 --2- 172.21.15.7:0/124229551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dfc102320 0x7f1dfc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.244+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 >> 172.21.15.7:0/124229551 conn(0x7f1dfc0fbcb0 msgr2=0x7f1dfc102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.245+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 shutdown_connections 2024-03-19T00:47:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.245+0000 7f1e02636640 1 -- 172.21.15.7:0/124229551 wait complete. 2024-03-19T00:47:40.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:47:40.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:47:40.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:39 smithi204 ceph-mon[66572]: pgmap v642: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 -- 172.21.15.7:0/775282416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741022f0 msgr2=0x7f5c74108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 --2- 172.21.15.7:0/775282416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741022f0 0x7f5c74108b80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009ca0 tx=0x7f5c6c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 -- 172.21.15.7:0/775282416 shutdown_connections 2024-03-19T00:47:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 --2- 172.21.15.7:0/775282416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741022f0 0x7f5c74108b80 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 --2- 172.21.15.7:0/775282416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c74101930 0x7f5c74101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 --2- 172.21.15.7:0/775282416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c74100740 0x7f5c74100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 -- 172.21.15.7:0/775282416 >> 172.21.15.7:0/775282416 conn(0x7f5c740fbcd0 msgr2=0x7f5c740fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 -- 172.21.15.7:0/775282416 shutdown_connections 2024-03-19T00:47:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.400+0000 7f5c79063640 1 -- 172.21.15.7:0/775282416 wait complete. 2024-03-19T00:47:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 Processor -- start 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 -- start start 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 0x7f5c74197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c74101930 0x7f5c74197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741022f0 0x7f5c7419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7410c250 con 0x7f5c74100740 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7410c0d0 con 0x7f5c74101930 2024-03-19T00:47:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c79063640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7410c3d0 con 0x7f5c741022f0 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 0x7f5c74197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 0x7f5c74197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50974/0 (socket says 172.21.15.7:50974) 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 -- 172.21.15.7:0/2136839823 learned_addr learned my addr 172.21.15.7:0/2136839823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c74101930 0x7f5c74197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c7359e640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741022f0 0x7f5c7419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 -- 172.21.15.7:0/2136839823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741022f0 msgr2=0x7f5c7419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741022f0 0x7f5c7419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 -- 172.21.15.7:0/2136839823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c74101930 msgr2=0x7f5c74197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c74101930 0x7f5c74197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.401+0000 7f5c72d9d640 1 -- 172.21.15.7:0/2136839823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7419f600 con 0x7f5c74100740 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.402+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 0x7f5c74197a30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e9c0 tx=0x7f5c6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.402+0000 7f5c70d99640 1 -- 172.21.15.7:0/2136839823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400cd80 con 0x7f5c74100740 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.402+0000 7f5c70d99640 1 -- 172.21.15.7:0/2136839823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c640035f0 con 0x7f5c74100740 2024-03-19T00:47:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.402+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7419f8f0 con 0x7f5c74100740 2024-03-19T00:47:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.402+0000 7f5c70d99640 1 -- 172.21.15.7:0/2136839823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64010690 con 0x7f5c74100740 2024-03-19T00:47:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.402+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7419fe40 con 0x7f5c74100740 2024-03-19T00:47:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.404+0000 7f5c70d99640 1 -- 172.21.15.7:0/2136839823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5c640037a0 con 0x7f5c74100740 2024-03-19T00:47:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.404+0000 7f5c7bc4a740 1 Processor -- start 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 -- start start 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddefe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf0440 0x55774ddf4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddf7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55774dd54500 con 0x55774ddefa50 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55774dcfcbe0 con 0x55774ddf0440 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55774dd505a0 con 0x55774ddf55b0 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c70d99640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c600a2800 0x7f5c600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddefe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf0440 0x55774ddf4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddf7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddefe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50978/0 (socket says 172.21.15.7:50978) 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7359e640 1 -- 172.21.15.7:0/542986627 learned_addr learned my addr 172.21.15.7:0/542986627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7359e640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c600a2800 0x7f5c600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c70d99640 1 -- 172.21.15.7:0/2136839823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5c640cf1b0 con 0x7f5c74100740 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c72d9d640 1 -- 172.21.15.7:0/542986627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 msgr2=0x55774ddf7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddf7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c72d9d640 1 -- 172.21.15.7:0/542986627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 msgr2=0x55774ddefe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddefe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c72d9d640 1 -- 172.21.15.7:0/542986627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774dd1e000 con 0x55774ddf0440 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c7359e640 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddefe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.405+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddf7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.406+0000 7f5c7359e640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c600a2800 0x7f5c600a4cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c032380 tx=0x7f5c6c0324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.406+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf0440 0x55774ddf4f40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5c64066520 tx=0x7f5c6406e4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2a035be3e0e163 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.406+0000 7f5c6959a640 1 -- 172.21.15.7:0/542986627 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64003b30 con 0x55774ddf0440 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.406+0000 7f5c6959a640 1 -- 172.21.15.7:0/542986627 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6409be90 con 0x55774ddf0440 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.406+0000 7f5c6959a640 1 -- 172.21.15.7:0/542986627 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c640861e0 con 0x55774ddf0440 2024-03-19T00:47:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/542986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf0440 msgr2=0x55774ddf4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf0440 0x55774ddf4f40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5c64066520 tx=0x7f5c6406e4c0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/542986627 shutdown_connections 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddf7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf0440 0x55774ddf4f40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/542986627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddefe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/542986627 >> 172.21.15.7:0/542986627 conn(0x55774ddeafc0 msgr2=0x55774ddeb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/542986627 shutdown_connections 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/542986627 wait complete. 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- start start 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 Processor -- start 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774de01470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55774ddf8560 con 0x55774ddf0440 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55774dd10160 con 0x55774ddefa50 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55774ddf86e0 con 0x55774ddf55b0 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c7bc4a740 10 client.0 _ll_register_callbacks cb 0x55774dd4ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50980/0 (socket says 172.21.15.7:50980) 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c6a59c640 1 -- 172.21.15.7:0/4241523661 learned_addr learned my addr 172.21.15.7:0/4241523661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.407+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c7359e640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774de01470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c6a59c640 1 -- 172.21.15.7:0/4241523661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 msgr2=0x55774de01470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774de01470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c6a59c640 1 -- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 msgr2=0x55774ddf9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c6a59c640 1 -- 172.21.15.7:0/4241523661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774db327d0 con 0x55774ddf0440 2024-03-19T00:47:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c7359e640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774de01470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c012720 tx=0x7f5c5c012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c80f34159cac1c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/4241523661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c004030 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/4241523661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5c5c0041d0 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c6409be90 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/4241523661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c5c025050 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c7bc4a740 10 client.17580 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55774da683f0 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.408+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/4241523661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6409be90 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/4241523661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c5c003990 con 0x55774ddf0440 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c7bc4a740 10 client.17580 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c7bc4a740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c7bc4a740 20 client.17580 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de05ed0 0x55774de08310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c7bc4a740 4 client.17580 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55774dd61a40 con 0x55774de05ed0 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de05ed0 0x55774de08310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c5a7fc640 10 client.17580 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.409+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de05ed0 0x55774de08310 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6958b7c459b01ef5 server_cookie=eb67069fc42f4c34 in_seq=0 out_seq=0 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/4241523661 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55774dd61a40 con 0x55774de05ed0 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c5a7fc640 10 client.17580 handle_command_reply: tid=0 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 client.17580 shutdown 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 msgr2=0x55774ddfa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c012720 tx=0x7f5c5c012be0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 shutdown_connections 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774de01470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de05ed0 0x55774de08310 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/4241523661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c5a7fc640 0 client.17580 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 >> 172.21.15.7:0/4241523661 conn(0x55774ddeafc0 msgr2=0x55774ddea1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 shutdown_connections 2024-03-19T00:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/4241523661 wait complete. 2024-03-19T00:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.410+0000 7f5c7bc4a740 20 client.17580 trim_cache size 0 max 16384 2024-03-19T00:47:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 Processor -- start 2024-03-19T00:47:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 -- start start 2024-03-19T00:47:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf55b0 0x55774de01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741acb50 con 0x55774ddefa50 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741aa420 con 0x55774ddf55b0 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741a8270 con 0x55774ddf0440 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50990/0 (socket says 172.21.15.7:50990) 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 -- 172.21.15.7:0/811913615 learned_addr learned my addr 172.21.15.7:0/811913615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c7359e640 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf55b0 0x55774de01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 -- 172.21.15.7:0/811913615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf0440 msgr2=0x55774ddfa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 -- 172.21.15.7:0/811913615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf55b0 msgr2=0x55774de01430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf55b0 0x55774de01430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.428+0000 7f5c72d9d640 1 -- 172.21.15.7:0/811913615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774da280f0 con 0x55774ddefa50 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5c6409b490 tx=0x7f5c640c61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3586a87e61262a45 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c68d99640 1 -- 172.21.15.7:0/811913615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6409b2d0 con 0x55774ddefa50 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c68d99640 1 -- 172.21.15.7:0/811913615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6409be90 con 0x55774ddefa50 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c68d99640 1 -- 172.21.15.7:0/811913615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64097e00 con 0x55774ddefa50 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/811913615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 msgr2=0x55774ddf9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5c6409b490 tx=0x7f5c640c61e0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/811913615 shutdown_connections 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddf55b0 0x55774de01430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf0440 0x55774ddfa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/811913615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddefa50 0x55774ddf9f60 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/811913615 >> 172.21.15.7:0/811913615 conn(0x55774ddeafc0 msgr2=0x55774ddeb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/811913615 shutdown_connections 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.429+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/811913615 wait complete. 2024-03-19T00:47:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 -- start start 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 Processor -- start 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddfe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741a88e0 con 0x55774ddf0440 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741a9d50 con 0x55774ddefa50 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741a8fb0 con 0x55774ddf55b0 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7bc4a740 10 client.0 _ll_register_callbacks cb 0x7f5c5c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51006/0 (socket says 172.21.15.7:51006) 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 -- 172.21.15.7:0/215942243 learned_addr learned my addr 172.21.15.7:0/215942243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddfe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c7359e640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 -- 172.21.15.7:0/215942243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 msgr2=0x55774ddffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 -- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 msgr2=0x55774ddfe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddfe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 -- 172.21.15.7:0/215942243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774de00240 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c6a59c640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfec40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c003e00 tx=0x7f5c5c003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cdce33bc02f499 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.430+0000 7f5c5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/215942243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c022040 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/215942243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5c5c0052d0 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c6409be90 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/215942243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c5c037050 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 10 client.17586 fetch_fsmap learned FSMap version 27 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55774de01580 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/215942243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6409be90 con 0x55774ddf0440 2024-03-19T00:47:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/215942243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5c5c010430 con 0x55774ddf0440 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 10 client.17586 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 20 client.17586 populate_metadata read hostname 'smithi007' 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de6c850 0x55774de6ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 4 client.17586 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55774dd5bd90 con 0x55774de6c850 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.431+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de6c850 0x55774de6ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c5a7fc640 10 client.17586 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c72d9d640 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de6c850 0x55774de6ec70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c58734ff495b10a server_cookie=d49d8e009538b839 in_seq=0 out_seq=0 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c5a7fc640 1 -- 172.21.15.7:0/215942243 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55774dd5bd90 con 0x55774de6c850 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c5a7fc640 10 client.17586 handle_command_reply: tid=0 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 client.17586 shutdown 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 msgr2=0x55774ddfec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfec40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c003e00 tx=0x7f5c5c003940 comp rx=0 tx=0).stop 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 shutdown_connections 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55774ddf55b0 0x55774ddffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55774de6c850 0x55774de6ec70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55774ddf0440 0x55774ddfec40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 --2- 172.21.15.7:0/215942243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55774ddefa50 0x55774ddfe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c5a7fc640 0 client.17586 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 >> 172.21.15.7:0/215942243 conn(0x55774ddeafc0 msgr2=0x55774ddf9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 shutdown_connections 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 1 -- 172.21.15.7:0/215942243 wait complete. 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.432+0000 7f5c7bc4a740 20 client.17586 trim_cache size 0 max 16384 2024-03-19T00:47:40.440 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "5e9af347-af3f-4747-a793-920f18c80946": { 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "5e9af347-af3f-4747-a793-920f18c80946", 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:47:40.441 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c600a2800 msgr2=0x7f5c600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c600a2800 0x7f5c600a4cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c032380 tx=0x7f5c6c0324f0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 msgr2=0x7f5c74197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 0x7f5c74197a30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e9c0 tx=0x7f5c6400ee80 comp rx=0 tx=0).stop 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 shutdown_connections 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5c600a2800 0x7f5c600a4cc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741022f0 0x7f5c7419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c74101930 0x7f5c74197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 --2- 172.21.15.7:0/2136839823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c74100740 0x7f5c74197a30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:47:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 >> 172.21.15.7:0/2136839823 conn(0x7f5c740fbcd0 msgr2=0x7f5c74069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:47:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 shutdown_connections 2024-03-19T00:47:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:47:40.433+0000 7f5c79063640 1 -- 172.21.15.7:0/2136839823 wait complete. 2024-03-19T00:47:40.444 INFO:tasks.cephfs.filesystem:scrub status for tag:5e9af347-af3f-4747-a793-920f18c80946 - {'path': '/', 'tag': '5e9af347-af3f-4747-a793-920f18c80946', 'options': 'recursive,force'} 2024-03-19T00:47:41.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:40 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:47:41.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:40 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:41.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:40 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:47:41.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:40 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:41.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:40 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:47:41.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:40 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:41 smithi044 ceph-mon[66366]: pgmap v643: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:41 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:47:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:41 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:41 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:47:42.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:41 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:42.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:41 smithi007 ceph-mon[62854]: pgmap v643: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:42.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:41 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:47:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:41 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:41 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:47:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:41 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:41 smithi204 ceph-mon[66572]: pgmap v643: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:47:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:41 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:47:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:41 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:41 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:47:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:41 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:47:44.067 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:43 smithi204 ceph-mon[66572]: pgmap v644: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:44.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:43 smithi044 ceph-mon[66366]: pgmap v644: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:44.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:43 smithi007 ceph-mon[62854]: pgmap v644: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:47:46.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:45 smithi044 ceph-mon[66366]: pgmap v645: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:47:46.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:45 smithi007 ceph-mon[62854]: pgmap v645: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:47:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:45 smithi204 ceph-mon[66572]: pgmap v645: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:47:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:47:47.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:46 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:47:47.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:47:47.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:46 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:47:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:47:47.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:46 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:47:48.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:47 smithi044 ceph-mon[66366]: pgmap v646: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:48.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:47 smithi007 ceph-mon[62854]: pgmap v646: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:47 smithi204 ceph-mon[66572]: pgmap v646: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:49.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:48 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:47:49.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:48 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:47:49.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:48 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:47:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:49 smithi044 ceph-mon[66366]: pgmap v647: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:49 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:47:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:49 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:47:50.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:49 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:47:50.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:49 smithi007 ceph-mon[62854]: pgmap v647: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:50.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:49 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:47:50.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:49 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:47:50.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:49 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:47:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:49 smithi204 ceph-mon[66572]: pgmap v647: 129 pgs: 129 active+clean; 2.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:49 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:47:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:49 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:47:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:49 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:47:52.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:51 smithi044 ceph-mon[66366]: pgmap v648: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:52.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:51 smithi007 ceph-mon[62854]: pgmap v648: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:51 smithi204 ceph-mon[66572]: pgmap v648: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:47:54.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:53 smithi044 ceph-mon[66366]: pgmap v649: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:53 smithi007 ceph-mon[62854]: pgmap v649: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:54.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:53 smithi204 ceph-mon[66572]: pgmap v649: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:47:56.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:55 smithi044 ceph-mon[66366]: pgmap v650: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:47:56.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:55 smithi007 ceph-mon[62854]: pgmap v650: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:47:56.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:55 smithi204 ceph-mon[66572]: pgmap v650: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:47:58.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:57 smithi044 ceph-mon[66366]: pgmap v651: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:58.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:57 smithi007 ceph-mon[62854]: pgmap v651: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:47:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:57 smithi204 ceph-mon[66572]: pgmap v651: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:00.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:47:59 smithi044 ceph-mon[66366]: pgmap v652: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:00.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:47:59 smithi007 ceph-mon[62854]: pgmap v652: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:47:59 smithi204 ceph-mon[66572]: pgmap v652: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:02.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:01 smithi044 ceph-mon[66366]: pgmap v653: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:02.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:01 smithi007 ceph-mon[62854]: pgmap v653: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:02.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:01 smithi204 ceph-mon[66572]: pgmap v653: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:04.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:03 smithi044 ceph-mon[66366]: pgmap v654: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:04.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:03 smithi007 ceph-mon[62854]: pgmap v654: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:04.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:03 smithi204 ceph-mon[66572]: pgmap v654: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:06.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:05 smithi044 ceph-mon[66366]: pgmap v655: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 KiB/s wr, 14 op/s 2024-03-19T00:48:06.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:05 smithi007 ceph-mon[62854]: pgmap v655: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 KiB/s wr, 14 op/s 2024-03-19T00:48:06.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:05 smithi204 ceph-mon[66572]: pgmap v655: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 KiB/s wr, 14 op/s 2024-03-19T00:48:08.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:07 smithi044 ceph-mon[66366]: pgmap v656: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-03-19T00:48:08.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:07 smithi007 ceph-mon[62854]: pgmap v656: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-03-19T00:48:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:07 smithi204 ceph-mon[66572]: pgmap v656: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-03-19T00:48:10.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:09 smithi044 ceph-mon[66366]: pgmap v657: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.8 KiB/s wr, 9 op/s 2024-03-19T00:48:10.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:09 smithi007 ceph-mon[62854]: pgmap v657: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.8 KiB/s wr, 9 op/s 2024-03-19T00:48:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:09 smithi204 ceph-mon[66572]: pgmap v657: 129 pgs: 129 active+clean; 2.8 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.8 KiB/s wr, 9 op/s 2024-03-19T00:48:10.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:48:10.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:48:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 -- 172.21.15.7:0/3299756814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8100740 msgr2=0x7f0ba8100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 --2- 172.21.15.7:0/3299756814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8100740 0x7f0ba8100b60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba002bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 -- 172.21.15.7:0/3299756814 shutdown_connections 2024-03-19T00:48:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 --2- 172.21.15.7:0/3299756814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 0x7f0ba8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 --2- 172.21.15.7:0/3299756814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ba8101930 0x7f0ba8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 --2- 172.21.15.7:0/3299756814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8100740 0x7f0ba8100b60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 -- 172.21.15.7:0/3299756814 >> 172.21.15.7:0/3299756814 conn(0x7f0ba80fbcb0 msgr2=0x7f0ba80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 -- 172.21.15.7:0/3299756814 shutdown_connections 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.561+0000 7f0bacc36640 1 -- 172.21.15.7:0/3299756814 wait complete. 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 Processor -- start 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 -- start start 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ba8100740 0x7f0ba8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 0x7f0ba8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 0x7f0ba819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba810c270 con 0x7f0ba8101930 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba810c0f0 con 0x7f0ba8100740 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0bacc36640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba810c3f0 con 0x7f0ba81022f0 2024-03-19T00:48:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0ba5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 0x7f0ba8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0ba5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 0x7f0ba8197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40326/0 (socket says 172.21.15.7:40326) 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/159116515 learned_addr learned my addr 172.21.15.7:0/159116515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0ba659c640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ba8100740 0x7f0ba8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.562+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 0x7f0ba819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/159116515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 msgr2=0x7f0ba819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 0x7f0ba819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/159116515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ba8100740 msgr2=0x7f0ba8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ba8100740 0x7f0ba8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/159116515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba819f690 con 0x7f0ba8101930 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 0x7f0ba819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 0x7f0ba8197fb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e9f0 tx=0x7f0b9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/159116515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000cd80 con 0x7f0ba8101930 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba819f980 con 0x7f0ba8101930 2024-03-19T00:48:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/159116515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b900108e0 con 0x7f0ba8101930 2024-03-19T00:48:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/159116515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90018920 con 0x7f0ba8101930 2024-03-19T00:48:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.563+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba819ff20 con 0x7f0ba8101930 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.565+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/159116515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0b90010a90 con 0x7f0ba8101930 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 Processor -- start 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 -- start start 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0b9f7fe640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b940a2800 0x7f0b940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b896e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b897440 0x56301b89bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b89c5b0 0x56301b89ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56301b7fb500 con 0x56301b89c5b0 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56301b7a3be0 con 0x56301b896a50 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0baf829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56301b7f75a0 con 0x56301b897440 2024-03-19T00:48:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b940a2800 0x7f0b940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/159116515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0b900cf1c0 con 0x7f0ba8101930 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b896e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b896e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47880/0 (socket says 172.21.15.7:47880) 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba659c640 1 -- 172.21.15.7:0/3199108757 learned_addr learned my addr 172.21.15.7:0/3199108757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b89c5b0 0x56301b89ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b897440 0x56301b89bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba659c640 1 -- 172.21.15.7:0/3199108757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b897440 msgr2=0x56301b89bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.566+0000 7f0ba659c640 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b897440 0x56301b89bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0ba659c640 1 -- 172.21.15.7:0/3199108757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b89c5b0 msgr2=0x56301b89ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0ba659c640 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b89c5b0 0x56301b89ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0ba659c640 1 -- 172.21.15.7:0/3199108757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301b7c5000 con 0x56301b896a50 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b940a2800 0x7f0b940a4cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005c40 tx=0x7f0b98009e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0ba659c640 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b896e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5018777aac3791aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0b9cff9640 1 -- 172.21.15.7:0/3199108757 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0045020 con 0x56301b896a50 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0b9cff9640 1 -- 172.21.15.7:0/3199108757 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba0004040 con 0x56301b896a50 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.567+0000 7f0b9cff9640 1 -- 172.21.15.7:0/3199108757 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0005500 con 0x56301b896a50 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- 172.21.15.7:0/3199108757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 msgr2=0x56301b896e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b896e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba0009560 comp rx=0 tx=0).stop 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- 172.21.15.7:0/3199108757 shutdown_connections 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b89c5b0 0x56301b89ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b897440 0x56301b89bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- 172.21.15.7:0/3199108757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b896e70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- 172.21.15.7:0/3199108757 >> 172.21.15.7:0/3199108757 conn(0x56301b891fc0 msgr2=0x56301b8923c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- 172.21.15.7:0/3199108757 shutdown_connections 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- 172.21.15.7:0/3199108757 wait complete. 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- start start 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 Processor -- start 2024-03-19T00:48:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b896a50 0x56301b8a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b89c5b0 0x56301b8a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56301b89f500 con 0x56301b897440 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56301b7b7160 con 0x56301b89c5b0 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56301b89f680 con 0x56301b896a50 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.568+0000 7f0baf829740 10 client.0 _ll_register_callbacks cb 0x56301b7f3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40336/0 (socket says 172.21.15.7:40336) 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/1724689852 learned_addr learned my addr 172.21.15.7:0/1724689852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba659c640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b896a50 0x56301b8a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b89c5b0 0x56301b8a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/1724689852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b896a50 msgr2=0x56301b8a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b896a50 0x56301b8a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b89c5b0 msgr2=0x56301b8a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b89c5b0 0x56301b8a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/1724689852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301b5d97d0 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a1380 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000cfd0 tx=0x7f0b9009b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1babe5a8f982b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0b8dffb640 1 -- 172.21.15.7:0/1724689852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900b4060 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0b8dffb640 1 -- 172.21.15.7:0/1724689852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0b900860b0 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba0004040 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0b8dffb640 1 -- 172.21.15.7:0/1724689852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b900046c0 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0baf829740 10 client.17598 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56301b50f3f0 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.569+0000 7f0b8dffb640 1 -- 172.21.15.7:0/1724689852 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba0004040 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0b8dffb640 1 -- 172.21.15.7:0/1724689852 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0b90004a40 con 0x56301b897440 2024-03-19T00:48:10.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0baf829740 10 client.17598 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0baf829740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0baf829740 20 client.17598 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0baf829740 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b8acdc0 0x56301b8af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0baf829740 4 client.17598 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56301b808a40 con 0x56301b8acdc0 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0ba659c640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b8acdc0 0x56301b8af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0b8dffb640 10 client.17598 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.570+0000 7f0ba659c640 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b8acdc0 0x56301b8af240 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25520272c0f5f62d server_cookie=b8ed78b6f96fe55 in_seq=0 out_seq=0 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0b8dffb640 1 -- 172.21.15.7:0/1724689852 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56301b808a40 con 0x56301b8acdc0 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0b8dffb640 10 client.17598 handle_command_reply: tid=0 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 client.17598 shutdown 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 msgr2=0x56301b8a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a1380 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000cfd0 tx=0x7f0b9009b100 comp rx=0 tx=0).stop 2024-03-19T00:48:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 shutdown_connections 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b89c5b0 0x56301b8a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b8acdc0 0x56301b8af240 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a1380 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 --2- 172.21.15.7:0/1724689852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b896a50 0x56301b8a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0b8dffb640 0 client.17598 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 >> 172.21.15.7:0/1724689852 conn(0x56301b891fc0 msgr2=0x56301b8911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 shutdown_connections 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 1 -- 172.21.15.7:0/1724689852 wait complete. 2024-03-19T00:48:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.571+0000 7f0baf829740 20 client.17598 trim_cache size 0 max 16384 2024-03-19T00:48:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 Processor -- start 2024-03-19T00:48:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 -- start start 2024-03-19T00:48:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 0x56301b890950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b897440 0x56301b8a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b890e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81acac0 con 0x56301b896a50 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81aa390 con 0x56301b897440 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0baf829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81a81e0 con 0x56301b89c5b0 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 0x56301b890950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 0x56301b890950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40352/0 (socket says 172.21.15.7:40352) 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 -- 172.21.15.7:0/1612568995 learned_addr learned my addr 172.21.15.7:0/1612568995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b897440 0x56301b8a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b890e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 -- 172.21.15.7:0/1612568995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 msgr2=0x56301b890e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b890e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 -- 172.21.15.7:0/1612568995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b897440 msgr2=0x56301b8a8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b897440 0x56301b8a8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 -- 172.21.15.7:0/1612568995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301b4cf0f0 con 0x56301b896a50 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.589+0000 7f0ba659c640 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 0x56301b890950 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0ba002c840 tx=0x7f0ba0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770935641efd5d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0b8ffff640 1 -- 172.21.15.7:0/1612568995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0045040 con 0x56301b896a50 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0b8ffff640 1 -- 172.21.15.7:0/1612568995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba00055a0 con 0x56301b896a50 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0b8ffff640 1 -- 172.21.15.7:0/1612568995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0041c00 con 0x56301b896a50 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 -- 172.21.15.7:0/1612568995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 msgr2=0x56301b890950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 0x56301b890950 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0ba002c840 tx=0x7f0ba0000bc0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 -- 172.21.15.7:0/1612568995 shutdown_connections 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b890e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b897440 0x56301b8a8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 --2- 172.21.15.7:0/1612568995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b896a50 0x56301b890950 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 -- 172.21.15.7:0/1612568995 >> 172.21.15.7:0/1612568995 conn(0x56301b891fc0 msgr2=0x56301b8a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 -- 172.21.15.7:0/1612568995 shutdown_connections 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 -- 172.21.15.7:0/1612568995 wait complete. 2024-03-19T00:48:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.590+0000 7f0baf829740 1 -- start start 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 Processor -- start 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b8a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b8a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81a8850 con 0x56301b897440 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81a9cc0 con 0x56301b896a50 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81a8f20 con 0x56301b89c5b0 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0baf829740 10 client.0 _ll_register_callbacks cb 0x7f0b90009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b8a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40368/0 (socket says 172.21.15.7:40368) 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/3162476137 learned_addr learned my addr 172.21.15.7:0/3162476137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba6d9d640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b8a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/3162476137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 msgr2=0x56301b8a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b8a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 msgr2=0x56301b8a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b8a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 -- 172.21.15.7:0/3162476137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301b8a7320 con 0x56301b897440 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba659c640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b8a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0ba5d9b640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a5d20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0b9009b730 tx=0x7f0b90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410beb4566d79a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.591+0000 7f0b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0b8dffb640 1 -- 172.21.15.7:0/3162476137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9009f940 con 0x56301b897440 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0b8dffb640 1 -- 172.21.15.7:0/3162476137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0b90010430 con 0x56301b897440 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba00055a0 con 0x56301b897440 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0b8dffb640 1 -- 172.21.15.7:0/3162476137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b900b4020 con 0x56301b897440 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 10 client.17604 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56301b8a8630 con 0x56301b897440 2024-03-19T00:48:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0b8dffb640 1 -- 172.21.15.7:0/3162476137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba00055a0 con 0x56301b897440 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0b8dffb640 1 -- 172.21.15.7:0/3162476137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0b900974f0 con 0x56301b897440 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 10 client.17604 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 20 client.17604 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b9138d0 0x56301b915cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 4 client.17604 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56301b802d90 con 0x56301b9138d0 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.592+0000 7f0ba659c640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b9138d0 0x56301b915cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0ba659c640 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b9138d0 0x56301b915cf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328345d746b02de8 server_cookie=e2b67d21eacc0ade in_seq=0 out_seq=0 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0b8dffb640 10 client.17604 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0b8dffb640 1 -- 172.21.15.7:0/3162476137 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56301b802d90 con 0x56301b9138d0 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0b8dffb640 10 client.17604 handle_command_reply: tid=0 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 client.17604 shutdown 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 msgr2=0x56301b8a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a5d20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0b9009b730 tx=0x7f0b90000bc0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 shutdown_connections 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56301b89c5b0 0x56301b8a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x56301b9138d0 0x56301b915cf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56301b897440 0x56301b8a5d20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 --2- 172.21.15.7:0/3162476137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56301b896a50 0x56301b8a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 >> 172.21.15.7:0/3162476137 conn(0x56301b891fc0 msgr2=0x56301b895220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0b8dffb640 0 client.17604 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 shutdown_connections 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 1 -- 172.21.15.7:0/3162476137 wait complete. 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.593+0000 7f0baf829740 20 client.17604 trim_cache size 0 max 16384 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:48:10.601 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:48:10.602 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:48:10.602 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:48:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b940a2800 msgr2=0x7f0b940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b940a2800 0x7f0b940a4cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005c40 tx=0x7f0b98009e50 comp rx=0 tx=0).stop 2024-03-19T00:48:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 msgr2=0x7f0ba8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 0x7f0ba8197fb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e9f0 tx=0x7f0b9000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 shutdown_connections 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0b940a2800 0x7f0b940a4cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba81022f0 0x7f0ba819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ba8101930 0x7f0ba8197fb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 --2- 172.21.15.7:0/159116515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ba8100740 0x7f0ba8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 >> 172.21.15.7:0/159116515 conn(0x7f0ba80fbcb0 msgr2=0x7f0ba80fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 shutdown_connections 2024-03-19T00:48:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.594+0000 7f0bacc36640 1 -- 172.21.15.7:0/159116515 wait complete. 2024-03-19T00:48:10.606 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:48:10.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:48:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 -- 172.21.15.7:0/2374713424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990102aa0 msgr2=0x7f0990102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 --2- 172.21.15.7:0/2374713424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990102aa0 0x7f0990102f20 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0978009900 tx=0x7f097802bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 -- 172.21.15.7:0/2374713424 shutdown_connections 2024-03-19T00:48:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 --2- 172.21.15.7:0/2374713424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0990103460 0x7f0990109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 --2- 172.21.15.7:0/2374713424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990102aa0 0x7f0990102f20 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 --2- 172.21.15.7:0/2374713424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09901018b0 0x7f0990101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 -- 172.21.15.7:0/2374713424 >> 172.21.15.7:0/2374713424 conn(0x7f09900fce40 msgr2=0x7f09900ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 -- 172.21.15.7:0/2374713424 shutdown_connections 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.707+0000 7f0994d37640 1 -- 172.21.15.7:0/2374713424 wait complete. 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 Processor -- start 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 -- start start 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09901018b0 0x7f09900755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0990102aa0 0x7f0990073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 0x7f0990074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f099010d350 con 0x7f0990103460 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f099010d1d0 con 0x7f0990102aa0 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0994d37640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f099010d4d0 con 0x7f09901018b0 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f098ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 0x7f0990074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f098ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 0x7f0990074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40386/0 (socket says 172.21.15.7:40386) 2024-03-19T00:48:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f098ed9d640 1 -- 172.21.15.7:0/1134538632 learned_addr learned my addr 172.21.15.7:0/1134538632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f0987fff640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0990102aa0 0x7f0990073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.708+0000 7f098e59c640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09901018b0 0x7f09900755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f098ed9d640 1 -- 172.21.15.7:0/1134538632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09901018b0 msgr2=0x7f09900755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f098ed9d640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09901018b0 0x7f09900755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f098ed9d640 1 -- 172.21.15.7:0/1134538632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0990102aa0 msgr2=0x7f0990073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f098ed9d640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0990102aa0 0x7f0990073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f098ed9d640 1 -- 172.21.15.7:0/1134538632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09900749e0 con 0x7f0990103460 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f098ed9d640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 0x7f0990074150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f098800eb10 tx=0x7f098800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988004b20 con 0x7f0990103460 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09901a1b40 con 0x7f0990103460 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0988004cc0 con 0x7f0990103460 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988010690 con 0x7f0990103460 2024-03-19T00:48:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.709+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09901a2060 con 0x7f0990103460 2024-03-19T00:48:10.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.710+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0990068af0 con 0x7f0990103460 2024-03-19T00:48:10.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.711+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0988005000 con 0x7f0990103460 2024-03-19T00:48:10.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.712+0000 7f09877fe640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f097c0a2800 0x7f097c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:10.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.712+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f09880cf5e0 con 0x7f0990103460 2024-03-19T00:48:10.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.712+0000 7f098e59c640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f097c0a2800 0x7f097c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:10.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.712+0000 7f098e59c640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f097c0a2800 0x7f097c0a4cc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09800095f0 tx=0x7f0980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:10.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.716+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0988053740 con 0x7f0990103460 2024-03-19T00:48:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.944+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0990108c50 con 0x7f0990103460 2024-03-19T00:48:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.945+0000 7f09877fe640 1 -- 172.21.15.7:0/1134538632 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f098809c2c0 con 0x7f0990103460 2024-03-19T00:48:10.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:48:10.947 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:48:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:48:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f097c0a2800 msgr2=0x7f097c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f097c0a2800 0x7f097c0a4cc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09800095f0 tx=0x7f0980005f90 comp rx=0 tx=0).stop 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 msgr2=0x7f0990074150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 0x7f0990074150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f098800eb10 tx=0x7f098800c2d0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 shutdown_connections 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f097c0a2800 0x7f097c0a4cc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990103460 0x7f0990074150 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0990102aa0 0x7f0990073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 --2- 172.21.15.7:0/1134538632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09901018b0 0x7f09900755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.947+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 >> 172.21.15.7:0/1134538632 conn(0x7f09900fce40 msgr2=0x7f0990105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.948+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 shutdown_connections 2024-03-19T00:48:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:10.948+0000 7f0994d37640 1 -- 172.21.15.7:0/1134538632 wait complete. 2024-03-19T00:48:10.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:48:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3356870708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 msgr2=0x7fe7e0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3356870708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0100b60 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009900 tx=0x7fe7d802bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3356870708 shutdown_connections 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3356870708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e01022f0 0x7fe7e0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3356870708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101930 0x7fe7e0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3356870708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0100b60 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3356870708 >> 172.21.15.7:0/3356870708 conn(0x7fe7e00fbc90 msgr2=0x7fe7e00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3356870708 shutdown_connections 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.063+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3356870708 wait complete. 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 Processor -- start 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 -- start start 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101930 0x7fe7e0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e01022f0 0x7fe7e019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e010d390 con 0x7fe7e0100740 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e010d210 con 0x7fe7e0101930 2024-03-19T00:48:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e6c75640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e010d510 con 0x7fe7e01022f0 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40404/0 (socket says 172.21.15.7:40404) 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 -- 172.21.15.7:0/3415686500 learned_addr learned my addr 172.21.15.7:0/3415686500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7dffff640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101930 0x7fe7e0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e5212640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e01022f0 0x7fe7e019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 -- 172.21.15.7:0/3415686500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e01022f0 msgr2=0x7fe7e019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e01022f0 0x7fe7e019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 -- 172.21.15.7:0/3415686500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101930 msgr2=0x7fe7e0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101930 0x7fe7e0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.064+0000 7fe7e4a11640 1 -- 172.21.15.7:0/3415686500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e019f6a0 con 0x7fe7e0100740 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.065+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0197a80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8000c00 tx=0x7fe7d8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.065+0000 7fe7ddffb640 1 -- 172.21.15.7:0/3415686500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8043020 con 0x7fe7e0100740 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.065+0000 7fe7ddffb640 1 -- 172.21.15.7:0/3415686500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d8004350 con 0x7fe7e0100740 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.065+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e019f930 con 0x7fe7e0100740 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.065+0000 7fe7ddffb640 1 -- 172.21.15.7:0/3415686500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8003b80 con 0x7fe7e0100740 2024-03-19T00:48:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.065+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e019fe80 con 0x7fe7e0100740 2024-03-19T00:48:11.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.067+0000 7fe7ddffb640 1 -- 172.21.15.7:0/3415686500 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe7d8046020 con 0x7fe7e0100740 2024-03-19T00:48:11.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.067+0000 7fe7e9859740 1 Processor -- start 2024-03-19T00:48:11.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.067+0000 7fe7e9859740 1 -- start start 2024-03-19T00:48:11.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73cdf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73cdfce0 0x564f73ce47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 0x564f73ce72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e9859740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564f73c434b0 con 0x564f73ce4e50 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e9859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564f73bebd70 con 0x564f73cdfce0 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e9859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564f73c3f550 con 0x564f73cdf2f0 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7ddffb640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7cc0a2800 0x7fe7cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 0x564f73ce72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e4a11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73cdfce0 0x564f73ce47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 0x564f73ce72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40410/0 (socket says 172.21.15.7:40410) 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 -- 172.21.15.7:0/860964285 learned_addr learned my addr 172.21.15.7:0/860964285 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e5212640 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73cdf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e5212640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7cc0a2800 0x7fe7cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7ddffb640 1 -- 172.21.15.7:0/3415686500 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7d80f0f40 con 0x7fe7e0100740 2024-03-19T00:48:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 -- 172.21.15.7:0/860964285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 msgr2=0x564f73cdf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73cdf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 -- 172.21.15.7:0/860964285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73cdfce0 msgr2=0x564f73ce47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73cdfce0 0x564f73ce47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 -- 172.21.15.7:0/860964285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f73c0e210 con 0x564f73ce4e50 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7dffff640 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 0x564f73ce72e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0011470 tx=0x7fe7d0011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acde5f55c3130db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7e5212640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7cc0a2800 0x7fe7cc0a4cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x564f73ce0540 tx=0x7fe7c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7d6ffd640 1 -- 172.21.15.7:0/860964285 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001e070 con 0x564f73ce4e50 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7d6ffd640 1 -- 172.21.15.7:0/860964285 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d0003c30 con 0x564f73ce4e50 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.068+0000 7fe7d6ffd640 1 -- 172.21.15.7:0/860964285 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000f880 con 0x564f73ce4e50 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 -- 172.21.15.7:0/860964285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 msgr2=0x564f73ce72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 0x564f73ce72e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0011470 tx=0x7fe7d0011870 comp rx=0 tx=0).stop 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 -- 172.21.15.7:0/860964285 shutdown_connections 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73ce4e50 0x564f73ce72e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73cdfce0 0x564f73ce47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 --2- 172.21.15.7:0/860964285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73cdf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 -- 172.21.15.7:0/860964285 >> 172.21.15.7:0/860964285 conn(0x564f73cda860 msgr2=0x564f73cdac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 -- 172.21.15.7:0/860964285 shutdown_connections 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 -- 172.21.15.7:0/860964285 wait complete. 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.069+0000 7fe7e9859740 1 -- start start 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 Processor -- start 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ce9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ce9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564f73ce7e00 con 0x564f73cdfce0 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564f73bff310 con 0x564f73ce4e50 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564f73ce7f80 con 0x564f73cdf2f0 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e9859740 10 client.0 _ll_register_callbacks cb 0x564f73bf32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ce9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ce9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40418/0 (socket says 172.21.15.7:40418) 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 -- 172.21.15.7:0/355179874 learned_addr learned my addr 172.21.15.7:0/355179874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ce9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7e5212640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 -- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 msgr2=0x564f73ce9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ce9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 -- 172.21.15.7:0/355179874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 msgr2=0x564f73cf0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 -- 172.21.15.7:0/355179874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f739573f0 con 0x564f73cdfce0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7dffff640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ce9cc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0011440 tx=0x7fe7d0004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96df9ba8894c9e82 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.070+0000 7fe7bffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7bffff640 1 -- 172.21.15.7:0/355179874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0028030 con 0x564f73cdfce0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7bffff640 1 -- 172.21.15.7:0/355179874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7d0003a30 con 0x564f73cdfce0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d0003c30 con 0x564f73cdfce0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7bffff640 1 -- 172.21.15.7:0/355179874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7d0032050 con 0x564f73cdfce0 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 10 client.17622 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:11.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f73a217d0 con 0x564f73cdfce0 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7bffff640 1 -- 172.21.15.7:0/355179874 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d0003c30 con 0x564f73cdfce0 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7bffff640 1 -- 172.21.15.7:0/355179874 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe7d0021030 con 0x564f73cdfce0 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 10 client.17622 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 10 client.17622 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 20 client.17622 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73cf5340 0x564f73cf77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 4 client.17622 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f73ca13e0 con 0x564f73cf5340 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.071+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73cf5340 0x564f73cf77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.072+0000 7fe7bffff640 10 client.17622 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.072+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73cf5340 0x564f73cf77e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f82fbf122c978390 server_cookie=3e1eba26ae24874b in_seq=0 out_seq=0 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.072+0000 7fe7bffff640 1 -- 172.21.15.7:0/355179874 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564f73ca13e0 con 0x564f73cf5340 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.072+0000 7fe7bffff640 10 client.17622 handle_command_reply: tid=0 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.072+0000 7fe7e9859740 1 client.17622 shutdown 2024-03-19T00:48:11.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 msgr2=0x564f73ce9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ce9cc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0011440 tx=0x7fe7d0004f00 comp rx=0 tx=0).stop 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 shutdown_connections 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73cf5340 0x564f73cf77e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ce9cc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7bffff640 0 client.17622 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 --2- 172.21.15.7:0/355179874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ce9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 >> 172.21.15.7:0/355179874 conn(0x564f73cda860 msgr2=0x564f73cd9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 shutdown_connections 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 1 -- 172.21.15.7:0/355179874 wait complete. 2024-03-19T00:48:11.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.073+0000 7fe7e9859740 20 client.17622 trim_cache size 0 max 16384 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 Processor -- start 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 -- start start 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 0x564f73ce1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdfce0 0x564f73ce1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01acb20 con 0x564f73cdf2f0 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01aa3f0 con 0x564f73ce4e50 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e9859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01a8240 con 0x564f73cdfce0 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e4a11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 0x564f73ce1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e4a11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 0x564f73ce1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40424/0 (socket says 172.21.15.7:40424) 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e4a11640 1 -- 172.21.15.7:0/709772144 learned_addr learned my addr 172.21.15.7:0/709772144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7e5212640 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.086+0000 7fe7dffff640 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdfce0 0x564f73ce1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7e4a11640 1 -- 172.21.15.7:0/709772144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdfce0 msgr2=0x564f73ce1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdfce0 0x564f73ce1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7e4a11640 1 -- 172.21.15.7:0/709772144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 msgr2=0x564f73cf0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7e4a11640 1 -- 172.21.15.7:0/709772144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f73cb0880 con 0x564f73cdf2f0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 0x564f73ce1140 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80bd4b0 tx=0x7fe7d8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eef76fafb1d4420 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7d67fc640 1 -- 172.21.15.7:0/709772144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80bd060 con 0x564f73cdf2f0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7d67fc640 1 -- 172.21.15.7:0/709772144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d80bdc20 con 0x564f73cdf2f0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.087+0000 7fe7d67fc640 1 -- 172.21.15.7:0/709772144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80bce80 con 0x564f73cdf2f0 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- 172.21.15.7:0/709772144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 msgr2=0x564f73ce1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 0x564f73ce1140 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80bd4b0 tx=0x7fe7d8041040 comp rx=0 tx=0).stop 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- 172.21.15.7:0/709772144 shutdown_connections 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73cf0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdfce0 0x564f73ce1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- 172.21.15.7:0/709772144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdf2f0 0x564f73ce1140 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- 172.21.15.7:0/709772144 >> 172.21.15.7:0/709772144 conn(0x564f73cda860 msgr2=0x564f73cdac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- 172.21.15.7:0/709772144 shutdown_connections 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- 172.21.15.7:0/709772144 wait complete. 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- start start 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 Processor -- start 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ceea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ceef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73ceff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01a88b0 con 0x564f73cdfce0 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01a9d20 con 0x564f73ce4e50 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01a8f80 con 0x564f73cdf2f0 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e9859740 10 client.0 _ll_register_callbacks cb 0x7fe7d001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ceef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ceef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40432/0 (socket says 172.21.15.7:40432) 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7dffff640 1 -- 172.21.15.7:0/516514872 learned_addr learned my addr 172.21.15.7:0/516514872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e5212640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73ceff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.088+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ceea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7dffff640 1 -- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 msgr2=0x564f73ceea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7dffff640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ceea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7dffff640 1 -- 172.21.15.7:0/516514872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 msgr2=0x564f73ceff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7dffff640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73ceff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7dffff640 1 -- 172.21.15.7:0/516514872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f73cf0570 con 0x564f73cdfce0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7dffff640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ceef70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0004ed0 tx=0x7fe7d00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd6c9fe9d765036 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7bffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7bffff640 1 -- 172.21.15.7:0/516514872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0028040 con 0x564f73cdfce0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7bffff640 1 -- 172.21.15.7:0/516514872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7d0003970 con 0x564f73cdfce0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d80bdc20 con 0x564f73cdfce0 2024-03-19T00:48:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7bffff640 1 -- 172.21.15.7:0/516514872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7d001e040 con 0x564f73cdfce0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7e9859740 10 client.17628 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f73d5bd60 con 0x564f73cdfce0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7bffff640 1 -- 172.21.15.7:0/516514872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d80bdc20 con 0x564f73cdfce0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7bffff640 1 -- 172.21.15.7:0/516514872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe7d0021050 con 0x564f73cdfce0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7e9859740 10 client.17628 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7e9859740 10 client.17628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.089+0000 7fe7e9859740 20 client.17628 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73d5c5f0 0x564f73d5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 4 client.17628 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f73c509f0 con 0x564f73d5c5f0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73d5c5f0 0x564f73d5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7bffff640 10 client.17628 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e4a11640 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73d5c5f0 0x564f73d5ea10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=592d94bb1a4a1120 server_cookie=4ea6db549e78fb33 in_seq=0 out_seq=0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7bffff640 1 -- 172.21.15.7:0/516514872 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f73c509f0 con 0x564f73d5c5f0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7bffff640 10 client.17628 handle_command_reply: tid=0 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 client.17628 shutdown 2024-03-19T00:48:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 msgr2=0x564f73ceef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ceef70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0004ed0 tx=0x7fe7d00049d0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 shutdown_connections 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x564f73d5c5f0 0x564f73d5ea10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f73ce4e50 0x564f73ceff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f73cdfce0 0x564f73ceef70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7bffff640 0 client.17628 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 --2- 172.21.15.7:0/516514872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f73cdf2f0 0x564f73ceea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 >> 172.21.15.7:0/516514872 conn(0x564f73cda860 msgr2=0x564f73ce1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 shutdown_connections 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.090+0000 7fe7e9859740 1 -- 172.21.15.7:0/516514872 wait complete. 2024-03-19T00:48:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e9859740 20 client.17628 trim_cache size 0 max 16384 2024-03-19T00:48:11.099 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:48:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7cc0a2800 msgr2=0x7fe7cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7cc0a2800 0x7fe7cc0a4cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x564f73ce0540 tx=0x7fe7c8005f90 comp rx=0 tx=0).stop 2024-03-19T00:48:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 msgr2=0x7fe7e0197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0197a80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8000c00 tx=0x7fe7d8005dc0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 shutdown_connections 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe7cc0a2800 0x7fe7cc0a4cc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7e01022f0 0x7fe7e019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101930 0x7fe7e0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 --2- 172.21.15.7:0/3415686500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100740 0x7fe7e0197a80 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 >> 172.21.15.7:0/3415686500 conn(0x7fe7e00fbc90 msgr2=0x7fe7e0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 shutdown_connections 2024-03-19T00:48:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.091+0000 7fe7e6c75640 1 -- 172.21.15.7:0/3415686500 wait complete. 2024-03-19T00:48:11.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:48:11.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 -- 172.21.15.7:0/2953099706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 msgr2=0x7f4ff8108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2953099706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff8108b70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009ca0 tx=0x7f4ff002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 -- 172.21.15.7:0/2953099706 shutdown_connections 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2953099706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff8108b70 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2953099706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8101920 0x7f4ff8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2953099706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff8100730 0x7f4ff8100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 -- 172.21.15.7:0/2953099706 >> 172.21.15.7:0/2953099706 conn(0x7f4ff80fbc80 msgr2=0x7f4ff80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 -- 172.21.15.7:0/2953099706 shutdown_connections 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.206+0000 7f4ffe036640 1 -- 172.21.15.7:0/2953099706 wait complete. 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 Processor -- start 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 -- start start 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8100730 0x7f4ff806aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff8101920 0x7f4ff819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810c260 con 0x7f4ff81022e0 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810c0e0 con 0x7f4ff8101920 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ffe036640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810c3e0 con 0x7f4ff8100730 2024-03-19T00:48:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40446/0 (socket says 172.21.15.7:40446) 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 -- 172.21.15.7:0/2808628406 learned_addr learned my addr 172.21.15.7:0/2808628406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8100730 0x7f4ff806aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff8101920 0x7f4ff819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 -- 172.21.15.7:0/2808628406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8100730 msgr2=0x7f4ff806aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8100730 0x7f4ff806aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 -- 172.21.15.7:0/2808628406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff8101920 msgr2=0x7f4ff819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff8101920 0x7f4ff819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff7fff640 1 -- 172.21.15.7:0/2808628406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff819f640 con 0x7f4ff81022e0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.207+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8100730 0x7f4ff806aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.208+0000 7f4ff7fff640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff819ed80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4ff002c700 tx=0x7f4ff00328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.208+0000 7f4ff4ff9640 1 -- 172.21.15.7:0/2808628406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0041020 con 0x7f4ff81022e0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.208+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff819f8d0 con 0x7f4ff81022e0 2024-03-19T00:48:11.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.208+0000 7f4ff4ff9640 1 -- 172.21.15.7:0/2808628406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ff0003e20 con 0x7f4ff81022e0 2024-03-19T00:48:11.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.208+0000 7f4ff4ff9640 1 -- 172.21.15.7:0/2808628406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0039040 con 0x7f4ff81022e0 2024-03-19T00:48:11.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.208+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff819fdc0 con 0x7f4ff81022e0 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f4ff4ff9640 1 -- 172.21.15.7:0/2808628406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f4ff0044020 con 0x7f4ff81022e0 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 Processor -- start 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 -- start start 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86d3e50 0x558ee86d62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558ee86324b0 con 0x558ee86d3e50 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ee85dad70 con 0x558ee86cece0 2024-03-19T00:48:11.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.210+0000 7f5000c1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ee862e550 con 0x558ee86ce2f0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86d3e50 0x558ee86d62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47984/0 (socket says 172.21.15.7:47984) 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 -- 172.21.15.7:0/504601105 learned_addr learned my addr 172.21.15.7:0/504601105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff4ff9640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4fe00a2800 0x7f4fe00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff7fff640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4fe00a2800 0x7f4fe00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 -- 172.21.15.7:0/504601105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 msgr2=0x558ee86ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 -- 172.21.15.7:0/504601105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86d3e50 msgr2=0x558ee86d62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86d3e50 0x558ee86d62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 -- 172.21.15.7:0/504601105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee85fd210 con 0x558ee86cece0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff4ff9640 1 -- 172.21.15.7:0/2808628406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4ff00f0f50 con 0x7f4ff81022e0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d37e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80114f0 tx=0x7f4fe80119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48b8193f3a5ed59 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4ff7fff640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4fe00a2800 0x7f4fe00a4cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00bd290 tx=0x7f4ff00a8040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4fddffb640 1 -- 172.21.15.7:0/504601105 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801e070 con 0x558ee86cece0 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4fddffb640 1 -- 172.21.15.7:0/504601105 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe8011250 con 0x558ee86cece0 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.211+0000 7f4fddffb640 1 -- 172.21.15.7:0/504601105 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8019930 con 0x558ee86cece0 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 -- 172.21.15.7:0/504601105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 msgr2=0x558ee86d37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d37e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80114f0 tx=0x7f4fe80119b0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 -- 172.21.15.7:0/504601105 shutdown_connections 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86d3e50 0x558ee86d62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d37e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 --2- 172.21.15.7:0/504601105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 -- 172.21.15.7:0/504601105 >> 172.21.15.7:0/504601105 conn(0x558ee86c9860 msgr2=0x558ee86c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 -- 172.21.15.7:0/504601105 shutdown_connections 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 -- 172.21.15.7:0/504601105 wait complete. 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.212+0000 7f5000c1d740 1 -- start start 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 Processor -- start 2024-03-19T00:48:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 0x558ee86d8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558ee86d6e00 con 0x558ee86ce2f0 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ee85ee310 con 0x558ee86cece0 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ee86d6f80 con 0x558ee86d3e50 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f5000c1d740 10 client.0 _ll_register_callbacks cb 0x558ee85e22d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 0x558ee86d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 0x558ee86d8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40466/0 (socket says 172.21.15.7:40466) 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff6ffd640 1 -- 172.21.15.7:0/3796513729 learned_addr learned my addr 172.21.15.7:0/3796513729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff77fe640 1 -- 172.21.15.7:0/3796513729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 msgr2=0x558ee86dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff77fe640 1 -- 172.21.15.7:0/3796513729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 msgr2=0x558ee86d8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 0x558ee86d8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff77fe640 1 -- 172.21.15.7:0/3796513729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee83463f0 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.213+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 0x558ee86d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d8d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558ee86ca7d0 tx=0x7f4fe800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5efdb86fcaba0869 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4fd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/3796513729 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801e040 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/3796513729 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4fe8011050 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe8011250 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/3796513729 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fe8030050 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 10 client.27485 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ee84107d0 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/3796513729 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe8011250 con 0x558ee86cece0 2024-03-19T00:48:11.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/3796513729 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4fe8005410 con 0x558ee86cece0 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 10 client.27485 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 10 client.27485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 20 client.27485 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee86e43d0 0x558ee86e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 4 client.27485 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.214+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee86903e0 con 0x558ee86e43d0 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.215+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee86e43d0 0x558ee86e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.215+0000 7f4fd6ffd640 10 client.27485 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.215+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee86e43d0 0x558ee86e6820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcb73bc65091b03d server_cookie=6fce489ce0cb0f50 in_seq=0 out_seq=0 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/3796513729 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558ee86903e0 con 0x558ee86e43d0 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f4fd6ffd640 10 client.27485 handle_command_reply: tid=0 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 client.27485 shutdown 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 msgr2=0x558ee86d8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d8d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558ee86ca7d0 tx=0x7f4fe800f7f0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 shutdown_connections 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee86e43d0 0x558ee86e6820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f4fd6ffd640 0 client.27485 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86cece0 0x558ee86d8d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 --2- 172.21.15.7:0/3796513729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86ce2f0 0x558ee86d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 >> 172.21.15.7:0/3796513729 conn(0x558ee86c9860 msgr2=0x558ee86c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 shutdown_connections 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 1 -- 172.21.15.7:0/3796513729 wait complete. 2024-03-19T00:48:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.216+0000 7f5000c1d740 20 client.27485 trim_cache size 0 max 16384 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 Processor -- start 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 -- start start 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86ce2f0 0x558ee86d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81acb90 con 0x558ee86cece0 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81aa460 con 0x558ee86ce2f0 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f5000c1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81a82b0 con 0x558ee86d3e50 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40480/0 (socket says 172.21.15.7:40480) 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f4ff77fe640 1 -- 172.21.15.7:0/326623888 learned_addr learned my addr 172.21.15.7:0/326623888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f4ff6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86ce2f0 0x558ee86d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.229+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4ff77fe640 1 -- 172.21.15.7:0/326623888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 msgr2=0x558ee86d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4ff77fe640 1 -- 172.21.15.7:0/326623888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86ce2f0 msgr2=0x558ee86d7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86ce2f0 0x558ee86d7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4ff77fe640 1 -- 172.21.15.7:0/326623888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee86cf980 con 0x558ee86cece0 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dfd50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80057c0 tx=0x7f4fe8011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c1c3a54722b2e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4fdd7fa640 1 -- 172.21.15.7:0/326623888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801e070 con 0x558ee86cece0 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4fdd7fa640 1 -- 172.21.15.7:0/326623888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe80044e0 con 0x558ee86cece0 2024-03-19T00:48:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.230+0000 7f4fdd7fa640 1 -- 172.21.15.7:0/326623888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8003a50 con 0x558ee86cece0 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- 172.21.15.7:0/326623888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 msgr2=0x558ee86dfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dfd50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80057c0 tx=0x7f4fe8011eb0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- 172.21.15.7:0/326623888 shutdown_connections 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86d3e50 0x558ee86d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dfd50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- 172.21.15.7:0/326623888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86ce2f0 0x558ee86d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- 172.21.15.7:0/326623888 >> 172.21.15.7:0/326623888 conn(0x558ee86c9860 msgr2=0x558ee86c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- 172.21.15.7:0/326623888 shutdown_connections 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- 172.21.15.7:0/326623888 wait complete. 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- start start 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 Processor -- start 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86dd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86d3e50 0x558ee86de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81a8920 con 0x558ee86cece0 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81a9d90 con 0x558ee86d3e50 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81a8ff0 con 0x558ee86ce2f0 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f5000c1d740 10 client.0 _ll_register_callbacks cb 0x7f4fe801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40492/0 (socket says 172.21.15.7:40492) 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.231+0000 7f4ff77fe640 1 -- 172.21.15.7:0/2852688403 learned_addr learned my addr 172.21.15.7:0/2852688403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86d3e50 0x558ee86de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86dd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff77fe640 1 -- 172.21.15.7:0/2852688403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 msgr2=0x558ee86dd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86dd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff77fe640 1 -- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86d3e50 msgr2=0x558ee86de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86d3e50 0x558ee86de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff77fe640 1 -- 172.21.15.7:0/2852688403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee86dedf0 con 0x558ee86cece0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86d3e50 0x558ee86de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4ff77fe640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dd860 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80095f0 tx=0x7f4fe8003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442711a6b1549a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4fd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/2852688403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8038070 con 0x558ee86cece0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/2852688403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4fe800f7d0 con 0x558ee86cece0 2024-03-19T00:48:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe80044e0 con 0x558ee86cece0 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/2852688403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fe8044050 con 0x558ee86cece0 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f5000c1d740 10 client.17646 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ee874a6d0 con 0x558ee86cece0 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/2852688403 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe80044e0 con 0x558ee86cece0 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.232+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/2852688403 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4fe8024980 con 0x558ee86cece0 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 10 client.17646 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 10 client.17646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 20 client.17646 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee874af40 0x558ee874d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 4 client.17646 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ee863f9f0 con 0x558ee874af40 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee874af40 0x558ee874d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f4fd6ffd640 10 client.17646 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:11.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f4ff6ffd640 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee874af40 0x558ee874d360 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8bf62c1b6df514f server_cookie=fcc3694e8f63b33f in_seq=0 out_seq=0 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f4fd6ffd640 1 -- 172.21.15.7:0/2852688403 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ee863f9f0 con 0x558ee874af40 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f4fd6ffd640 10 client.17646 handle_command_reply: tid=0 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 client.17646 shutdown 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 msgr2=0x558ee86dd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dd860 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80095f0 tx=0x7f4fe8003850 comp rx=0 tx=0).stop 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 shutdown_connections 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x558ee874af40 0x558ee874d360 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ee86d3e50 0x558ee86de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ee86cece0 0x558ee86dd860 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 --2- 172.21.15.7:0/2852688403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ee86ce2f0 0x558ee86dd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f4fd6ffd640 0 client.17646 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.233+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 >> 172.21.15.7:0/2852688403 conn(0x558ee86c9860 msgr2=0x558ee86d87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 shutdown_connections 2024-03-19T00:48:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f5000c1d740 1 -- 172.21.15.7:0/2852688403 wait complete. 2024-03-19T00:48:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f5000c1d740 20 client.17646 trim_cache size 0 max 16384 2024-03-19T00:48:11.243 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4fe00a2800 msgr2=0x7f4fe00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4fe00a2800 0x7f4fe00a4cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00bd290 tx=0x7f4ff00a8040 comp rx=0 tx=0).stop 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 msgr2=0x7f4ff819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff819ed80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4ff002c700 tx=0x7f4ff00328a0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 shutdown_connections 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4fe00a2800 0x7f4fe00a4cc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ff81022e0 0x7f4ff819ed80 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ff8101920 0x7f4ff819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 --2- 172.21.15.7:0/2808628406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff8100730 0x7f4ff806aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 >> 172.21.15.7:0/2808628406 conn(0x7f4ff80fbc80 msgr2=0x7f4ff8104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 shutdown_connections 2024-03-19T00:48:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.234+0000 7f4ffe036640 1 -- 172.21.15.7:0/2808628406 wait complete. 2024-03-19T00:48:11.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:48:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.349+0000 7f243b40c640 1 -- 172.21.15.7:0/1420227518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2434101910 msgr2=0x7f2434101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.349+0000 7f243b40c640 1 --2- 172.21.15.7:0/1420227518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2434101910 0x7f2434101d90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2428009900 tx=0x7f242802bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.349+0000 7f243b40c640 1 -- 172.21.15.7:0/1420227518 shutdown_connections 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.349+0000 7f243b40c640 1 --2- 172.21.15.7:0/1420227518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24341022d0 0x7f2434108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 --2- 172.21.15.7:0/1420227518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2434101910 0x7f2434101d90 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 --2- 172.21.15.7:0/1420227518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2434100720 0x7f2434100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- 172.21.15.7:0/1420227518 >> 172.21.15.7:0/1420227518 conn(0x7f24340fbc50 msgr2=0x7f24340fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- 172.21.15.7:0/1420227518 shutdown_connections 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- 172.21.15.7:0/1420227518 wait complete. 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 Processor -- start 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- start start 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2434100720 0x7f2434197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2434101910 0x7f2434197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 0x7f243419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f243410c250 con 0x7f24341022d0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f243410c0d0 con 0x7f2434101910 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f243b40c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f243410c3d0 con 0x7f2434100720 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.350+0000 7f24399a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 0x7f243419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 0x7f243419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40514/0 (socket says 172.21.15.7:40514) 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24391a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2434100720 0x7f2434197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 -- 172.21.15.7:0/2232105463 learned_addr learned my addr 172.21.15.7:0/2232105463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24389a7640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2434101910 0x7f2434197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 -- 172.21.15.7:0/2232105463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2434100720 msgr2=0x7f2434197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2434100720 0x7f2434197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 -- 172.21.15.7:0/2232105463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2434101910 msgr2=0x7f2434197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2434101910 0x7f2434197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 -- 172.21.15.7:0/2232105463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243419f670 con 0x7f24341022d0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24399a9640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 0x7f243419f000 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f242c00e970 tx=0x7f242c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24327fc640 1 -- 172.21.15.7:0/2232105463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c00ccc0 con 0x7f24341022d0 2024-03-19T00:48:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24327fc640 1 -- 172.21.15.7:0/2232105463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f242c00ce60 con 0x7f24341022d0 2024-03-19T00:48:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243419f960 con 0x7f24341022d0 2024-03-19T00:48:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f24327fc640 1 -- 172.21.15.7:0/2232105463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c010430 con 0x7f24341022d0 2024-03-19T00:48:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.351+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243419fe80 con 0x7f24341022d0 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.353+0000 7f24327fc640 1 -- 172.21.15.7:0/2232105463 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f242c004f90 con 0x7f24341022d0 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 Processor -- start 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 -- start start 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333c2f0 0x562c6333c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333cce0 0x562c633417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 0x562c633442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c632a04b0 con 0x562c63341e50 2024-03-19T00:48:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562c63248d70 con 0x562c6333cce0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f243dff0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c6329c550 con 0x562c6333c2f0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 0x562c633442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24327fc640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f241c0a2800 0x7f241c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 0x562c633442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40526/0 (socket says 172.21.15.7:40526) 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 -- 172.21.15.7:0/3132901354 learned_addr learned my addr 172.21.15.7:0/3132901354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24391a8640 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333cce0 0x562c633417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333c2f0 0x562c6333c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24399a9640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f241c0a2800 0x7f241c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24327fc640 1 -- 172.21.15.7:0/2232105463 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f242c0cf160 con 0x7f24341022d0 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 -- 172.21.15.7:0/3132901354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333c2f0 msgr2=0x562c6333c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333c2f0 0x562c6333c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 -- 172.21.15.7:0/3132901354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333cce0 msgr2=0x562c633417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333cce0 0x562c633417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24389a7640 1 -- 172.21.15.7:0/3132901354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6326b210 con 0x562c63341e50 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.354+0000 7f24391a8640 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333cce0 0x562c633417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f24389a7640 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 0x562c633442e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f24280096c0 tx=0x7f242802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b38886d0c27f68d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f24399a9640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f241c0a2800 0x7f241c0a4cc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f242c09b4a0 tx=0x7f242c090c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f241b7fe640 1 -- 172.21.15.7:0/3132901354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428004040 con 0x562c63341e50 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f241b7fe640 1 -- 172.21.15.7:0/3132901354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24280041e0 con 0x562c63341e50 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f241b7fe640 1 -- 172.21.15.7:0/3132901354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428043450 con 0x562c63341e50 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 -- 172.21.15.7:0/3132901354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 msgr2=0x562c633442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 0x562c633442e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f24280096c0 tx=0x7f242802bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 -- 172.21.15.7:0/3132901354 shutdown_connections 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c63341e50 0x562c633442e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333cce0 0x562c633417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 --2- 172.21.15.7:0/3132901354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333c2f0 0x562c6333c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.355+0000 7f243dff0740 1 -- 172.21.15.7:0/3132901354 >> 172.21.15.7:0/3132901354 conn(0x562c63337860 msgr2=0x562c63337c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 -- 172.21.15.7:0/3132901354 shutdown_connections 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 -- 172.21.15.7:0/3132901354 wait complete. 2024-03-19T00:48:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 -- start start 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 Processor -- start 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c63346800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c63346d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c6334dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c63344da0 con 0x562c6333cce0 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562c6325c310 con 0x562c6333c2f0 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562c63344f20 con 0x562c63341e50 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f243dff0740 10 client.0 _ll_register_callbacks cb 0x562c632502d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f24391a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c63346d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f24391a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c63346d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40528/0 (socket says 172.21.15.7:40528) 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f24391a8640 1 -- 172.21.15.7:0/989547406 learned_addr learned my addr 172.21.15.7:0/989547406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f24389a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c63346800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.356+0000 7f24389a7640 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c6334dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f24391a8640 1 -- 172.21.15.7:0/989547406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 msgr2=0x562c6334dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f24391a8640 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c6334dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f24391a8640 1 -- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 msgr2=0x562c63346800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f24391a8640 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c63346800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f24391a8640 1 -- 172.21.15.7:0/989547406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c62fb43f0 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f24391a8640 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c63346d40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2424012cd0 tx=0x7f242400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a707458701f695 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f2418ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f2418ff9640 1 -- 172.21.15.7:0/989547406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24240120d0 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f2418ff9640 1 -- 172.21.15.7:0/989547406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2424003970 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24280041e0 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f2418ff9640 1 -- 172.21.15.7:0/989547406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2424025050 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f243dff0740 10 client.17658 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c6307e7d0 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.357+0000 7f2418ff9640 1 -- 172.21.15.7:0/989547406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24280041e0 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f2418ff9640 1 -- 172.21.15.7:0/989547406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f24240046e0 con 0x562c6333cce0 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f243dff0740 10 client.17658 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f243dff0740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f243dff0740 20 client.17658 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f243dff0740 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c63352740 0x562c63354bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f243dff0740 4 client.17658 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c632fe3e0 con 0x562c63352740 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f24389a7640 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c63352740 0x562c63354bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f2418ff9640 10 client.17658 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.358+0000 7f24389a7640 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c63352740 0x562c63354bc0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7c659f6df8ce29 server_cookie=caf57af3c8e3858f in_seq=0 out_seq=0 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f2418ff9640 1 -- 172.21.15.7:0/989547406 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562c632fe3e0 con 0x562c63352740 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f2418ff9640 10 client.17658 handle_command_reply: tid=0 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 client.17658 shutdown 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 msgr2=0x562c63346d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c63346d40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2424012cd0 tx=0x7f242400f790 comp rx=0 tx=0).stop 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 shutdown_connections 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c6334dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c63352740 0x562c63354bc0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c63346d40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 --2- 172.21.15.7:0/989547406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c63346800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f2418ff9640 0 client.17658 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 >> 172.21.15.7:0/989547406 conn(0x562c63337860 msgr2=0x562c63336ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 shutdown_connections 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 1 -- 172.21.15.7:0/989547406 wait complete. 2024-03-19T00:48:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.359+0000 7f243dff0740 20 client.17658 trim_cache size 0 max 16384 2024-03-19T00:48:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 Processor -- start 2024-03-19T00:48:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 -- start start 2024-03-19T00:48:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 0x562c63336310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333cce0 0x562c63336850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c63341e50 0x562c63336d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24341acbf0 con 0x562c6333c2f0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24341aa4c0 con 0x562c63341e50 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f243dff0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24341a8310 con 0x562c6333cce0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 0x562c63336310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 0x562c63336310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40536/0 (socket says 172.21.15.7:40536) 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 -- 172.21.15.7:0/9142755 learned_addr learned my addr 172.21.15.7:0/9142755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24391a8640 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333cce0 0x562c63336850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c63341e50 0x562c63336d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 -- 172.21.15.7:0/9142755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333cce0 msgr2=0x562c63336850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333cce0 0x562c63336850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 -- 172.21.15.7:0/9142755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c63341e50 msgr2=0x562c63336d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c63341e50 0x562c63336d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 -- 172.21.15.7:0/9142755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6330d880 con 0x562c6333c2f0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24391a8640 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333cce0 0x562c63336850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.373+0000 7f24389a7640 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 0x562c63336310 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f242802cc00 tx=0x7f2428046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c07606a6079c348 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f241affd640 1 -- 172.21.15.7:0/9142755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242805a4a0 con 0x562c6333c2f0 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f241affd640 1 -- 172.21.15.7:0/9142755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24280054e0 con 0x562c6333c2f0 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f241affd640 1 -- 172.21.15.7:0/9142755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428004b50 con 0x562c6333c2f0 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 -- 172.21.15.7:0/9142755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 msgr2=0x562c63336310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 0x562c63336310 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f242802cc00 tx=0x7f2428046040 comp rx=0 tx=0).stop 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 -- 172.21.15.7:0/9142755 shutdown_connections 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c63341e50 0x562c63336d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c6333cce0 0x562c63336850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 --2- 172.21.15.7:0/9142755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333c2f0 0x562c63336310 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 -- 172.21.15.7:0/9142755 >> 172.21.15.7:0/9142755 conn(0x562c63337860 msgr2=0x562c6333e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 -- 172.21.15.7:0/9142755 shutdown_connections 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 -- 172.21.15.7:0/9142755 wait complete. 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.374+0000 7f243dff0740 1 -- start start 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 Processor -- start 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c633b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c633b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c633bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24341a8980 con 0x562c6333cce0 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24341a9df0 con 0x562c6333c2f0 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24341a9050 con 0x562c63341e50 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f243dff0740 10 client.0 _ll_register_callbacks cb 0x7f2424009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c633b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c633b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40546/0 (socket says 172.21.15.7:40546) 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 -- 172.21.15.7:0/3283032558 learned_addr learned my addr 172.21.15.7:0/3283032558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24389a7640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c633b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24389a7640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c633bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 -- 172.21.15.7:0/3283032558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 msgr2=0x562c633bb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c633bb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 -- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 msgr2=0x562c633b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c633b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 -- 172.21.15.7:0/3283032558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c633bbe30 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24389a7640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c633b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f24391a8640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c633b4800 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x562c63347ce0 tx=0x7f242400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8115a89565894a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.375+0000 7f2418ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f2418ff9640 1 -- 172.21.15.7:0/3283032558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242402e060 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f2418ff9640 1 -- 172.21.15.7:0/3283032558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f242401cb70 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24280054e0 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f2418ff9640 1 -- 172.21.15.7:0/3283032558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2424035050 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 10 client.17664 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c633bd130 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f2418ff9640 1 -- 172.21.15.7:0/3283032558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24280054e0 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f2418ff9640 1 -- 172.21.15.7:0/3283032558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2424028040 con 0x562c6333cce0 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 10 client.17664 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 20 client.17664 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c633bdad0 0x562c633bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 4 client.17664 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c632ad9f0 con 0x562c633bdad0 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.376+0000 7f24389a7640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c633bdad0 0x562c633bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f24389a7640 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c633bdad0 0x562c633bfef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b978b0905f3fc6b5 server_cookie=a9b30be2acd9123b in_seq=0 out_seq=0 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f2418ff9640 10 client.17664 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f2418ff9640 1 -- 172.21.15.7:0/3283032558 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c632ad9f0 con 0x562c633bdad0 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f2418ff9640 10 client.17664 handle_command_reply: tid=0 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 client.17664 shutdown 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 msgr2=0x562c633b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c633b4800 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x562c63347ce0 tx=0x7f242400fd10 comp rx=0 tx=0).stop 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 shutdown_connections 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562c63341e50 0x562c633bb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x562c633bdad0 0x562c633bfef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c6333cce0 0x562c633b4800 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 --2- 172.21.15.7:0/3283032558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c6333c2f0 0x562c633b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f2418ff9640 0 client.17664 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 >> 172.21.15.7:0/3283032558 conn(0x562c63337860 msgr2=0x562c6333aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 shutdown_connections 2024-03-19T00:48:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 1 -- 172.21.15.7:0/3283032558 wait complete. 2024-03-19T00:48:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.377+0000 7f243dff0740 20 client.17664 trim_cache size 0 max 16384 2024-03-19T00:48:11.386 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f241c0a2800 msgr2=0x7f241c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f241c0a2800 0x7f241c0a4cc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f242c09b4a0 tx=0x7f242c090c80 comp rx=0 tx=0).stop 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 msgr2=0x7f243419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 0x7f243419f000 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f242c00e970 tx=0x7f242c00ed70 comp rx=0 tx=0).stop 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 shutdown_connections 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f241c0a2800 0x7f241c0a4cc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24341022d0 0x7f243419f000 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2434101910 0x7f2434197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 --2- 172.21.15.7:0/2232105463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2434100720 0x7f2434197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 >> 172.21.15.7:0/2232105463 conn(0x7f24340fbc50 msgr2=0x7f24340fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 shutdown_connections 2024-03-19T00:48:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:11.378+0000 7f243b40c640 1 -- 172.21.15.7:0/2232105463 wait complete. 2024-03-19T00:48:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:11 smithi044 ceph-mon[66366]: pgmap v658: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:48:12.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:11 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1134538632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:48:12.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:11 smithi007 ceph-mon[62854]: pgmap v658: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:48:12.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:11 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1134538632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:48:12.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:11 smithi204 ceph-mon[66572]: pgmap v658: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:48:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:11 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1134538632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:48:12.391 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:48:12.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:48:12.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:48:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 -- 172.21.15.7:0/1935030187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778100740 msgr2=0x7fb778100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 --2- 172.21.15.7:0/1935030187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778100740 0x7fb778100b60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb774009900 tx=0x7fb77402bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 -- 172.21.15.7:0/1935030187 shutdown_connections 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 --2- 172.21.15.7:0/1935030187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7781022f0 0x7fb778108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 --2- 172.21.15.7:0/1935030187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb778101930 0x7fb778101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 --2- 172.21.15.7:0/1935030187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778100740 0x7fb778100b60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 -- 172.21.15.7:0/1935030187 >> 172.21.15.7:0/1935030187 conn(0x7fb7780fbcd0 msgr2=0x7fb7780fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 -- 172.21.15.7:0/1935030187 shutdown_connections 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 -- 172.21.15.7:0/1935030187 wait complete. 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 Processor -- start 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.509+0000 7fb77f91b640 1 -- start start 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb778100740 0x7fb778197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb778101930 0x7fb778197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 0x7fb77819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810c240 con 0x7fb7781022f0 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810c0c0 con 0x7fb778101930 2024-03-19T00:48:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810c3c0 con 0x7fb778100740 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 0x7fb77819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb778101930 0x7fb778197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 0x7fb77819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40570/0 (socket says 172.21.15.7:40570) 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 -- 172.21.15.7:0/3470361805 learned_addr learned my addr 172.21.15.7:0/3470361805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb778100740 0x7fb778197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 -- 172.21.15.7:0/3470361805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb778100740 msgr2=0x7fb778197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb778100740 0x7fb778197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 -- 172.21.15.7:0/3470361805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb778101930 msgr2=0x7fb778197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb778101930 0x7fb778197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 -- 172.21.15.7:0/3470361805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77819f6b0 con 0x7fb7781022f0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77deb8640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 0x7fb77819f040 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb77000eb10 tx=0x7fb77000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3470361805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770004b20 con 0x7fb7781022f0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3470361805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb770004cc0 con 0x7fb7781022f0 2024-03-19T00:48:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77819f9a0 con 0x7fb7781022f0 2024-03-19T00:48:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3470361805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770010690 con 0x7fb7781022f0 2024-03-19T00:48:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.510+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77819fea0 con 0x7fb7781022f0 2024-03-19T00:48:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.513+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3470361805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb770005020 con 0x7fb7781022f0 2024-03-19T00:48:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.513+0000 7fb7825e2740 1 Processor -- start 2024-03-19T00:48:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.513+0000 7fb7825e2740 1 -- start start 2024-03-19T00:48:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa0a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa0f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa12390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb7825e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b73f96f160 con 0x55b73fa0a3a0 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb7825e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b73f917e10 con 0x55b73fa0ff00 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb7825e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b73f96b200 con 0x55b73fa0ad90 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb76e7fc640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7640a2800 0x7fb7640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa0a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa0a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40586/0 (socket says 172.21.15.7:40586) 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77ceb6640 1 -- 172.21.15.7:0/3093047666 learned_addr learned my addr 172.21.15.7:0/3093047666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa0f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa12390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77deb8640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7640a2800 0x7fb7640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3470361805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7700cf3b0 con 0x7fb7781022f0 2024-03-19T00:48:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 -- 172.21.15.7:0/3093047666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 msgr2=0x55b73fa12390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa12390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 -- 172.21.15.7:0/3093047666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 msgr2=0x55b73fa0a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa0a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 -- 172.21.15.7:0/3093047666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73f74b7e0 con 0x55b73fa0ad90 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa12390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.514+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa0f890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb774036040 tx=0x7fb774009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ee4d2ffe633d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb77deb8640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7640a2800 0x7fb7640a4cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb77000e300 tx=0x7fb770066750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7637fe640 1 -- 172.21.15.7:0/3093047666 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774045020 con 0x55b73fa0ad90 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7637fe640 1 -- 172.21.15.7:0/3093047666 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb774004040 con 0x55b73fa0ad90 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7637fe640 1 -- 172.21.15.7:0/3093047666 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740046c0 con 0x55b73fa0ad90 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 -- 172.21.15.7:0/3093047666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 msgr2=0x55b73fa0f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa0f890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb774036040 tx=0x7fb774009560 comp rx=0 tx=0).stop 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 -- 172.21.15.7:0/3093047666 shutdown_connections 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa12390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa0f890 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 --2- 172.21.15.7:0/3093047666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa0a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 -- 172.21.15.7:0/3093047666 >> 172.21.15.7:0/3093047666 conn(0x55b73fa05910 msgr2=0x55b73fa05d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 -- 172.21.15.7:0/3093047666 shutdown_connections 2024-03-19T00:48:12.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.515+0000 7fb7825e2740 1 -- 172.21.15.7:0/3093047666 wait complete. 2024-03-19T00:48:12.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 -- start start 2024-03-19T00:48:12.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 Processor -- start 2024-03-19T00:48:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0a3a0 0x55b73fa14790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 0x55b73fa14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ff00 0x55b73fa1bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b73fa12e50 con 0x55b73fa0ad90 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b73f92b1a0 con 0x55b73fa0a3a0 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b73fa12fd0 con 0x55b73fa0ff00 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb7825e2740 10 client.0 _ll_register_callbacks cb 0x55b73f682c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb77d6b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 0x55b73fa14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb77d6b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 0x55b73fa14cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40590/0 (socket says 172.21.15.7:40590) 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb77d6b7640 1 -- 172.21.15.7:0/4221329127 learned_addr learned my addr 172.21.15.7:0/4221329127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0a3a0 0x55b73fa14790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.516+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ff00 0x55b73fa1bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77d6b7640 1 -- 172.21.15.7:0/4221329127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ff00 msgr2=0x55b73fa1bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ff00 0x55b73fa1bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77d6b7640 1 -- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0a3a0 msgr2=0x55b73fa14790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0a3a0 0x55b73fa14790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77d6b7640 1 -- 172.21.15.7:0/4221329127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73f7abf70 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0a3a0 0x55b73fa14790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 0x55b73fa14cd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb774043000 tx=0x7fb774004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38896e6fab3df9ab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb760ff9640 1 -- 172.21.15.7:0/4221329127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740035f0 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb760ff9640 1 -- 172.21.15.7:0/4221329127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb774003790 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb774004040 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb760ff9640 1 -- 172.21.15.7:0/4221329127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb77405b050 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb7825e2740 10 client.17676 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b73f643170 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb760ff9640 1 -- 172.21.15.7:0/4221329127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb774004040 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.517+0000 7fb760ff9640 1 -- 172.21.15.7:0/4221329127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb774004e30 con 0x55b73fa0ad90 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb7825e2740 10 client.17676 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb7825e2740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb7825e2740 20 client.17676 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb7825e2740 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa20200 0x55b73fa225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb7825e2740 4 client.17676 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b73f97c6a0 con 0x55b73fa20200 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa20200 0x55b73fa225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa20200 0x55b73fa225f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4df2ac75b68e8d4 server_cookie=16c34072fe6bd33e in_seq=0 out_seq=0 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.518+0000 7fb760ff9640 10 client.17676 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb760ff9640 1 -- 172.21.15.7:0/4221329127 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b73f97c6a0 con 0x55b73fa20200 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb760ff9640 10 client.17676 handle_command_reply: tid=0 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 client.17676 shutdown 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 msgr2=0x55b73fa14cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 0x55b73fa14cd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb774043000 tx=0x7fb774004510 comp rx=0 tx=0).stop 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 shutdown_connections 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ff00 0x55b73fa1bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa20200 0x55b73fa225f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ad90 0x55b73fa14cd0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 --2- 172.21.15.7:0/4221329127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0a3a0 0x55b73fa14790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb760ff9640 0 client.17676 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.519+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 >> 172.21.15.7:0/4221329127 conn(0x55b73fa05910 msgr2=0x55b73fa04b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.520+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 shutdown_connections 2024-03-19T00:48:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.520+0000 7fb7825e2740 1 -- 172.21.15.7:0/4221329127 wait complete. 2024-03-19T00:48:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.520+0000 7fb7825e2740 20 client.17676 trim_cache size 0 max 16384 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 Processor -- start 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 -- start start 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0a3a0 0x55b73fa042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ad90 0x55b73fa047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 0x55b73fa04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781ac9a0 con 0x55b73fa0ff00 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781aa270 con 0x55b73fa0ad90 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb7825e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781a80c0 con 0x55b73fa0a3a0 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 0x55b73fa04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.537+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 0x55b73fa04d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40594/0 (socket says 172.21.15.7:40594) 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77d6b7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ad90 0x55b73fa047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 -- 172.21.15.7:0/1212214598 learned_addr learned my addr 172.21.15.7:0/1212214598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0a3a0 0x55b73fa042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 -- 172.21.15.7:0/1212214598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0a3a0 msgr2=0x55b73fa042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0a3a0 0x55b73fa042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 -- 172.21.15.7:0/1212214598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ad90 msgr2=0x55b73fa047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ad90 0x55b73fa047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 -- 172.21.15.7:0/1212214598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73fa22c20 con 0x55b73fa0ff00 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ad90 0x55b73fa047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 0x55b73fa04d20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb768023970 tx=0x7fb7680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd5e7674b7c83a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb762ffd640 1 -- 172.21.15.7:0/1212214598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768003d70 con 0x55b73fa0ff00 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb762ffd640 1 -- 172.21.15.7:0/1212214598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb768012cf0 con 0x55b73fa0ff00 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.538+0000 7fb762ffd640 1 -- 172.21.15.7:0/1212214598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680047a0 con 0x55b73fa0ff00 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 -- 172.21.15.7:0/1212214598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 msgr2=0x55b73fa04d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 0x55b73fa04d20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb768023970 tx=0x7fb7680046c0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 -- 172.21.15.7:0/1212214598 shutdown_connections 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0ff00 0x55b73fa04d20 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ad90 0x55b73fa047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1212214598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0a3a0 0x55b73fa042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 -- 172.21.15.7:0/1212214598 >> 172.21.15.7:0/1212214598 conn(0x55b73fa05910 msgr2=0x55b73fa14790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 -- 172.21.15.7:0/1212214598 shutdown_connections 2024-03-19T00:48:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 -- 172.21.15.7:0/1212214598 wait complete. 2024-03-19T00:48:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 -- start start 2024-03-19T00:48:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 Processor -- start 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa823b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.539+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb7825e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa89930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb7825e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781a8730 con 0x55b73fa0a3a0 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb7825e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781a9ba0 con 0x55b73fa0ff00 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb7825e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781a8e00 con 0x55b73fa0ad90 2024-03-19T00:48:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb7825e2740 10 client.0 _ll_register_callbacks cb 0x7fb768009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77ceb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa823b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40610/0 (socket says 172.21.15.7:40610) 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77ceb6640 1 -- 172.21.15.7:0/1150309737 learned_addr learned my addr 172.21.15.7:0/1150309737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa89930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 -- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 msgr2=0x55b73fa89930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa89930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 -- 172.21.15.7:0/1150309737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 msgr2=0x55b73fa823b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa823b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 -- 172.21.15.7:0/1150309737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73fa05260 con 0x55b73fa0ad90 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb77d6b7640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa828f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb774005c40 tx=0x7fb7740057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f91425ff4f6bef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb760ff9640 1 -- 172.21.15.7:0/1150309737 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774045040 con 0x55b73fa0ad90 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb768012cf0 con 0x55b73fa0ad90 2024-03-19T00:48:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb760ff9640 1 -- 172.21.15.7:0/1150309737 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb774041a20 con 0x55b73fa0ad90 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.540+0000 7fb760ff9640 1 -- 172.21.15.7:0/1150309737 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb774041c20 con 0x55b73fa0ad90 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 10 client.27439 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b73fa8b090 con 0x55b73fa0ad90 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb760ff9640 1 -- 172.21.15.7:0/1150309737 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb768012cf0 con 0x55b73fa0ad90 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb760ff9640 1 -- 172.21.15.7:0/1150309737 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb774004d30 con 0x55b73fa0ad90 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 10 client.27439 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 10 client.27439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 20 client.27439 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa8b920 0x55b73fa8dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 4 client.27439 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b73f9769f0 con 0x55b73fa8b920 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa8b920 0x55b73fa8dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb77ceb6640 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa8b920 0x55b73fa8dd40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0725a4d656be56 server_cookie=2e92d6201a04f852 in_seq=0 out_seq=0 2024-03-19T00:48:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.541+0000 7fb760ff9640 10 client.27439 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb760ff9640 1 -- 172.21.15.7:0/1150309737 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b73f9769f0 con 0x55b73fa8b920 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb760ff9640 10 client.27439 handle_command_reply: tid=0 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 client.27439 shutdown 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 msgr2=0x55b73fa828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa828f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb774005c40 tx=0x7fb7740057f0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 shutdown_connections 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b73fa0ff00 0x55b73fa89930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b73fa0ad90 0x55b73fa828f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b73fa8b920 0x55b73fa8dd40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.542+0000 7fb7825e2740 1 --2- 172.21.15.7:0/1150309737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b73fa0a3a0 0x55b73fa823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 >> 172.21.15.7:0/1150309737 conn(0x55b73fa05910 msgr2=0x55b73fa042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb760ff9640 0 client.27439 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 shutdown_connections 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb7825e2740 1 -- 172.21.15.7:0/1150309737 wait complete. 2024-03-19T00:48:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb7825e2740 20 client.27439 trim_cache size 0 max 16384 2024-03-19T00:48:12.550 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:48:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:48:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "30a7ae58-bc97-46dc-92cd-627ddbc494d4", 2024-03-19T00:48:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:48:12.550 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:48:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7640a2800 msgr2=0x7fb7640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7640a2800 0x7fb7640a4cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb77000e300 tx=0x7fb770066750 comp rx=0 tx=0).stop 2024-03-19T00:48:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 msgr2=0x7fb77819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 0x7fb77819f040 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb77000eb10 tx=0x7fb77000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 shutdown_connections 2024-03-19T00:48:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb7640a2800 0x7fb7640a4cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7781022f0 0x7fb77819f040 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb778101930 0x7fb778197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 --2- 172.21.15.7:0/3470361805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb778100740 0x7fb778197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 >> 172.21.15.7:0/3470361805 conn(0x7fb7780fbcd0 msgr2=0x7fb778069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 shutdown_connections 2024-03-19T00:48:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.543+0000 7fb77f91b640 1 -- 172.21.15.7:0/3470361805 wait complete. 2024-03-19T00:48:12.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:48:12.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:48:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.658+0000 7f27f8359640 1 -- 172.21.15.7:0/1226384270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0101930 msgr2=0x7f27f0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.658+0000 7f27f8359640 1 --2- 172.21.15.7:0/1226384270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0101930 0x7f27f0101db0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009900 tx=0x7f27e402bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- 172.21.15.7:0/1226384270 shutdown_connections 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 --2- 172.21.15.7:0/1226384270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27f01022f0 0x7f27f0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 --2- 172.21.15.7:0/1226384270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0101930 0x7f27f0101db0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 --2- 172.21.15.7:0/1226384270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f0100740 0x7f27f0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- 172.21.15.7:0/1226384270 >> 172.21.15.7:0/1226384270 conn(0x7f27f00fbcb0 msgr2=0x7f27f00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- 172.21.15.7:0/1226384270 shutdown_connections 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- 172.21.15.7:0/1226384270 wait complete. 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 Processor -- start 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- start start 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 0x7f27f0069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27f0101930 0x7f27f019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f01022f0 0x7f27f019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27f010c250 con 0x7f27f0100740 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27f010c0d0 con 0x7f27f0101930 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f8359640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27f010c3d0 con 0x7f27f01022f0 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f60f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 0x7f27f0069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f60f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 0x7f27f0069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40626/0 (socket says 172.21.15.7:40626) 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f60f5640 1 -- 172.21.15.7:0/288172532 learned_addr learned my addr 172.21.15.7:0/288172532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f68f6640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f01022f0 0x7f27f019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.659+0000 7f27f58f4640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27f0101930 0x7f27f019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f60f5640 1 -- 172.21.15.7:0/288172532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f01022f0 msgr2=0x7f27f019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f60f5640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f01022f0 0x7f27f019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f60f5640 1 -- 172.21.15.7:0/288172532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27f0101930 msgr2=0x7f27f019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f60f5640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27f0101930 0x7f27f019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f60f5640 1 -- 172.21.15.7:0/288172532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f019f670 con 0x7f27f0100740 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f60f5640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 0x7f27f0069680 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00ea80 tx=0x7f27ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27e37fe640 1 -- 172.21.15.7:0/288172532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec00cd80 con 0x7f27f0100740 2024-03-19T00:48:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27e37fe640 1 -- 172.21.15.7:0/288172532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec011920 con 0x7f27f0100740 2024-03-19T00:48:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f019f960 con 0x7f27f0100740 2024-03-19T00:48:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27e37fe640 1 -- 172.21.15.7:0/288172532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec01a9e0 con 0x7f27f0100740 2024-03-19T00:48:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.660+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f019fed0 con 0x7f27f0100740 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27e37fe640 1 -- 172.21.15.7:0/288172532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f27ec011ad0 con 0x7f27f0100740 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 Processor -- start 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 -- start start 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c63de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63e440 0x55e37c642f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c645a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e37c5a2500 con 0x55e37c6435b0 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e37c54abe0 con 0x55e37c63e440 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27fb016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e37c59e5a0 con 0x55e37c63da50 2024-03-19T00:48:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c645a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f60f5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63e440 0x55e37c642f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c63de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c63de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:36858/0 (socket says 172.21.15.7:36858) 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 -- 172.21.15.7:0/1245093513 learned_addr learned my addr 172.21.15.7:0/1245093513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f60f5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63e440 0x55e37c642f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48134/0 (socket says 172.21.15.7:48134) 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27e37fe640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27dc0a2880 0x7f27dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 -- 172.21.15.7:0/1245093513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63e440 msgr2=0x55e37c642f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63e440 0x55e37c642f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 -- 172.21.15.7:0/1245093513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 msgr2=0x55e37c645a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.663+0000 7f27f68f6640 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c645a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27f68f6640 1 -- 172.21.15.7:0/1245093513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37c56c000 con 0x55e37c63da50 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27e37fe640 1 -- 172.21.15.7:0/288172532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f27ec0cf250 con 0x7f27f0100740 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27f68f6640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27dc0a2880 0x7f27dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27f68f6640 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c63de70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009d90 tx=0x7f27e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df3c129044e8349 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27e0bfe640 1 -- 172.21.15.7:0/1245093513 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8017070 con 0x55e37c63da50 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27f68f6640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27dc0a2880 0x7f27dc0a4d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f27e8023000 tx=0x7f27e8024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27e0bfe640 1 -- 172.21.15.7:0/1245093513 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e80044b0 con 0x55e37c63da50 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.664+0000 7f27e0bfe640 1 -- 172.21.15.7:0/1245093513 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e800f880 con 0x55e37c63da50 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- 172.21.15.7:0/1245093513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 msgr2=0x55e37c63de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c63de70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009d90 tx=0x7f27e8011810 comp rx=0 tx=0).stop 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- 172.21.15.7:0/1245093513 shutdown_connections 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c645a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63e440 0x55e37c642f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- 172.21.15.7:0/1245093513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c63de70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- 172.21.15.7:0/1245093513 >> 172.21.15.7:0/1245093513 conn(0x55e37c638fc0 msgr2=0x55e37c6393c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- 172.21.15.7:0/1245093513 shutdown_connections 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- 172.21.15.7:0/1245093513 wait complete. 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- start start 2024-03-19T00:48:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 Processor -- start 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c647ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c6483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c6435b0 0x55e37c64f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e37c646560 con 0x55e37c63e440 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e37c55e160 con 0x55e37c6435b0 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e37c6466e0 con 0x55e37c63da50 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.665+0000 7f27fb016740 10 client.0 _ll_register_callbacks cb 0x55e37c59aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c6483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c6483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40640/0 (socket says 172.21.15.7:40640) 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 -- 172.21.15.7:0/1067852502 learned_addr learned my addr 172.21.15.7:0/1067852502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f60f5640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c647ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f68f6640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c6435b0 0x55e37c64f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 -- 172.21.15.7:0/1067852502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 msgr2=0x55e37c647ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c647ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 -- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c6435b0 msgr2=0x55e37c64f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c6435b0 0x55e37c64f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 -- 172.21.15.7:0/1067852502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37c3807d0 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27f58f4640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c6483e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f27e402c170 tx=0x7f27e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512deb8ef6d19e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1067852502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4005d50 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1067852502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f27e4005030 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e80044b0 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1067852502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27e4052050 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27fb016740 10 client.17694 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.666+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e37c2b63f0 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1067852502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e80044b0 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1067852502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27e40035f0 con 0x55e37c63e440 2024-03-19T00:48:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27fb016740 10 client.17694 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27fb016740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27fb016740 20 client.17694 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27fb016740 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c653dc0 0x55e37c656220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27fb016740 4 client.17694 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e37c5afa40 con 0x55e37c653dc0 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27f60f5640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c653dc0 0x55e37c656220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27f60f5640 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c653dc0 0x55e37c656220 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ee928e19f6712 server_cookie=f71da35d192d9f7f in_seq=0 out_seq=0 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.667+0000 7f27d9ffb640 10 client.17694 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1067852502 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e37c5afa40 con 0x55e37c653dc0 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27d9ffb640 10 client.17694 handle_command_reply: tid=0 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 client.17694 shutdown 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 msgr2=0x55e37c6483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c6483e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f27e402c170 tx=0x7f27e4004270 comp rx=0 tx=0).stop 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 shutdown_connections 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c6435b0 0x55e37c64f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c653dc0 0x55e37c656220 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c6483e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 --2- 172.21.15.7:0/1067852502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63da50 0x55e37c647ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27d9ffb640 0 client.17694 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 >> 172.21.15.7:0/1067852502 conn(0x55e37c638fc0 msgr2=0x55e37c6381e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 shutdown_connections 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 1 -- 172.21.15.7:0/1067852502 wait complete. 2024-03-19T00:48:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.668+0000 7f27fb016740 20 client.17694 trim_cache size 0 max 16384 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 Processor -- start 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 -- start start 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c63f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63e440 0x55e37c63fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c64f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01acb10 con 0x55e37c6435b0 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01aa3e0 con 0x55e37c63da50 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27fb016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01a8230 con 0x55e37c63e440 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c64f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c64f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40644/0 (socket says 172.21.15.7:40644) 2024-03-19T00:48:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 -- 172.21.15.7:0/685630805 learned_addr learned my addr 172.21.15.7:0/685630805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f60f5640 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c63f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f58f4640 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63e440 0x55e37c63fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 -- 172.21.15.7:0/685630805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63e440 msgr2=0x55e37c63fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63e440 0x55e37c63fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 -- 172.21.15.7:0/685630805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 msgr2=0x55e37c63f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c63f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 -- 172.21.15.7:0/685630805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37c2760f0 con 0x55e37c6435b0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f60f5640 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c63f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.686+0000 7f27f68f6640 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c64f3e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x55e37c63eca0 tx=0x7f27e8009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd575777ad052438 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27dbfff640 1 -- 172.21.15.7:0/685630805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8017070 con 0x55e37c6435b0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27dbfff640 1 -- 172.21.15.7:0/685630805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e800f870 con 0x55e37c6435b0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27dbfff640 1 -- 172.21.15.7:0/685630805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80055e0 con 0x55e37c6435b0 2024-03-19T00:48:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 -- 172.21.15.7:0/685630805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 msgr2=0x55e37c64f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c64f3e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x55e37c63eca0 tx=0x7f27e8009350 comp rx=0 tx=0).stop 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 -- 172.21.15.7:0/685630805 shutdown_connections 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c6435b0 0x55e37c64f3e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c63e440 0x55e37c63fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 --2- 172.21.15.7:0/685630805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c63f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 -- 172.21.15.7:0/685630805 >> 172.21.15.7:0/685630805 conn(0x55e37c638fc0 msgr2=0x55e37c6393a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 -- 172.21.15.7:0/685630805 shutdown_connections 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 -- 172.21.15.7:0/685630805 wait complete. 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.687+0000 7f27fb016740 1 -- start start 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 Processor -- start 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c64cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c64d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c6435b0 0x55e37c64e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01a88a0 con 0x55e37c63e440 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01a9d10 con 0x55e37c63da50 2024-03-19T00:48:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01a8f70 con 0x55e37c6435b0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 10 client.0 _ll_register_callbacks cb 0x7f27e4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c64d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c64d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40646/0 (socket says 172.21.15.7:40646) 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 -- 172.21.15.7:0/1113292257 learned_addr learned my addr 172.21.15.7:0/1113292257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f68f6640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c6435b0 0x55e37c64e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f60f5640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c64cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 -- 172.21.15.7:0/1113292257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c6435b0 msgr2=0x55e37c64e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c6435b0 0x55e37c64e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 -- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 msgr2=0x55e37c64cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c64cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 -- 172.21.15.7:0/1113292257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37c64e840 con 0x55e37c63e440 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27f58f4640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c64d2b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55e37c634b10 tx=0x7f27e4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750c5b5c28543ce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1113292257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4005d50 con 0x55e37c63e440 2024-03-19T00:48:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1113292257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f27e40046c0 con 0x55e37c63e440 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.688+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e800f870 con 0x55e37c63e440 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1113292257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27e4003e60 con 0x55e37c63e440 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 10 client.17700 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e37c6ba300 con 0x55e37c63e440 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1113292257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e800f870 con 0x55e37c63e440 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1113292257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27e4056430 con 0x55e37c63e440 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 10 client.17700 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 20 client.17700 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c6bad20 0x55e37c6bd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 4 client.17700 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e37c5a9d90 con 0x55e37c6bad20 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27f60f5640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c6bad20 0x55e37c6bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27d9ffb640 10 client.17700 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.689+0000 7f27f60f5640 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c6bad20 0x55e37c6bd140 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebe13e847247b24 server_cookie=bd2f1cad4b4de1e6 in_seq=0 out_seq=0 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27d9ffb640 1 -- 172.21.15.7:0/1113292257 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e37c5a9d90 con 0x55e37c6bad20 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27d9ffb640 10 client.17700 handle_command_reply: tid=0 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 client.17700 shutdown 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 msgr2=0x55e37c64d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c64d2b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55e37c634b10 tx=0x7f27e4003820 comp rx=0 tx=0).stop 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 shutdown_connections 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e37c6435b0 0x55e37c64e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e37c6bad20 0x55e37c6bd140 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e37c63e440 0x55e37c64d2b0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27d9ffb640 0 client.17700 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 --2- 172.21.15.7:0/1113292257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e37c63da50 0x55e37c64cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 >> 172.21.15.7:0/1113292257 conn(0x55e37c638fc0 msgr2=0x55e37c647e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 shutdown_connections 2024-03-19T00:48:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 1 -- 172.21.15.7:0/1113292257 wait complete. 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27fb016740 20 client.17700 trim_cache size 0 max 16384 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "30a7ae58-bc97-46dc-92cd-627ddbc494d4": { 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "30a7ae58-bc97-46dc-92cd-627ddbc494d4", 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:48:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:48:12.700 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:48:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27dc0a2880 msgr2=0x7f27dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27f8359640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27dc0a2880 0x7f27dc0a4d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f27e8023000 tx=0x7f27e8024000 comp rx=0 tx=0).stop 2024-03-19T00:48:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 msgr2=0x7f27f0069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27f8359640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 0x7f27f0069680 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00ea80 tx=0x7f27ec00ee80 comp rx=0 tx=0).stop 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.690+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 shutdown_connections 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f27dc0a2880 0x7f27dc0a4d40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27f01022f0 0x7f27f019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27f0101930 0x7f27f019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 --2- 172.21.15.7:0/288172532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f0100740 0x7f27f0069680 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 >> 172.21.15.7:0/288172532 conn(0x7f27f00fbcb0 msgr2=0x7f27f0104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 shutdown_connections 2024-03-19T00:48:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:12.691+0000 7f27f8359640 1 -- 172.21.15.7:0/288172532 wait complete. 2024-03-19T00:48:12.703 INFO:tasks.cephfs.filesystem:scrub status for tag:30a7ae58-bc97-46dc-92cd-627ddbc494d4 - {'path': '/', 'tag': '30a7ae58-bc97-46dc-92cd-627ddbc494d4', 'options': 'recursive,force'} 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: pgmap v659: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:13 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: pgmap v659: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:13 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: pgmap v659: 129 pgs: 129 active+clean; 2.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:14.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:13 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:16.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:15 smithi044 ceph-mon[66366]: pgmap v660: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:48:16.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:15 smithi007 ceph-mon[62854]: pgmap v660: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:48:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:15 smithi204 ceph-mon[66572]: pgmap v660: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-03-19T00:48:18.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:17 smithi044 ceph-mon[66366]: pgmap v661: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-03-19T00:48:18.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:17 smithi007 ceph-mon[62854]: pgmap v661: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-03-19T00:48:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:17 smithi204 ceph-mon[66572]: pgmap v661: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-03-19T00:48:20.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:19 smithi044 ceph-mon[66366]: pgmap v662: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:48:20.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:19 smithi007 ceph-mon[62854]: pgmap v662: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:48:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:19 smithi204 ceph-mon[66572]: pgmap v662: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-03-19T00:48:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:21 smithi044 ceph-mon[66366]: pgmap v663: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:21 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:48:22.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:21 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:48:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:21 smithi007 ceph-mon[62854]: pgmap v663: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:21 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:48:22.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:21 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:48:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:21 smithi204 ceph-mon[66572]: pgmap v663: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:21 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:48:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:21 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:48:24.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:23 smithi044 ceph-mon[66366]: pgmap v664: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:48:24.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:23 smithi007 ceph-mon[62854]: pgmap v664: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:48:24.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:23 smithi204 ceph-mon[66572]: pgmap v664: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:48:26.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:25 smithi044 ceph-mon[66366]: pgmap v665: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:48:26.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:25 smithi007 ceph-mon[62854]: pgmap v665: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:48:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:25 smithi204 ceph-mon[66572]: pgmap v665: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:48:28.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:27 smithi044 ceph-mon[66366]: pgmap v666: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:28.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:27 smithi007 ceph-mon[62854]: pgmap v666: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:28.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:27 smithi204 ceph-mon[66572]: pgmap v666: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:30.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:29 smithi044 ceph-mon[66366]: pgmap v667: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:30.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:29 smithi007 ceph-mon[62854]: pgmap v667: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:29 smithi204 ceph-mon[66572]: pgmap v667: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:32.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:31 smithi007 ceph-mon[62854]: pgmap v668: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:48:32.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:31 smithi204 ceph-mon[66572]: pgmap v668: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:48:32.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:31 smithi044 ceph-mon[66366]: pgmap v668: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-03-19T00:48:34.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:33 smithi007 ceph-mon[62854]: pgmap v669: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:33 smithi204 ceph-mon[66572]: pgmap v669: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:34.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:33 smithi044 ceph-mon[66366]: pgmap v669: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:499/600: open fd = 3 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:500/600: open fd = 3 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.063 INFO:tasks.workunit.client.0.smithi007.stdout:501/600: open fd = 3 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:502/600: open fd = 3 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:503/600: open fd = 3 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:504/600: open fd = 3 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:505/600: open fd = 3 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:506/600: open fd = 3 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:507/600: open fd = 3 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:508/600: open fd = 3 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:509/600: open fd = 3 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:510/600: open fd = 3 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:511/600: open fd = 3 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:512/600: open fd = 3 2024-03-19T00:48:35.066 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:513/600: open fd = 3 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:514/600: open fd = 3 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:515/600: open fd = 3 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:516/600: open fd = 3 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:517/600: open fd = 3 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:518/600: open fd = 3 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.068 INFO:tasks.workunit.client.0.smithi007.stdout:519/600: open fd = 3 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:520/600: open fd = 3 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:521/600: open fd = 3 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:522/600: open fd = 3 2024-03-19T00:48:35.069 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:523/600: open fd = 3 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:524/600: open fd = 3 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:525/600: open fd = 3 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.070 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:526/600: open fd = 3 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:527/600: open fd = 3 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:528/600: open fd = 3 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:529/600: open fd = 3 2024-03-19T00:48:35.071 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:530/600: open fd = 3 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:531/600: open fd = 3 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:532/600: open fd = 3 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.072 INFO:tasks.workunit.client.0.smithi007.stdout:533/600: open fd = 3 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:534/600: open fd = 3 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:535/600: open fd = 3 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:536/600: open fd = 3 2024-03-19T00:48:35.073 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:537/600: open fd = 3 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:538/600: open fd = 3 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:539/600: open fd = 3 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.074 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:540/600: open fd = 3 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:541/600: open fd = 3 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:542/600: open fd = 3 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:543/600: open fd = 3 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.075 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:544/600: open fd = 3 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:545/600: open fd = 3 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:546/600: open fd = 3 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:547/600: open fd = 3 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.076 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:548/600: open fd = 3 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:549/600: open fd = 3 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:550/600: open fd = 3 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.077 INFO:tasks.workunit.client.0.smithi007.stdout:551/600: open fd = 3 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:552/600: open fd = 3 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:553/600: open fd = 3 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:554/600: open fd = 3 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.078 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:555/600: open fd = 3 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:556/600: open fd = 3 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:557/600: open fd = 3 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:558/600: open fd = 3 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.079 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:559/600: open fd = 3 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:560/600: open fd = 3 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:48:35.080 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:48:36.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:35 smithi007 ceph-mon[62854]: pgmap v670: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:35 smithi204 ceph-mon[66572]: pgmap v670: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:36.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:35 smithi044 ceph-mon[66366]: pgmap v670: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:38.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:37 smithi007 ceph-mon[62854]: pgmap v671: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:38.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:37 smithi204 ceph-mon[66572]: pgmap v671: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:38.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:37 smithi044 ceph-mon[66366]: pgmap v671: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:40.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:39 smithi007 ceph-mon[62854]: pgmap v672: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:48:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:39 smithi204 ceph-mon[66572]: pgmap v672: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:48:40.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:39 smithi044 ceph-mon[66366]: pgmap v672: 129 pgs: 129 active+clean; 2.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:48:42.158 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:41 smithi044 ceph-mon[66366]: pgmap v673: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:48:42.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:41 smithi007 ceph-mon[62854]: pgmap v673: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:48:42.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:41 smithi204 ceph-mon[66572]: pgmap v673: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:48:42.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:48:42.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/916171111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 msgr2=0x7fa3dc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/916171111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc101db0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009900 tx=0x7fa3c802bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/916171111 shutdown_connections 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/916171111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc1022f0 0x7fa3dc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/916171111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc101db0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/916171111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3dc100740 0x7fa3dc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/916171111 >> 172.21.15.7:0/916171111 conn(0x7fa3dc0fbcb0 msgr2=0x7fa3dc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/916171111 shutdown_connections 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/916171111 wait complete. 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 Processor -- start 2024-03-19T00:48:42.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- start start 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100740 0x7fa3dc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3dc1022f0 0x7fa3dc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10d3f0 con 0x7fa3dc101930 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10d270 con 0x7fa3dc100740 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.821+0000 7fa3e2ddf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10d570 con 0x7fa3dc1022f0 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100740 0x7fa3dc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38492/0 (socket says 172.21.15.7:38492) 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 -- 172.21.15.7:0/4202374963 learned_addr learned my addr 172.21.15.7:0/4202374963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100740 0x7fa3dc197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35250/0 (socket says 172.21.15.7:35250) 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3e137c640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3dc1022f0 0x7fa3dc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 -- 172.21.15.7:0/4202374963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3dc1022f0 msgr2=0x7fa3dc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3dc1022f0 0x7fa3dc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 -- 172.21.15.7:0/4202374963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100740 msgr2=0x7fa3dc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100740 0x7fa3dc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 -- 172.21.15.7:0/4202374963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc19f750 con 0x7fa3dc101930 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc1980c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006010 tx=0x7fa3c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3d9ffb640 1 -- 172.21.15.7:0/4202374963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8043020 con 0x7fa3dc101930 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3d9ffb640 1 -- 172.21.15.7:0/4202374963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3c8004440 con 0x7fa3dc101930 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc19f9e0 con 0x7fa3dc101930 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.822+0000 7fa3d9ffb640 1 -- 172.21.15.7:0/4202374963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8003a80 con 0x7fa3dc101930 2024-03-19T00:48:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.823+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc19ff80 con 0x7fa3dc101930 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3d9ffb640 1 -- 172.21.15.7:0/4202374963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa3c8046020 con 0x7fa3dc101930 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 Processor -- start 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 -- start start 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ecfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673ed4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673ed7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558673e34500 con 0x558673ecfa50 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558673ddcbe0 con 0x558673ed0440 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3d9ffb640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3cc0a2880 0x7fa3cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.825+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558673e305a0 con 0x558673ed55b0 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673ed4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673ed4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35254/0 (socket says 172.21.15.7:35254) 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/3097408873 learned_addr learned my addr 172.21.15.7:0/3097408873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e137c640 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ecfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673ed7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3d9ffb640 1 -- 172.21.15.7:0/4202374963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa3c80f0ff0 con 0x7fa3dc101930 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e137c640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3cc0a2880 0x7fa3cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/3097408873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 msgr2=0x558673ed7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673ed7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/3097408873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 msgr2=0x558673ecfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ecfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/3097408873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558673dfe000 con 0x558673ed0440 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673ed4f40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009a40 tx=0x7fa3d0009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=792f69924abc4066 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3e137c640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3cc0a2880 0x7fa3cc0a4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x558673ed0ca0 tx=0x7fa3d4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3c6ffd640 1 -- 172.21.15.7:0/3097408873 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0017070 con 0x558673ed0440 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3c6ffd640 1 -- 172.21.15.7:0/3097408873 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3d0013470 con 0x558673ed0440 2024-03-19T00:48:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.826+0000 7fa3c6ffd640 1 -- 172.21.15.7:0/3097408873 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00125d0 con 0x558673ed0440 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 -- 172.21.15.7:0/3097408873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 msgr2=0x558673ed4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673ed4f40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009a40 tx=0x7fa3d0009e40 comp rx=0 tx=0).stop 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 -- 172.21.15.7:0/3097408873 shutdown_connections 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673ed7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673ed4f40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/3097408873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ecfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 -- 172.21.15.7:0/3097408873 >> 172.21.15.7:0/3097408873 conn(0x558673ecafc0 msgr2=0x558673ecb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 -- 172.21.15.7:0/3097408873 shutdown_connections 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 -- 172.21.15.7:0/3097408873 wait complete. 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 -- start start 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.827+0000 7fa3e59ca740 1 Processor -- start 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ecfa50 0x558673ed9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673eda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ee1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558673ed8560 con 0x558673ed55b0 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558673df0160 con 0x558673ed0440 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558673ed86e0 con 0x558673ecfa50 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e59ca740 10 client.0 _ll_register_callbacks cb 0x558673e2ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e137c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ee1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ecfa50 0x558673ed9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673eda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e137c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ee1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38512/0 (socket says 172.21.15.7:38512) 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e137c640 1 -- 172.21.15.7:0/2547138491 learned_addr learned my addr 172.21.15.7:0/2547138491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 msgr2=0x558673eda3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673eda3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/2547138491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 msgr2=0x558673ee1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ee1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/2547138491 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558673c127d0 con 0x558673ecfa50 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e137c640 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ee1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ecfa50 0x558673ed9eb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x558673ed0840 tx=0x7fa3d000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a272cb9fb39bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.828+0000 7fa3bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3bbfff640 1 -- 172.21.15.7:0/2547138491 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0017050 con 0x558673ecfa50 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3bbfff640 1 -- 172.21.15.7:0/2547138491 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa3d001c450 con 0x558673ecfa50 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d0013470 con 0x558673ecfa50 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3bbfff640 1 -- 172.21.15.7:0/2547138491 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3d002f050 con 0x558673ecfa50 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 10 client.27466 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558673b483f0 con 0x558673ecfa50 2024-03-19T00:48:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3bbfff640 1 -- 172.21.15.7:0/2547138491 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3d0013470 con 0x558673ecfa50 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3bbfff640 1 -- 172.21.15.7:0/2547138491 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d00265d0 con 0x558673ecfa50 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 10 client.27466 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 20 client.27466 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673ee59d0 0x558673ee7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 4 client.27466 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.829+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558673e41a40 con 0x558673ee59d0 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.830+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673ee59d0 0x558673ee7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.830+0000 7fa3bbfff640 10 client.27466 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.830+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673ee59d0 0x558673ee7dc0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2698805cb1c51b20 server_cookie=e60be2c7947f6246 in_seq=0 out_seq=0 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.830+0000 7fa3bbfff640 1 -- 172.21.15.7:0/2547138491 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558673e41a40 con 0x558673ee59d0 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.830+0000 7fa3bbfff640 10 client.27466 handle_command_reply: tid=0 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.830+0000 7fa3e59ca740 1 client.27466 shutdown 2024-03-19T00:48:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ecfa50 msgr2=0x558673ed9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ecfa50 0x558673ed9eb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x558673ed0840 tx=0x7fa3d000ef60 comp rx=0 tx=0).stop 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 shutdown_connections 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673ee59d0 0x558673ee7dc0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ee1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673eda3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/2547138491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ecfa50 0x558673ed9eb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3bbfff640 0 client.27466 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 >> 172.21.15.7:0/2547138491 conn(0x558673ecafc0 msgr2=0x558673eca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 shutdown_connections 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 1 -- 172.21.15.7:0/2547138491 wait complete. 2024-03-19T00:48:42.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.831+0000 7fa3e59ca740 20 client.27466 trim_cache size 0 max 16384 2024-03-19T00:48:42.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 Processor -- start 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 -- start start 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ecfa50 0x558673ed94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed0440 0x558673ee13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ed9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1acbb0 con 0x558673ed55b0 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1aa480 con 0x558673ecfa50 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a82d0 con 0x558673ed0440 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e137c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ed9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ecfa50 0x558673ed94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e137c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ed9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38518/0 (socket says 172.21.15.7:38518) 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e137c640 1 -- 172.21.15.7:0/4081528419 learned_addr learned my addr 172.21.15.7:0/4081528419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed0440 0x558673ee13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/4081528419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed0440 msgr2=0x558673ee13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed0440 0x558673ee13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/4081528419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 msgr2=0x558673ed9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ed9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/4081528419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558673b080f0 con 0x558673ecfa50 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e137c640 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ed9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.849+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ecfa50 0x558673ed94d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009150 tx=0x7fa3d0009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d75766d578e66797 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3c67fc640 1 -- 172.21.15.7:0/4081528419 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0017050 con 0x558673ecfa50 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3c67fc640 1 -- 172.21.15.7:0/4081528419 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3d00138b0 con 0x558673ecfa50 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3c67fc640 1 -- 172.21.15.7:0/4081528419 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d001c6b0 con 0x558673ecfa50 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 -- 172.21.15.7:0/4081528419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ecfa50 msgr2=0x558673ed94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ecfa50 0x558673ed94d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009150 tx=0x7fa3d0009e40 comp rx=0 tx=0).stop 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 -- 172.21.15.7:0/4081528419 shutdown_connections 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ed55b0 0x558673ed9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed0440 0x558673ee13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/4081528419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ecfa50 0x558673ed94d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 -- 172.21.15.7:0/4081528419 >> 172.21.15.7:0/4081528419 conn(0x558673ecafc0 msgr2=0x558673eceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 -- 172.21.15.7:0/4081528419 shutdown_connections 2024-03-19T00:48:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.850+0000 7fa3e59ca740 1 -- 172.21.15.7:0/4081528419 wait complete. 2024-03-19T00:48:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 -- start start 2024-03-19T00:48:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 Processor -- start 2024-03-19T00:48:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ede550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673edea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673edfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a8940 con 0x558673ecfa50 2024-03-19T00:48:42.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a9db0 con 0x558673ed0440 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a9010 con 0x558673ed55b0 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e59ca740 10 client.0 _ll_register_callbacks cb 0x7fa3c802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ede550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e0b7b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ede550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38532/0 (socket says 172.21.15.7:38532) 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/1712494352 learned_addr learned my addr 172.21.15.7:0/1712494352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673edea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.851+0000 7fa3e137c640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673edfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/1712494352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 msgr2=0x558673edfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673edfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 msgr2=0x558673edea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673edea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e0b7b640 1 -- 172.21.15.7:0/1712494352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558673ee0090 con 0x558673ecfa50 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e0b7b640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ede550 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00094e0 tx=0x7fa3d0026f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2824125d0fd3925b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3bbfff640 1 -- 172.21.15.7:0/1712494352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0017070 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3bbfff640 1 -- 172.21.15.7:0/1712494352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa3d00265e0 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d00138b0 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3bbfff640 1 -- 172.21.15.7:0/1712494352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3d0046050 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 10 client.17718 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558673ee13a0 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3bbfff640 1 -- 172.21.15.7:0/1712494352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3d00138b0 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3bbfff640 1 -- 172.21.15.7:0/1712494352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d003b3f0 con 0x558673ecfa50 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 10 client.17718 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 20 client.17718 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673f4bf90 0x558673f4e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.852+0000 7fa3e59ca740 4 client.17718 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558673e3bd90 con 0x558673f4bf90 2024-03-19T00:48:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673f4bf90 0x558673f4e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3bbfff640 10 client.17718 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3dbfff640 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673f4bf90 0x558673f4e380 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7319ecdd89b58ed6 server_cookie=2ad30182dac52818 in_seq=0 out_seq=0 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3bbfff640 1 -- 172.21.15.7:0/1712494352 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558673e3bd90 con 0x558673f4bf90 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3bbfff640 10 client.17718 handle_command_reply: tid=0 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 client.17718 shutdown 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 msgr2=0x558673ede550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ede550 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00094e0 tx=0x7fa3d0026f10 comp rx=0 tx=0).stop 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 shutdown_connections 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558673ed55b0 0x558673edfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558673ed0440 0x558673edea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x558673f4bf90 0x558673f4e380 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 --2- 172.21.15.7:0/1712494352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558673ecfa50 0x558673ede550 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3bbfff640 0 client.17718 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.853+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 >> 172.21.15.7:0/1712494352 conn(0x558673ecafc0 msgr2=0x558673ec9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 shutdown_connections 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e59ca740 1 -- 172.21.15.7:0/1712494352 wait complete. 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e59ca740 20 client.17718 trim_cache size 0 max 16384 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:48:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3cc0a2880 msgr2=0x7fa3cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3cc0a2880 0x7fa3cc0a4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x558673ed0ca0 tx=0x7fa3d4009500 comp rx=0 tx=0).stop 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 msgr2=0x7fa3dc1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc1980c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006010 tx=0x7fa3c8004040 comp rx=0 tx=0).stop 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 shutdown_connections 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa3cc0a2880 0x7fa3cc0a4d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3dc1022f0 0x7fa3dc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3dc101930 0x7fa3dc1980c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 --2- 172.21.15.7:0/4202374963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3dc100740 0x7fa3dc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 >> 172.21.15.7:0/4202374963 conn(0x7fa3dc0fbcb0 msgr2=0x7fa3dc104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 shutdown_connections 2024-03-19T00:48:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.854+0000 7fa3e2ddf640 1 -- 172.21.15.7:0/4202374963 wait complete. 2024-03-19T00:48:42.866 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:48:42.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 -- 172.21.15.7:0/2822861504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 msgr2=0x7fa054100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 --2- 172.21.15.7:0/2822861504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa054100f00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa048009900 tx=0x7fa04802bcd0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 -- 172.21.15.7:0/2822861504 shutdown_connections 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 --2- 172.21.15.7:0/2822861504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa054101440 0x7fa0541018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 --2- 172.21.15.7:0/2822861504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa054100f00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 --2- 172.21.15.7:0/2822861504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054100140 0x7fa054100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 -- 172.21.15.7:0/2822861504 >> 172.21.15.7:0/2822861504 conn(0x7fa0540fbce0 msgr2=0x7fa0540fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 -- 172.21.15.7:0/2822861504 shutdown_connections 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 -- 172.21.15.7:0/2822861504 wait complete. 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.968+0000 7fa05917e640 1 Processor -- start 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 -- start start 2024-03-19T00:48:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa054100140 0x7fa054199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa05419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054101440 0x7fa0541a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410f560 con 0x7fa054100aa0 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410f3e0 con 0x7fa054100140 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05917e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410f6e0 con 0x7fa054101440 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa05419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa05419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38548/0 (socket says 172.21.15.7:38548) 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05259c640 1 -- 172.21.15.7:0/4241409292 learned_addr learned my addr 172.21.15.7:0/4241409292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa052d9d640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa054100140 0x7fa054199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054101440 0x7fa0541a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 -- 172.21.15.7:0/4241409292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa054100140 msgr2=0x7fa054199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa054100140 0x7fa054199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 -- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 msgr2=0x7fa05419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa05419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 -- 172.21.15.7:0/4241409292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0541a18d0 con 0x7fa054101440 2024-03-19T00:48:42.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05259c640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa05419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:42.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.969+0000 7fa05359e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054101440 0x7fa0541a1230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00ec50 tx=0x7fa04c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.970+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c003b70 con 0x7fa054101440 2024-03-19T00:48:42.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.970+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04c003d10 con 0x7fa054101440 2024-03-19T00:48:42.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.970+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0541a1bc0 con 0x7fa054101440 2024-03-19T00:48:42.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.970+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010760 con 0x7fa054101440 2024-03-19T00:48:42.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.970+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a2110 con 0x7fa054101440 2024-03-19T00:48:42.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.971+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05410b0e0 con 0x7fa054101440 2024-03-19T00:48:42.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.971+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa04c004020 con 0x7fa054101440 2024-03-19T00:48:42.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.972+0000 7fa03bfff640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa03c0a2880 0x7fa03c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:42.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.972+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa04c0cf320 con 0x7fa054101440 2024-03-19T00:48:42.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.972+0000 7fa052d9d640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa03c0a2880 0x7fa03c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:42.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.972+0000 7fa052d9d640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa03c0a2880 0x7fa03c0a4d40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa05419ab60 tx=0x7fa044005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:42.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:42.977+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa04c09c060 con 0x7fa054101440 2024-03-19T00:48:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.199+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa05402d1f0 con 0x7fa054101440 2024-03-19T00:48:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.199+0000 7fa03bfff640 1 -- 172.21.15.7:0/4241409292 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7fa04c09b960 con 0x7fa054101440 2024-03-19T00:48:43.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:48:43.201 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:48:43.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa03c0a2880 msgr2=0x7fa03c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa03c0a2880 0x7fa03c0a4d40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa05419ab60 tx=0x7fa044005f90 comp rx=0 tx=0).stop 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054101440 msgr2=0x7fa0541a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054101440 0x7fa0541a1230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00ec50 tx=0x7fa04c00c6a0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 shutdown_connections 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa03c0a2880 0x7fa03c0a4d40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa054101440 0x7fa0541a1230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa054100aa0 0x7fa05419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 --2- 172.21.15.7:0/4241409292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa054100140 0x7fa054199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 >> 172.21.15.7:0/4241409292 conn(0x7fa0540fbce0 msgr2=0x7fa0540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 shutdown_connections 2024-03-19T00:48:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.202+0000 7fa05917e640 1 -- 172.21.15.7:0/4241409292 wait complete. 2024-03-19T00:48:43.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:48:43.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 -- 172.21.15.7:0/664816222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66401033e0 msgr2=0x7f6640109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 --2- 172.21.15.7:0/664816222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66401033e0 0x7f6640109c70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ca0 tx=0x7f663002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 -- 172.21.15.7:0/664816222 shutdown_connections 2024-03-19T00:48:43.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 --2- 172.21.15.7:0/664816222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66401033e0 0x7f6640109c70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 --2- 172.21.15.7:0/664816222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6640102a20 0x7f6640102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 --2- 172.21.15.7:0/664816222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6640101830 0x7f6640101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 -- 172.21.15.7:0/664816222 >> 172.21.15.7:0/664816222 conn(0x7f66400fcd60 msgr2=0x7f66400ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 -- 172.21.15.7:0/664816222 shutdown_connections 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.317+0000 7f66458b2640 1 -- 172.21.15.7:0/664816222 wait complete. 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 Processor -- start 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 -- start start 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 0x7f66401979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6640102a20 0x7f6640197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66401033e0 0x7f664019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f664010d360 con 0x7f6640101830 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f664010d1e0 con 0x7f6640102a20 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f66458b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f664010d4e0 con 0x7f66401033e0 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f663effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 0x7f66401979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f663effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 0x7f66401979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38568/0 (socket says 172.21.15.7:38568) 2024-03-19T00:48:43.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f663effd640 1 -- 172.21.15.7:0/2295104942 learned_addr learned my addr 172.21.15.7:0/2295104942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f663e7fc640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6640102a20 0x7f6640197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66401033e0 0x7f664019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.318+0000 7f663effd640 1 -- 172.21.15.7:0/2295104942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66401033e0 msgr2=0x7f664019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f663effd640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66401033e0 0x7f664019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f663effd640 1 -- 172.21.15.7:0/2295104942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6640102a20 msgr2=0x7f6640197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f663effd640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6640102a20 0x7f6640197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f663effd640 1 -- 172.21.15.7:0/2295104942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664019f5d0 con 0x7f6640101830 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f663effd640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 0x7f66401979b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f663800e9c0 tx=0x7f663800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f66448b0640 1 -- 172.21.15.7:0/2295104942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800cd80 con 0x7f6640101830 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f66448b0640 1 -- 172.21.15.7:0/2295104942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66380035f0 con 0x7f6640101830 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664019f8c0 con 0x7f6640101830 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f66448b0640 1 -- 172.21.15.7:0/2295104942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638010690 con 0x7f6640101830 2024-03-19T00:48:43.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.319+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664019fdc0 con 0x7f6640101830 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f66448b0640 1 -- 172.21.15.7:0/2295104942 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f66380037a0 con 0x7f6640101830 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 Processor -- start 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 -- start start 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139ad2f0 0x55f4139ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139adce0 0x55f4139b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139b2e50 0x55f4139b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4139114b0 con 0x55f4139ad2f0 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f4138b9d70 con 0x55f4139b2e50 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f664857b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f41390d550 con 0x55f4139adce0 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f66448b0640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f66280a2800 0x7f66280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139ad2f0 0x55f4139ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139ad2f0 0x55f4139ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38578/0 (socket says 172.21.15.7:38578) 2024-03-19T00:48:43.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f663f7fe640 1 -- 172.21.15.7:0/1718541675 learned_addr learned my addr 172.21.15.7:0/1718541675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f663effd640 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139adce0 0x55f4139b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f663e7fc640 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139b2e50 0x55f4139b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.322+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f66280a2800 0x7f66280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f66448b0640 1 -- 172.21.15.7:0/2295104942 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f66380cf310 con 0x7f6640101830 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663e7fc640 1 -- 172.21.15.7:0/1718541675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139adce0 msgr2=0x55f4139b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663e7fc640 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139adce0 0x55f4139b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663e7fc640 1 -- 172.21.15.7:0/1718541675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139ad2f0 msgr2=0x55f4139ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663e7fc640 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139ad2f0 0x55f4139ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663e7fc640 1 -- 172.21.15.7:0/1718541675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4138dc210 con 0x55f4139b2e50 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f66280a2800 0x7f66280a4cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55f4139ae540 tx=0x7f6630009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f663e7fc640 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139b2e50 0x55f4139b52e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6634009bf0 tx=0x7f66340107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29440f16c1658da1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f6625ffb640 1 -- 172.21.15.7:0/1718541675 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663401e070 con 0x55f4139b2e50 2024-03-19T00:48:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f6625ffb640 1 -- 172.21.15.7:0/1718541675 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66340198f0 con 0x55f4139b2e50 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.323+0000 7f6625ffb640 1 -- 172.21.15.7:0/1718541675 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634011990 con 0x55f4139b2e50 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 -- 172.21.15.7:0/1718541675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139b2e50 msgr2=0x55f4139b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139b2e50 0x55f4139b52e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6634009bf0 tx=0x7f66340107a0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 -- 172.21.15.7:0/1718541675 shutdown_connections 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139b2e50 0x55f4139b52e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139adce0 0x55f4139b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 --2- 172.21.15.7:0/1718541675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139ad2f0 0x55f4139ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 -- 172.21.15.7:0/1718541675 >> 172.21.15.7:0/1718541675 conn(0x55f4139a8860 msgr2=0x55f4139a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 -- 172.21.15.7:0/1718541675 shutdown_connections 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 -- 172.21.15.7:0/1718541675 wait complete. 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 -- start start 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.324+0000 7f664857b740 1 Processor -- start 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4139b5e00 con 0x55f4139b2e50 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f4138cd310 con 0x55f4139adce0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4139b5f80 con 0x55f4139ad2f0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f664857b740 10 client.0 _ll_register_callbacks cb 0x55f4138c12d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38590/0 (socket says 172.21.15.7:38590) 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663f7fe640 1 -- 172.21.15.7:0/2135164852 learned_addr learned my addr 172.21.15.7:0/2135164852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663effd640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 -- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 msgr2=0x55f4139b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 -- 172.21.15.7:0/2135164852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 msgr2=0x55f4139bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 -- 172.21.15.7:0/2135164852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4136253f0 con 0x55f4139adce0 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f663e7fc640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139b7d20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55f4139a97d0 tx=0x7f66340196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679be65f39f4e6fb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f661effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.325+0000 7f661effd640 1 -- 172.21.15.7:0/2135164852 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663401e070 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f661effd640 1 -- 172.21.15.7:0/2135164852 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6634005040 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66340198f0 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f661effd640 1 -- 172.21.15.7:0/2135164852 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6634031050 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 10 client.27581 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4136ef7d0 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f661effd640 1 -- 172.21.15.7:0/2135164852 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66340198f0 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f661effd640 1 -- 172.21.15.7:0/2135164852 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f66340035f0 con 0x55f4139adce0 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 10 client.27581 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 10 client.27581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 20 client.27581 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4139c3350 0x55f4139c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 4 client.27581 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f41396f3e0 con 0x55f4139c3350 2024-03-19T00:48:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.326+0000 7f663effd640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4139c3350 0x55f4139c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f661effd640 10 client.27581 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f663effd640 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4139c3350 0x55f4139c5740 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dd1b4484de08ce1 server_cookie=bd118b9121809bb0 in_seq=0 out_seq=0 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f661effd640 1 -- 172.21.15.7:0/2135164852 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f41396f3e0 con 0x55f4139c3350 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f661effd640 10 client.27581 handle_command_reply: tid=0 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f664857b740 1 client.27581 shutdown 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 msgr2=0x55f4139b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f664857b740 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139b7d20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55f4139a97d0 tx=0x7f66340196f0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.327+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 shutdown_connections 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f4139c3350 0x55f4139c5740 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139b7d20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 --2- 172.21.15.7:0/2135164852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f661effd640 0 client.27581 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 >> 172.21.15.7:0/2135164852 conn(0x55f4139a8860 msgr2=0x55f4139a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 shutdown_connections 2024-03-19T00:48:43.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 1 -- 172.21.15.7:0/2135164852 wait complete. 2024-03-19T00:48:43.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.328+0000 7f664857b740 20 client.27581 trim_cache size 0 max 16384 2024-03-19T00:48:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 Processor -- start 2024-03-19T00:48:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 -- start start 2024-03-19T00:48:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66401aa8f0 con 0x55f4139b2e50 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66401a81c0 con 0x55f4139adce0 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f664857b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66401a6010 con 0x55f4139ad2f0 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38602/0 (socket says 172.21.15.7:38602) 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 -- 172.21.15.7:0/3009351521 learned_addr learned my addr 172.21.15.7:0/3009351521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663effd640 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663e7fc640 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 -- 172.21.15.7:0/3009351521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 msgr2=0x55f4139af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 -- 172.21.15.7:0/3009351521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 msgr2=0x55f4139af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 -- 172.21.15.7:0/3009351521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41397e880 con 0x55f4139b2e50 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.341+0000 7f663f7fe640 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6630005e30 tx=0x7f663002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc9dcb2a84d3b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f66257fa640 1 -- 172.21.15.7:0/3009351521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630039040 con 0x55f4139b2e50 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f66257fa640 1 -- 172.21.15.7:0/3009351521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6630054070 con 0x55f4139b2e50 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f66257fa640 1 -- 172.21.15.7:0/3009351521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630034b40 con 0x55f4139b2e50 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 -- 172.21.15.7:0/3009351521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 msgr2=0x55f4139bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6630005e30 tx=0x7f663002e040 comp rx=0 tx=0).stop 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 -- 172.21.15.7:0/3009351521 shutdown_connections 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bed20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 --2- 172.21.15.7:0/3009351521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 -- 172.21.15.7:0/3009351521 >> 172.21.15.7:0/3009351521 conn(0x55f4139a8860 msgr2=0x55f4139b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 -- 172.21.15.7:0/3009351521 shutdown_connections 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 -- 172.21.15.7:0/3009351521 wait complete. 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.342+0000 7f664857b740 1 -- start start 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 Processor -- start 2024-03-19T00:48:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139bcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66401a6680 con 0x55f4139b2e50 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66401a7af0 con 0x55f4139adce0 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66401a6d50 con 0x55f4139ad2f0 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f664857b740 10 client.0 _ll_register_callbacks cb 0x7f663401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38604/0 (socket says 172.21.15.7:38604) 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 -- 172.21.15.7:0/2913066591 learned_addr learned my addr 172.21.15.7:0/2913066591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663e7fc640 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139bcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 -- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 msgr2=0x55f4139bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 -- 172.21.15.7:0/2913066591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 msgr2=0x55f4139bcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139bcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 -- 172.21.15.7:0/2913066591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4139be1e0 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f663f7fe640 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bdbf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6630009a60 tx=0x7f66300324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5fbacf4e44f2c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.343+0000 7f661effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f661effd640 1 -- 172.21.15.7:0/2913066591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630039040 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f661effd640 1 -- 172.21.15.7:0/2913066591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6630004d40 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6630054070 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f661effd640 1 -- 172.21.15.7:0/2913066591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6630004410 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 10 client.17742 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f413a29860 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f661effd640 1 -- 172.21.15.7:0/2913066591 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6630054070 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f661effd640 1 -- 172.21.15.7:0/2913066591 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6630005150 con 0x55f4139b2e50 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 10 client.17742 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 10 client.17742 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:48:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 20 client.17742 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f413a2a2e0 0x55f413a2c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 4 client.17742 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f41391e9f0 con 0x55f413a2a2e0 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.344+0000 7f663effd640 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f413a2a2e0 0x55f413a2c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f661effd640 10 client.17742 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f663effd640 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f413a2a2e0 0x55f413a2c700 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c901af63f75b6c5 server_cookie=7462bacd2a22a5e2 in_seq=0 out_seq=0 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f661effd640 1 -- 172.21.15.7:0/2913066591 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f41391e9f0 con 0x55f413a2a2e0 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f661effd640 10 client.17742 handle_command_reply: tid=0 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 client.17742 shutdown 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 msgr2=0x55f4139bdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bdbf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6630009a60 tx=0x7f66300324f0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 shutdown_connections 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55f413a2a2e0 0x55f413a2c700 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4139b2e50 0x55f4139bdbf0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4139adce0 0x55f4139bcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 --2- 172.21.15.7:0/2913066591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4139ad2f0 0x55f4139bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f661effd640 0 client.17742 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 >> 172.21.15.7:0/2913066591 conn(0x55f4139a8860 msgr2=0x55f4139b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 shutdown_connections 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 1 -- 172.21.15.7:0/2913066591 wait complete. 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.345+0000 7f664857b740 20 client.17742 trim_cache size 0 max 16384 2024-03-19T00:48:43.353 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f66280a2800 msgr2=0x7f66280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f66280a2800 0x7f66280a4cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55f4139ae540 tx=0x7f6630009360 comp rx=0 tx=0).stop 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 msgr2=0x7f66401979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 0x7f66401979b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f663800e9c0 tx=0x7f663800ee80 comp rx=0 tx=0).stop 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 shutdown_connections 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f66280a2800 0x7f66280a4cc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66401033e0 0x7f664019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6640102a20 0x7f6640197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 --2- 172.21.15.7:0/2295104942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640101830 0x7f66401979b0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 >> 172.21.15.7:0/2295104942 conn(0x7f66400fcd60 msgr2=0x7f6640105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 shutdown_connections 2024-03-19T00:48:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.346+0000 7f66458b2640 1 -- 172.21.15.7:0/2295104942 wait complete. 2024-03-19T00:48:43.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:48:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 -- 172.21.15.7:0/3520633523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 msgr2=0x7ffb10104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 --2- 172.21.15.7:0/3520633523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10104760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009900 tx=0x7ffb0c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 -- 172.21.15.7:0/3520633523 shutdown_connections 2024-03-19T00:48:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 --2- 172.21.15.7:0/3520633523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb101075d0 0x7ffb101099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 --2- 172.21.15.7:0/3520633523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb10104ca0 0x7ffb10107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 --2- 172.21.15.7:0/3520633523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10104760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 -- 172.21.15.7:0/3520633523 >> 172.21.15.7:0/3520633523 conn(0x7ffb100fbc80 msgr2=0x7ffb100fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 -- 172.21.15.7:0/3520633523 shutdown_connections 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.461+0000 7ffb16c61640 1 -- 172.21.15.7:0/3520633523 wait complete. 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 Processor -- start 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 -- start start 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb10104ca0 0x7ffb10197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb101075d0 0x7ffb1019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1010d390 con 0x7ffb10104ca0 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1010d210 con 0x7ffb10102370 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb16c61640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1010d510 con 0x7ffb101075d0 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb10104ca0 0x7ffb10197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35404/0 (socket says 172.21.15.7:35404) 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb149fd640 1 -- 172.21.15.7:0/276903235 learned_addr learned my addr 172.21.15.7:0/276903235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb151fe640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb101075d0 0x7ffb1019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 -- 172.21.15.7:0/276903235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb101075d0 msgr2=0x7ffb1019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb101075d0 0x7ffb1019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 -- 172.21.15.7:0/276903235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 msgr2=0x7ffb10197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 -- 172.21.15.7:0/276903235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1019f680 con 0x7ffb10104ca0 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb149fd640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.462+0000 7ffb07fff640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb10104ca0 0x7ffb10197fa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000e9c0 tx=0x7ffb0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.463+0000 7ffb05ffb640 1 -- 172.21.15.7:0/276903235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000ccc0 con 0x7ffb10104ca0 2024-03-19T00:48:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.463+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1019f970 con 0x7ffb10104ca0 2024-03-19T00:48:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.463+0000 7ffb05ffb640 1 -- 172.21.15.7:0/276903235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb00005020 con 0x7ffb10104ca0 2024-03-19T00:48:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.463+0000 7ffb05ffb640 1 -- 172.21.15.7:0/276903235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0001c070 con 0x7ffb10104ca0 2024-03-19T00:48:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.463+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1019fec0 con 0x7ffb10104ca0 2024-03-19T00:48:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb05ffb640 1 -- 172.21.15.7:0/276903235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7ffb000051d0 con 0x7ffb10104ca0 2024-03-19T00:48:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 Processor -- start 2024-03-19T00:48:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 -- start start 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fcfb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fd00e50 0x55ea0fd032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea0fc5f4b0 con 0x55ea0fcfbce0 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ea0fc07d70 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb19853740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea0fc5b550 con 0x55ea0fd00e50 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.465+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd007e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38630/0 (socket says 172.21.15.7:38630) 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb149fd640 1 -- 172.21.15.7:0/3443950870 learned_addr learned my addr 172.21.15.7:0/3443950870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fcfb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb07fff640 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fd00e50 0x55ea0fd032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb05ffb640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ffaf80a2800 0x7ffaf80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 -- 172.21.15.7:0/3443950870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fd00e50 msgr2=0x55ea0fd032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fd00e50 0x55ea0fd032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 -- 172.21.15.7:0/3443950870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 msgr2=0x55ea0fd007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 -- 172.21.15.7:0/3443950870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0fc2a210 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ffaf80a2800 0x7ffaf80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb05ffb640 1 -- 172.21.15.7:0/276903235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ffb000ced50 con 0x7ffb10104ca0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffb151fe640 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fcfb710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009fd0 tx=0x7ffb08011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10c6df669e06416 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffaf6ffd640 1 -- 172.21.15.7:0/3443950870 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08017070 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffaf6ffd640 1 -- 172.21.15.7:0/3443950870 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0800fbf0 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.466+0000 7ffaf6ffd640 1 -- 172.21.15.7:0/3443950870 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0801b980 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 -- 172.21.15.7:0/3443950870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 msgr2=0x55ea0fcfb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fcfb710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009fd0 tx=0x7ffb08011920 comp rx=0 tx=0).stop 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb151fe640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ffaf80a2800 0x7ffaf80a4cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffb08022000 tx=0x7ffb08023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 -- 172.21.15.7:0/3443950870 shutdown_connections 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fd00e50 0x55ea0fd032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 --2- 172.21.15.7:0/3443950870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fcfb710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 -- 172.21.15.7:0/3443950870 >> 172.21.15.7:0/3443950870 conn(0x55ea0fcf6860 msgr2=0x55ea0fcf6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 -- 172.21.15.7:0/3443950870 shutdown_connections 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 -- 172.21.15.7:0/3443950870 wait complete. 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 -- start start 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.467+0000 7ffb19853740 1 Processor -- start 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fd0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea0fd03e00 con 0x55ea0fd00e50 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ea0fc1b310 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea0fd03f80 con 0x55ea0fcfbce0 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 10 client.0 _ll_register_callbacks cb 0x55ea0fc0f2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fd0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fd0cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38644/0 (socket says 172.21.15.7:38644) 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 -- 172.21.15.7:0/893643451 learned_addr learned my addr 172.21.15.7:0/893643451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 -- 172.21.15.7:0/893643451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 msgr2=0x55ea0fd05d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 -- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 msgr2=0x55ea0fd05810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 -- 172.21.15.7:0/893643451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0f9733f0 con 0x55ea0fd00e50 2024-03-19T00:48:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb149fd640 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb151fe640 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fd0cd90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ffb08003fd0 tx=0x7ffb080039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f382b80c4ab4e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffaeffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffaeffff640 1 -- 172.21.15.7:0/893643451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08035060 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffaeffff640 1 -- 172.21.15.7:0/893643451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ffb0801b720 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.468+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0800fbf0 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffaeffff640 1 -- 172.21.15.7:0/893643451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb0803c050 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 10 client.17754 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea0fa3d7d0 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffaeffff640 1 -- 172.21.15.7:0/893643451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0800fbf0 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffaeffff640 1 -- 172.21.15.7:0/893643451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffb08026560 con 0x55ea0fd00e50 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 10 client.17754 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 10 client.17754 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:48:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 20 client.17754 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd11440 0x55ea0fd13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 4 client.17754 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0fcbd3e0 con 0x55ea0fd11440 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.469+0000 7ffb149fd640 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd11440 0x55ea0fd13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.470+0000 7ffb149fd640 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd11440 0x55ea0fd13890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a711e099711c316f server_cookie=c6cdd24eca20529a in_seq=0 out_seq=0 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.470+0000 7ffaeffff640 10 client.17754 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffaeffff640 1 -- 172.21.15.7:0/893643451 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ea0fcbd3e0 con 0x55ea0fd11440 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffaeffff640 10 client.17754 handle_command_reply: tid=0 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 client.17754 shutdown 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 msgr2=0x55ea0fd0cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fd0cd90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ffb08003fd0 tx=0x7ffb080039b0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 shutdown_connections 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd11440 0x55ea0fd13890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fd0cd90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffaeffff640 0 client.17754 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 --2- 172.21.15.7:0/893643451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 >> 172.21.15.7:0/893643451 conn(0x55ea0fcf6860 msgr2=0x55ea0fcf5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 shutdown_connections 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 1 -- 172.21.15.7:0/893643451 wait complete. 2024-03-19T00:48:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.471+0000 7ffb19853740 20 client.17754 trim_cache size 0 max 16384 2024-03-19T00:48:43.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 Processor -- start 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 -- start start 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fcf5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101acaf0 con 0x55ea0fd00e50 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101aa3c0 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb19853740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101a8210 con 0x55ea0fcfbce0 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb151fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fcf5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35426/0 (socket says 172.21.15.7:35426) 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb149fd640 1 -- 172.21.15.7:0/2071744202 learned_addr learned my addr 172.21.15.7:0/2071744202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.484+0000 7ffb151fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fcf5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38648/0 (socket says 172.21.15.7:38648) 2024-03-19T00:48:43.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb07fff640 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb149fd640 1 -- 172.21.15.7:0/2071744202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 msgr2=0x55ea0fd05d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb149fd640 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb149fd640 1 -- 172.21.15.7:0/2071744202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 msgr2=0x55ea0fcf5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb149fd640 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fcf5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb149fd640 1 -- 172.21.15.7:0/2071744202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0fcfaf30 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb07fff640 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb149fd640 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55ea0fcfc750 tx=0x7ffb0c02c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21837593f52b1ccf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffaf67fc640 1 -- 172.21.15.7:0/2071744202 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c02cc90 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffaf67fc640 1 -- 172.21.15.7:0/2071744202 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0c005050 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffaf67fc640 1 -- 172.21.15.7:0/2071744202 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0049b0 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb19853740 1 -- 172.21.15.7:0/2071744202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 msgr2=0x55ea0fd05810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.485+0000 7ffb19853740 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55ea0fcfc750 tx=0x7ffb0c02c7e0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- 172.21.15.7:0/2071744202 shutdown_connections 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fd00e50 0x55ea0fcf5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 --2- 172.21.15.7:0/2071744202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd05810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- 172.21.15.7:0/2071744202 >> 172.21.15.7:0/2071744202 conn(0x55ea0fcf6860 msgr2=0x55ea0fcf6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- 172.21.15.7:0/2071744202 shutdown_connections 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- 172.21.15.7:0/2071744202 wait complete. 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- start start 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 Processor -- start 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fd00e50 0x55ea0fd7a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101a8880 con 0x55ea0fcfbce0 2024-03-19T00:48:43.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101a9cf0 con 0x55ea0fd00e50 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101a8f50 con 0x55ea0fcfb2f0 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb19853740 10 client.0 _ll_register_callbacks cb 0x7ffb00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb149fd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd73270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38664/0 (socket says 172.21.15.7:38664) 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb07fff640 1 -- 172.21.15.7:0/3606543075 learned_addr learned my addr 172.21.15.7:0/3606543075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.486+0000 7ffb151fe640 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fd00e50 0x55ea0fd7a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb07fff640 1 -- 172.21.15.7:0/3606543075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfb2f0 msgr2=0x55ea0fd72d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb07fff640 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd72d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb07fff640 1 -- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fd00e50 msgr2=0x55ea0fd7a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb07fff640 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fd00e50 0x55ea0fd7a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb07fff640 1 -- 172.21.15.7:0/3606543075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0fccc880 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb07fff640 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd73270 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ffb000cc490 tx=0x7ffb00005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a7db24c9acb044 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffaeffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffaeffff640 1 -- 172.21.15.7:0/3606543075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00005560 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffaeffff640 1 -- 172.21.15.7:0/3606543075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ffb000a3640 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c005050 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffaeffff640 1 -- 172.21.15.7:0/3606543075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb0009ba30 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb19853740 10 client.17760 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea0fd7baa0 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffaeffff640 1 -- 172.21.15.7:0/3606543075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0c005050 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.487+0000 7ffaeffff640 1 -- 172.21.15.7:0/3606543075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffb0001f030 con 0x55ea0fcfbce0 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 10 client.17760 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 10 client.17760 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 20 client.17760 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:43.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd7c4d0 0x55ea0fd7e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 4 client.17760 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea0fc6c9f0 con 0x55ea0fd7c4d0 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb149fd640 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd7c4d0 0x55ea0fd7e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffaeffff640 10 client.17760 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb149fd640 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd7c4d0 0x55ea0fd7e8f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27a5de2d56a3ebf9 server_cookie=cdaf873946f2a4f2 in_seq=0 out_seq=0 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffaeffff640 1 -- 172.21.15.7:0/3606543075 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea0fc6c9f0 con 0x55ea0fd7c4d0 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffaeffff640 10 client.17760 handle_command_reply: tid=0 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 client.17760 shutdown 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 msgr2=0x55ea0fd73270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd73270 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ffb000cc490 tx=0x7ffb00005750 comp rx=0 tx=0).stop 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 shutdown_connections 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55ea0fd7c4d0 0x55ea0fd7e8f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea0fd00e50 0x55ea0fd7a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea0fcfbce0 0x55ea0fd73270 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffaeffff640 0 client.17760 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 --2- 172.21.15.7:0/3606543075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea0fcfb2f0 0x55ea0fd72d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.488+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 >> 172.21.15.7:0/3606543075 conn(0x55ea0fcf6860 msgr2=0x55ea0fcfd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 shutdown_connections 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb19853740 1 -- 172.21.15.7:0/3606543075 wait complete. 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb19853740 20 client.17760 trim_cache size 0 max 16384 2024-03-19T00:48:43.497 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:48:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ffaf80a2800 msgr2=0x7ffaf80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ffaf80a2800 0x7ffaf80a4cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffb08022000 tx=0x7ffb08023000 comp rx=0 tx=0).stop 2024-03-19T00:48:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb10104ca0 msgr2=0x7ffb10197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb10104ca0 0x7ffb10197fa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000e9c0 tx=0x7ffb0000ee80 comp rx=0 tx=0).stop 2024-03-19T00:48:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 shutdown_connections 2024-03-19T00:48:43.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ffaf80a2800 0x7ffaf80a4cc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb101075d0 0x7ffb1019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb10104ca0 0x7ffb10197fa0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 --2- 172.21.15.7:0/276903235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10102370 0x7ffb10197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 >> 172.21.15.7:0/276903235 conn(0x7ffb100fbc80 msgr2=0x7ffb10102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 shutdown_connections 2024-03-19T00:48:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.489+0000 7ffb16c61640 1 -- 172.21.15.7:0/276903235 wait complete. 2024-03-19T00:48:43.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 -- 172.21.15.7:0/2086117078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 msgr2=0x7f32dc107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2086117078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc107030 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009900 tx=0x7f32cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 -- 172.21.15.7:0/2086117078 shutdown_connections 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2086117078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32dc107570 0x7f32dc109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2086117078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc107030 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2086117078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32dc102310 0x7f32dc104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 -- 172.21.15.7:0/2086117078 >> 172.21.15.7:0/2086117078 conn(0x7f32dc0fbc80 msgr2=0x7f32dc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 -- 172.21.15.7:0/2086117078 shutdown_connections 2024-03-19T00:48:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.605+0000 7f32e34ac640 1 -- 172.21.15.7:0/2086117078 wait complete. 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 Processor -- start 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 -- start start 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32dc102310 0x7f32dc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32dc107570 0x7f32dc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc10d330 con 0x7f32dc104c40 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc10d1b0 con 0x7f32dc102310 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e34ac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc10d4b0 con 0x7f32dc107570 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38686/0 (socket says 172.21.15.7:38686) 2024-03-19T00:48:43.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e1248640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32dc102310 0x7f32dc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 -- 172.21.15.7:0/2749373398 learned_addr learned my addr 172.21.15.7:0/2749373398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e1a49640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32dc107570 0x7f32dc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 -- 172.21.15.7:0/2749373398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32dc107570 msgr2=0x7f32dc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32dc107570 0x7f32dc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 -- 172.21.15.7:0/2749373398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32dc102310 msgr2=0x7f32dc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32dc102310 0x7f32dc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 -- 172.21.15.7:0/2749373398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc19f6c0 con 0x7f32dc104c40 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.606+0000 7f32e0a47640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc197fe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000c00 tx=0x7f32cc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.607+0000 7f32d27fc640 1 -- 172.21.15.7:0/2749373398 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc043020 con 0x7f32dc104c40 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.607+0000 7f32d27fc640 1 -- 172.21.15.7:0/2749373398 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32cc004350 con 0x7f32dc104c40 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.607+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc19f950 con 0x7f32dc104c40 2024-03-19T00:48:43.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.607+0000 7f32d27fc640 1 -- 172.21.15.7:0/2749373398 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc003b10 con 0x7f32dc104c40 2024-03-19T00:48:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.607+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc19fea0 con 0x7f32dc104c40 2024-03-19T00:48:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32d27fc640 1 -- 172.21.15.7:0/2749373398 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f32cc046020 con 0x7f32dc104c40 2024-03-19T00:48:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 Processor -- start 2024-03-19T00:48:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 -- start start 2024-03-19T00:48:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 0x55e5999ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999cece0 0x55e5999d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e5999d62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5999324b0 con 0x55e5999ce2f0 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e5998dad70 con 0x55e5999d3e50 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e6172740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e59992e550 con 0x55e5999cece0 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e1a49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 0x55e5999ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e1a49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 0x55e5999ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38690/0 (socket says 172.21.15.7:38690) 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e1a49640 1 -- 172.21.15.7:0/2265495847 learned_addr learned my addr 172.21.15.7:0/2265495847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e1248640 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999cece0 0x55e5999d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32e0a47640 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e5999d62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.610+0000 7f32d27fc640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32c80a2800 0x7f32c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1248640 1 -- 172.21.15.7:0/2265495847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 msgr2=0x55e5999d62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1248640 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e5999d62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1248640 1 -- 172.21.15.7:0/2265495847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 msgr2=0x55e5999ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1248640 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 0x55e5999ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1248640 1 -- 172.21.15.7:0/2265495847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5998fd210 con 0x55e5999cece0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1a49640 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 0x55e5999ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1a49640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32c80a2800 0x7f32c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32d27fc640 1 -- 172.21.15.7:0/2749373398 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f32cc0f0b50 con 0x7f32dc104c40 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1248640 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999cece0 0x55e5999d37e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009c20 tx=0x7f32d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a712969fce75e12b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32c77fe640 1 -- 172.21.15.7:0/2265495847 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8004050 con 0x55e5999cece0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32e1a49640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32c80a2800 0x7f32c80a4cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55e5999cf540 tx=0x7f32d4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32c77fe640 1 -- 172.21.15.7:0/2265495847 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d80054d0 con 0x55e5999cece0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.611+0000 7f32c77fe640 1 -- 172.21.15.7:0/2265495847 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8004c30 con 0x55e5999cece0 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- 172.21.15.7:0/2265495847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999cece0 msgr2=0x55e5999d37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999cece0 0x55e5999d37e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009c20 tx=0x7f32d8011810 comp rx=0 tx=0).stop 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- 172.21.15.7:0/2265495847 shutdown_connections 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e5999d62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999cece0 0x55e5999d37e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- 172.21.15.7:0/2265495847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999ce2f0 0x55e5999ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- 172.21.15.7:0/2265495847 >> 172.21.15.7:0/2265495847 conn(0x55e5999c9860 msgr2=0x55e5999c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- 172.21.15.7:0/2265495847 shutdown_connections 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- 172.21.15.7:0/2265495847 wait complete. 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- start start 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 Processor -- start 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5999d6e00 con 0x55e5999d3e50 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e5998ee310 con 0x55e5999cece0 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e5999d6f80 con 0x55e5999ce2f0 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.612+0000 7f32e6172740 10 client.0 _ll_register_callbacks cb 0x55e5998e22d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1248640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999dfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38698/0 (socket says 172.21.15.7:38698) 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 -- 172.21.15.7:0/246332372 learned_addr learned my addr 172.21.15.7:0/246332372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e0a47640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 -- 172.21.15.7:0/246332372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 msgr2=0x55e5999d87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999d87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 -- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 msgr2=0x55e5999d8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999d8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 -- 172.21.15.7:0/246332372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5996463f0 con 0x55e5999d3e50 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1248640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999d87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e1a49640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999dfd50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f32d4010be0 tx=0x7f32d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565201ff6abbb17f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32c4ff9640 1 -- 172.21.15.7:0/246332372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d402e060 con 0x55e5999d3e50 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32c4ff9640 1 -- 172.21.15.7:0/246332372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f32d401b3d0 con 0x55e5999d3e50 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d80054d0 con 0x55e5999d3e50 2024-03-19T00:48:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32c4ff9640 1 -- 172.21.15.7:0/246332372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32d4035050 con 0x55e5999d3e50 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e6172740 10 client.17772 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.613+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5997107d0 con 0x55e5999d3e50 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32c4ff9640 1 -- 172.21.15.7:0/246332372 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d80054d0 con 0x55e5999d3e50 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32c4ff9640 1 -- 172.21.15.7:0/246332372 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f32d400f580 con 0x55e5999d3e50 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e6172740 10 client.17772 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e6172740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e6172740 20 client.17772 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e6172740 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e5999e4730 0x55e5999e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e6172740 4 client.17772 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5999903e0 con 0x55e5999e4730 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e1248640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e5999e4730 0x55e5999e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32e1248640 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e5999e4730 0x55e5999e6bf0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6a2739d4c3160c server_cookie=c798ad8152ee8783 in_seq=0 out_seq=0 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.614+0000 7f32c4ff9640 10 client.17772 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32c4ff9640 1 -- 172.21.15.7:0/246332372 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5999903e0 con 0x55e5999e4730 2024-03-19T00:48:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32c4ff9640 10 client.17772 handle_command_reply: tid=0 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 client.17772 shutdown 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 msgr2=0x55e5999dfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999dfd50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f32d4010be0 tx=0x7f32d4004ab0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 shutdown_connections 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e5999e4730 0x55e5999e6bf0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999dfd50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999d8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 --2- 172.21.15.7:0/246332372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999d87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32c4ff9640 0 client.17772 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 >> 172.21.15.7:0/246332372 conn(0x55e5999c9860 msgr2=0x55e5999c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 shutdown_connections 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 1 -- 172.21.15.7:0/246332372 wait complete. 2024-03-19T00:48:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.615+0000 7f32e6172740 20 client.17772 trim_cache size 0 max 16384 2024-03-19T00:48:43.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.628+0000 7f32e6172740 1 Processor -- start 2024-03-19T00:48:43.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 -- start start 2024-03-19T00:48:43.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1acb60 con 0x55e5999d3e50 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1aa430 con 0x55e5999cece0 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e6172740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1a8280 con 0x55e5999ce2f0 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1248640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e0a47640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38704/0 (socket says 172.21.15.7:38704) 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 -- 172.21.15.7:0/3771139264 learned_addr learned my addr 172.21.15.7:0/3771139264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 -- 172.21.15.7:0/3771139264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 msgr2=0x55e5999c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 -- 172.21.15.7:0/3771139264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 msgr2=0x55e5999c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 -- 172.21.15.7:0/3771139264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59999f880 con 0x55e5999d3e50 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e0a47640 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32e1a49640 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999eb4b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f32d4033040 tx=0x7f32d4009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb667f897f19bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32c6ffd640 1 -- 172.21.15.7:0/3771139264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d400fd70 con 0x55e5999d3e50 2024-03-19T00:48:43.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32c6ffd640 1 -- 172.21.15.7:0/3771139264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d4028400 con 0x55e5999d3e50 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.629+0000 7f32c6ffd640 1 -- 172.21.15.7:0/3771139264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4039640 con 0x55e5999d3e50 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 -- 172.21.15.7:0/3771139264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 msgr2=0x55e5999eb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999eb4b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f32d4033040 tx=0x7f32d4009a70 comp rx=0 tx=0).stop 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 -- 172.21.15.7:0/3771139264 shutdown_connections 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999d3e50 0x55e5999eb4b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999cece0 0x55e5999c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 --2- 172.21.15.7:0/3771139264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e5999c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 -- 172.21.15.7:0/3771139264 >> 172.21.15.7:0/3771139264 conn(0x55e5999c9860 msgr2=0x55e5999d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 -- 172.21.15.7:0/3771139264 shutdown_connections 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 -- 172.21.15.7:0/3771139264 wait complete. 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 -- start start 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.630+0000 7f32e6172740 1 Processor -- start 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e599a45ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 0x55e599a46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e599a4d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1a88f0 con 0x55e5999cece0 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1a9d60 con 0x55e5999d3e50 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1a8fc0 con 0x55e5999ce2f0 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 10 client.0 _ll_register_callbacks cb 0x7f32d4003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 0x55e599a46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 0x55e599a46020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38718/0 (socket says 172.21.15.7:38718) 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 -- 172.21.15.7:0/1180679219 learned_addr learned my addr 172.21.15.7:0/1180679219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e1248640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e599a45ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e1a49640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e599a4d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 -- 172.21.15.7:0/1180679219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 msgr2=0x55e599a45ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e599a45ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 -- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 msgr2=0x55e599a4d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e599a4d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 -- 172.21.15.7:0/1180679219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e599a4d650 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e1a49640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e599a4d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e0a47640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 0x55e599a46020 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006010 tx=0x7f32cc0b9560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3fb6539aba2b7c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32c4ff9640 1 -- 172.21.15.7:0/1180679219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0bcaf0 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32c4ff9640 1 -- 172.21.15.7:0/1180679219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f32cc0bb490 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.631+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d4028400 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32c4ff9640 1 -- 172.21.15.7:0/1180679219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32cc0d6020 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 10 client.17778 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e599a4ea20 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32c4ff9640 1 -- 172.21.15.7:0/1180679219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d4028400 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32c4ff9640 1 -- 172.21.15.7:0/1180679219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f32cc088250 con 0x55e5999cece0 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 10 client.17778 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 20 client.17778 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e599a4f420 0x55e599a51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 4 client.17778 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e59993f9f0 con 0x55e599a4f420 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.632+0000 7f32e1248640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e599a4f420 0x55e599a51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32c4ff9640 10 client.17778 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e1248640 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e599a4f420 0x55e599a51840 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6403646c255ffa0d server_cookie=269c0a8a028054cb in_seq=0 out_seq=0 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32c4ff9640 1 -- 172.21.15.7:0/1180679219 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e59993f9f0 con 0x55e599a4f420 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32c4ff9640 10 client.17778 handle_command_reply: tid=0 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 client.17778 shutdown 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 msgr2=0x55e599a46020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 0x55e599a46020 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006010 tx=0x7f32cc0b9560 comp rx=0 tx=0).stop 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 shutdown_connections 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5999d3e50 0x55e599a4d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55e599a4f420 0x55e599a51840 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5999cece0 0x55e599a46020 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32c4ff9640 0 client.17778 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 --2- 172.21.15.7:0/1180679219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e5999ce2f0 0x55e599a45ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 >> 172.21.15.7:0/1180679219 conn(0x55e5999c9860 msgr2=0x55e5999ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 shutdown_connections 2024-03-19T00:48:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 1 -- 172.21.15.7:0/1180679219 wait complete. 2024-03-19T00:48:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e6172740 20 client.17778 trim_cache size 0 max 16384 2024-03-19T00:48:43.641 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.633+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32c80a2800 msgr2=0x7f32c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32c80a2800 0x7f32c80a4cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55e5999cf540 tx=0x7f32d4004a40 comp rx=0 tx=0).stop 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 msgr2=0x7f32dc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc197fe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000c00 tx=0x7f32cc005dc0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 shutdown_connections 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f32c80a2800 0x7f32c80a4cc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32dc107570 0x7f32dc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32dc104c40 0x7f32dc197fe0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 --2- 172.21.15.7:0/2749373398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32dc102310 0x7f32dc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 >> 172.21.15.7:0/2749373398 conn(0x7f32dc0fbc80 msgr2=0x7f32dc102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 shutdown_connections 2024-03-19T00:48:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:43.634+0000 7f32e34ac640 1 -- 172.21.15.7:0/2749373398 wait complete. 2024-03-19T00:48:44.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:43 smithi007 ceph-mon[62854]: pgmap v674: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:44.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:43 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/4241409292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:48:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:43 smithi204 ceph-mon[66572]: pgmap v674: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:44.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:43 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/4241409292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:48:44.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:43 smithi044 ceph-mon[66366]: pgmap v674: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:48:44.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:43 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/4241409292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:48:44.646 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T00:48:44.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:48:44.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T00:48:44.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 -- 172.21.15.7:0/3266364782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4102320 msgr2=0x7fb1a4104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3266364782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4102320 0x7fb1a4104710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ca0 tx=0x7fb19402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 -- 172.21.15.7:0/3266364782 shutdown_connections 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3266364782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4107580 0x7fb1a4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3266364782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a4104c50 0x7fb1a4107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3266364782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4102320 0x7fb1a4104710 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 -- 172.21.15.7:0/3266364782 >> 172.21.15.7:0/3266364782 conn(0x7fb1a40fbcb0 msgr2=0x7fb1a40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 -- 172.21.15.7:0/3266364782 shutdown_connections 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.764+0000 7fb1ab427640 1 -- 172.21.15.7:0/3266364782 wait complete. 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 Processor -- start 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 -- start start 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4102320 0x7fb1a4101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 0x7fb1a41002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a4107580 0x7fb1a4100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410d390 con 0x7fb1a4104c50 2024-03-19T00:48:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410d210 con 0x7fb1a4102320 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1ab427640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410d510 con 0x7fb1a4107580 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1a89c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 0x7fb1a41002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1a89c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 0x7fb1a41002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37220/0 (socket says 172.21.15.7:37220) 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1a89c2640 1 -- 172.21.15.7:0/3062532335 learned_addr learned my addr 172.21.15.7:0/3062532335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4102320 0x7fb1a4101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.765+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a4107580 0x7fb1a4100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a89c2640 1 -- 172.21.15.7:0/3062532335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a4107580 msgr2=0x7fb1a4100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a4107580 0x7fb1a4100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a89c2640 1 -- 172.21.15.7:0/3062532335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4102320 msgr2=0x7fb1a4101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4102320 0x7fb1a4101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a89c2640 1 -- 172.21.15.7:0/3062532335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41010a0 con 0x7fb1a4104c50 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4102320 0x7fb1a4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 0x7fb1a41002d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb19800e9f0 tx=0x7fb19800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a27fc640 1 -- 172.21.15.7:0/3062532335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19800ccc0 con 0x7fb1a4104c50 2024-03-19T00:48:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a27fc640 1 -- 172.21.15.7:0/3062532335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1980054d0 con 0x7fb1a4104c50 2024-03-19T00:48:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41a1c20 con 0x7fb1a4104c50 2024-03-19T00:48:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1a27fc640 1 -- 172.21.15.7:0/3062532335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198010430 con 0x7fb1a4104c50 2024-03-19T00:48:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.766+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a2170 con 0x7fb1a4104c50 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a27fc640 1 -- 172.21.15.7:0/3062532335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fb198005020 con 0x7fb1a4104c50 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 Processor -- start 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 -- start start 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021ba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ba0211a160 con 0x55ba021baf00 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba020c2e10 con 0x55ba021b53a0 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ba02116200 con 0x55ba021b5d90 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a27fc640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb18c0a2880 0x7fb18c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a89c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a91c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a91c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021ba890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:60258/0 (socket says 172.21.15.7:60258) 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a91c3640 1 -- 172.21.15.7:0/2536010662 learned_addr learned my addr 172.21.15.7:0/2536010662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.769+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb18c0a2880 0x7fb18c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a27fc640 1 -- 172.21.15.7:0/3062532335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb1980cf150 con 0x7fb1a4104c50 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a91c3640 1 -- 172.21.15.7:0/2536010662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 msgr2=0x55ba021b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a91c3640 1 -- 172.21.15.7:0/2536010662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 msgr2=0x55ba021bd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a91c3640 1 -- 172.21.15.7:0/2536010662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba01ef67e0 con 0x55ba021b5d90 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021ba890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ca0 tx=0x7fb194005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab86226338993e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb18c0a2880 0x7fb18c0a4d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55ba021b65f0 tx=0x7fb19c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb18b7fe640 1 -- 172.21.15.7:0/2536010662 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194043070 con 0x55ba021b5d90 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb18b7fe640 1 -- 172.21.15.7:0/2536010662 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb194003e20 con 0x55ba021b5d90 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.770+0000 7fb18b7fe640 1 -- 172.21.15.7:0/2536010662 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194039040 con 0x55ba021b5d90 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 -- 172.21.15.7:0/2536010662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 msgr2=0x55ba021ba890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021ba890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ca0 tx=0x7fb194005ef0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 -- 172.21.15.7:0/2536010662 shutdown_connections 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021ba890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/2536010662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 -- 172.21.15.7:0/2536010662 >> 172.21.15.7:0/2536010662 conn(0x55ba021b0910 msgr2=0x55ba021b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 -- 172.21.15.7:0/2536010662 shutdown_connections 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 -- 172.21.15.7:0/2536010662 wait complete. 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 -- start start 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.771+0000 7fb1ae00e740 1 Processor -- start 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ba021bde50 con 0x55ba021baf00 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba020d61a0 con 0x55ba021b53a0 2024-03-19T00:48:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ba021bdfd0 con 0x55ba021b5d90 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1ae00e740 10 client.0 _ll_register_callbacks cb 0x55ba01e2dc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a99c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021bf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52544/0 (socket says 172.21.15.7:52544) 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 -- 172.21.15.7:0/35368363 learned_addr learned my addr 172.21.15.7:0/35368363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 -- 172.21.15.7:0/35368363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 msgr2=0x55ba021bfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021bfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 -- 172.21.15.7:0/35368363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 msgr2=0x55ba021c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.772+0000 7fb1a91c3640 1 -- 172.21.15.7:0/35368363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba01f56f70 con 0x55ba021b53a0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021bf7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb19402c630 tx=0x7fb1940329f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660fa448019805bf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb188ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb188ff9640 1 -- 172.21.15.7:0/35368363 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19404c030 con 0x55ba021b53a0 2024-03-19T00:48:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb194003e20 con 0x55ba021b53a0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb188ff9640 1 -- 172.21.15.7:0/35368363 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb194003b80 con 0x55ba021b53a0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb188ff9640 1 -- 172.21.15.7:0/35368363 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb194059050 con 0x55ba021b53a0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 10 client.27635 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba01dee170 con 0x55ba021b53a0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb188ff9640 1 -- 172.21.15.7:0/35368363 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb194003e20 con 0x55ba021b53a0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb188ff9640 1 -- 172.21.15.7:0/35368363 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb194048a00 con 0x55ba021b53a0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 10 client.27635 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 10 client.27635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 20 client.27635 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba021cb210 0x55ba021cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 4 client.27635 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.773+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba021276a0 con 0x55ba021cb210 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.774+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba021cb210 0x55ba021cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.774+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba021cb210 0x55ba021cd600 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef81a8a694cbbb70 server_cookie=ebf39c3edd632b60 in_seq=0 out_seq=0 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.774+0000 7fb188ff9640 10 client.27635 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb188ff9640 1 -- 172.21.15.7:0/35368363 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fb1980aa9c0 con 0x55ba021cb210 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb188ff9640 10 client.27635 handle_command_reply: tid=0 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 client.27635 shutdown 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 msgr2=0x55ba021bf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021bf7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb19402c630 tx=0x7fb1940329f0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 shutdown_connections 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba021cb210 0x55ba021cd600 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/35368363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b53a0 0x55ba021bf7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb188ff9640 0 client.27635 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 >> 172.21.15.7:0/35368363 conn(0x55ba021b0910 msgr2=0x55ba021afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 shutdown_connections 2024-03-19T00:48:44.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 1 -- 172.21.15.7:0/35368363 wait complete. 2024-03-19T00:48:44.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.775+0000 7fb1ae00e740 20 client.27635 trim_cache size 0 max 16384 2024-03-19T00:48:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 Processor -- start 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 -- start start 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b53a0 0x55ba021bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b5d90 0x55ba021c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41aed50 con 0x55ba021baf00 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41ac620 con 0x55ba021b5d90 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41aa470 con 0x55ba021b53a0 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37240/0 (socket says 172.21.15.7:37240) 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 -- 172.21.15.7:0/1688982930 learned_addr learned my addr 172.21.15.7:0/1688982930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b5d90 0x55ba021c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b53a0 0x55ba021bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 -- 172.21.15.7:0/1688982930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b53a0 msgr2=0x55ba021bedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b53a0 0x55ba021bedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 -- 172.21.15.7:0/1688982930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b5d90 msgr2=0x55ba021c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b5d90 0x55ba021c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 -- 172.21.15.7:0/1688982930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba021bac20 con 0x55ba021baf00 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.793+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bf300 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb19c004280 tx=0x7fb19c009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabd59bdbfabac32 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb18affd640 1 -- 172.21.15.7:0/1688982930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0203f0 con 0x55ba021baf00 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb18affd640 1 -- 172.21.15.7:0/1688982930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb19c020a40 con 0x55ba021baf00 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb18affd640 1 -- 172.21.15.7:0/1688982930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00fa60 con 0x55ba021baf00 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1688982930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 msgr2=0x55ba021bf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bf300 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb19c004280 tx=0x7fb19c009760 comp rx=0 tx=0).stop 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1688982930 shutdown_connections 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021baf00 0x55ba021bf300 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021b5d90 0x55ba021c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1688982930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b53a0 0x55ba021bedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1688982930 >> 172.21.15.7:0/1688982930 conn(0x55ba021b0910 msgr2=0x55ba021b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1688982930 shutdown_connections 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1688982930 wait complete. 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.794+0000 7fb1ae00e740 1 -- start start 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 Processor -- start 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 0x55ba021c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021baf00 0x55ba021c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41aaae0 con 0x55ba021b53a0 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41abf50 con 0x55ba021baf00 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41ab1b0 con 0x55ba021b5d90 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1ae00e740 10 client.0 _ll_register_callbacks cb 0x7fb198097310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 0x55ba021c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 0x55ba021c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37256/0 (socket says 172.21.15.7:37256) 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 -- 172.21.15.7:0/1419277597 learned_addr learned my addr 172.21.15.7:0/1419277597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a99c4640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021baf00 0x55ba021c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 -- 172.21.15.7:0/1419277597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 msgr2=0x55ba021c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 -- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021baf00 msgr2=0x55ba021c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021baf00 0x55ba021c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.795+0000 7fb1a91c3640 1 -- 172.21.15.7:0/1419277597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba021c5970 con 0x55ba021b53a0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1a91c3640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 0x55ba021c68e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb194048d70 tx=0x7fb194032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae0c9756c151f87 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb188ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb188ff9640 1 -- 172.21.15.7:0/1419277597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19404c070 con 0x55ba021b53a0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb188ff9640 1 -- 172.21.15.7:0/1419277597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb194003d90 con 0x55ba021b53a0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19c020a40 con 0x55ba021b53a0 2024-03-19T00:48:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb188ff9640 1 -- 172.21.15.7:0/1419277597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb194039070 con 0x55ba021b53a0 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 10 client.17796 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba022321b0 con 0x55ba021b53a0 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb188ff9640 1 -- 172.21.15.7:0/1419277597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb19c020a40 con 0x55ba021b53a0 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb188ff9640 1 -- 172.21.15.7:0/1419277597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb1940485e0 con 0x55ba021b53a0 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 10 client.17796 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 20 client.17796 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba02232b10 0x55ba02234f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 4 client.17796 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba021219f0 con 0x55ba02232b10 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.796+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba02232b10 0x55ba02234f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.797+0000 7fb188ff9640 10 client.17796 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.797+0000 7fb1a89c2640 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba02232b10 0x55ba02234f30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c4307a83888331 server_cookie=85be462986e9d93a in_seq=0 out_seq=0 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb188ff9640 1 -- 172.21.15.7:0/1419277597 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba021219f0 con 0x55ba02232b10 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb188ff9640 10 client.17796 handle_command_reply: tid=0 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 client.17796 shutdown 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 msgr2=0x55ba021c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 0x55ba021c68e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb194048d70 tx=0x7fb194032f60 comp rx=0 tx=0).stop 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 shutdown_connections 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba021baf00 0x55ba021c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ba021b5d90 0x55ba021c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55ba02232b10 0x55ba02234f30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 --2- 172.21.15.7:0/1419277597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ba021b53a0 0x55ba021c68e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 >> 172.21.15.7:0/1419277597 conn(0x55ba021b0910 msgr2=0x55ba021af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb188ff9640 0 client.17796 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 shutdown_connections 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 1 -- 172.21.15.7:0/1419277597 wait complete. 2024-03-19T00:48:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.798+0000 7fb1ae00e740 20 client.17796 trim_cache size 0 max 16384 2024-03-19T00:48:44.805 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:48:44.805 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-19T00:48:44.805 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "b9ef34b7-bc51-4d21-9ad6-9f01d26c4a0a", 2024-03-19T00:48:44.805 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-19T00:48:44.805 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb18c0a2880 msgr2=0x7fb18c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb18c0a2880 0x7fb18c0a4d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55ba021b65f0 tx=0x7fb19c005f90 comp rx=0 tx=0).stop 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 msgr2=0x7fb1a41002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 0x7fb1a41002d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb19800e9f0 tx=0x7fb19800edf0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 shutdown_connections 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fb18c0a2880 0x7fb18c0a4d40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a4107580 0x7fb1a4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a4104c50 0x7fb1a41002d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 --2- 172.21.15.7:0/3062532335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a4102320 0x7fb1a4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 >> 172.21.15.7:0/3062532335 conn(0x7fb1a40fbcb0 msgr2=0x7fb1a4102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 shutdown_connections 2024-03-19T00:48:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.799+0000 7fb1ab427640 1 -- 172.21.15.7:0/3062532335 wait complete. 2024-03-19T00:48:44.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:48:44.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:48:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3738572707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 msgr2=0x7fdf60109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3738572707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf60109c60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009ca0 tx=0x7fdf5802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3738572707 shutdown_connections 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3738572707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf60109c60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3738572707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf60102a10 0x7fdf60102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3738572707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60101820 0x7fdf60101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3738572707 >> 172.21.15.7:0/3738572707 conn(0x7fdf600fcd70 msgr2=0x7fdf600ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3738572707 shutdown_connections 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3738572707 wait complete. 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 Processor -- start 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.915+0000 7fdf67ef8640 1 -- start start 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf60101820 0x7fdf60197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60102a10 0x7fdf60197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf6019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6010d350 con 0x7fdf601033d0 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6010d1d0 con 0x7fdf60102a10 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6010d4d0 con 0x7fdf60101820 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf6019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf6019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37270/0 (socket says 172.21.15.7:37270) 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf65493640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60102a10 0x7fdf60197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 -- 172.21.15.7:0/3244061843 learned_addr learned my addr 172.21.15.7:0/3244061843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf65c94640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf60101820 0x7fdf60197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 -- 172.21.15.7:0/3244061843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf60101820 msgr2=0x7fdf60197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf60101820 0x7fdf60197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 -- 172.21.15.7:0/3244061843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60102a10 msgr2=0x7fdf60197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60102a10 0x7fdf60197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 -- 172.21.15.7:0/3244061843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6019f640 con 0x7fdf601033d0 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf65493640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60102a10 0x7fdf60197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf66495640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf6019efd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdf5802c700 tx=0x7fdf580328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf56ffd640 1 -- 172.21.15.7:0/3244061843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58041020 con 0x7fdf601033d0 2024-03-19T00:48:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf56ffd640 1 -- 172.21.15.7:0/3244061843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf58003e20 con 0x7fdf601033d0 2024-03-19T00:48:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6019f8d0 con 0x7fdf601033d0 2024-03-19T00:48:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.916+0000 7fdf56ffd640 1 -- 172.21.15.7:0/3244061843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58039040 con 0x7fdf601033d0 2024-03-19T00:48:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.917+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6019fe10 con 0x7fdf601033d0 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf56ffd640 1 -- 172.21.15.7:0/3244061843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fdf58044020 con 0x7fdf601033d0 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 Processor -- start 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 -- start start 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506cfa50 0x55b3506cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf56ffd640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdf4c0a2880 0x7fdf4c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b350634500 con 0x55b3506d0440 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b3505dcbe0 con 0x55b3506cfa50 2024-03-19T00:48:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3506305a0 con 0x55b3506d55b0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf65c94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf65c94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37284/0 (socket says 172.21.15.7:37284) 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf65c94640 1 -- 172.21.15.7:0/3322968269 learned_addr learned my addr 172.21.15.7:0/3322968269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf65493640 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506cfa50 0x55b3506cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.919+0000 7fdf66495640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdf4c0a2880 0x7fdf4c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf56ffd640 1 -- 172.21.15.7:0/3244061843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdf580f0ee0 con 0x7fdf601033d0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 -- 172.21.15.7:0/3322968269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 msgr2=0x55b3506d7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506d7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 -- 172.21.15.7:0/3322968269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 msgr2=0x55b3506d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 -- 172.21.15.7:0/3322968269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3505fe000 con 0x55b3506cfa50 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65c94640 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf66495640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdf4c0a2880 0x7fdf4c0a4d40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf580c5790 tx=0x7fdf580882e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf65493640 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506cfa50 0x55b3506cfe70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf60198f30 tx=0x7fdf50005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f999445b945104d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf4bfff640 1 -- 172.21.15.7:0/3322968269 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50011050 con 0x55b3506cfa50 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf4bfff640 1 -- 172.21.15.7:0/3322968269 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5000fd30 con 0x55b3506cfa50 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.920+0000 7fdf4bfff640 1 -- 172.21.15.7:0/3322968269 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500047a0 con 0x55b3506cfa50 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3322968269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506cfa50 msgr2=0x55b3506cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506cfa50 0x55b3506cfe70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf60198f30 tx=0x7fdf50005850 comp rx=0 tx=0).stop 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3322968269 shutdown_connections 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3322968269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506cfa50 0x55b3506cfe70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3322968269 >> 172.21.15.7:0/3322968269 conn(0x55b3506cafc0 msgr2=0x55b3506cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3322968269 shutdown_connections 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3322968269 wait complete. 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 -- start start 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.921+0000 7fdf6abc3740 1 Processor -- start 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 0x55b3506d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506e1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b3506d8560 con 0x55b3506cfa50 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b3505f0160 con 0x55b3506d0440 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b3506d86e0 con 0x55b3506d55b0 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf6abc3740 10 client.0 _ll_register_callbacks cb 0x55b35062ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65493640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 0x55b3506d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65493640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 0x55b3506d9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37294/0 (socket says 172.21.15.7:37294) 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65493640 1 -- 172.21.15.7:0/2593165998 learned_addr learned my addr 172.21.15.7:0/2593165998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65493640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 -- 172.21.15.7:0/2593165998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 msgr2=0x55b3506e1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506e1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 -- 172.21.15.7:0/2593165998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 msgr2=0x55b3506d9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 0x55b3506d9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 -- 172.21.15.7:0/2593165998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3504127d0 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65493640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 0x55b3506d9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf65c94640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506da420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55b3506d0840 tx=0x7fdf5c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b2c70dcf8b0d6c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf497fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.922+0000 7fdf497fa640 1 -- 172.21.15.7:0/2593165998 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c004030 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5000fd30 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf497fa640 1 -- 172.21.15.7:0/2593165998 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdf5c0041d0 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf497fa640 1 -- 172.21.15.7:0/2593165998 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c025050 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 10 client.27653 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3503483f0 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf497fa640 1 -- 172.21.15.7:0/2593165998 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5000fd30 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf497fa640 1 -- 172.21.15.7:0/2593165998 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdf5c0038b0 con 0x55b3506d0440 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 10 client.27653 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 10 client.27653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 20 client.27653 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3506e5c80 0x55b3506e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 4 client.27653 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b350641a40 con 0x55b3506e5c80 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.923+0000 7fdf65493640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3506e5c80 0x55b3506e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf497fa640 10 client.27653 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf65493640 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3506e5c80 0x55b3506e8100 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed229b81419ea83 server_cookie=64a9576740aea845 in_seq=0 out_seq=0 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf497fa640 1 -- 172.21.15.7:0/2593165998 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b350641a40 con 0x55b3506e5c80 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf497fa640 10 client.27653 handle_command_reply: tid=0 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf6abc3740 1 client.27653 shutdown 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 msgr2=0x55b3506da420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.924+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506da420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55b3506d0840 tx=0x7fdf5c012b60 comp rx=0 tx=0).stop 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 shutdown_connections 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506d55b0 0x55b3506e1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506da420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b3506e5c80 0x55b3506e8100 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/2593165998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506cfa50 0x55b3506d9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf497fa640 0 client.27653 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 >> 172.21.15.7:0/2593165998 conn(0x55b3506cafc0 msgr2=0x55b3506ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 shutdown_connections 2024-03-19T00:48:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 1 -- 172.21.15.7:0/2593165998 wait complete. 2024-03-19T00:48:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.925+0000 7fdf6abc3740 20 client.27653 trim_cache size 0 max 16384 2024-03-19T00:48:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 Processor -- start 2024-03-19T00:48:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 -- start start 2024-03-19T00:48:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d55b0 0x55b3506e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601acb60 con 0x55b3506d0440 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601aa430 con 0x55b3506d55b0 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601a8280 con 0x55b3506cfa50 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf65c94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf65c94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37304/0 (socket says 172.21.15.7:37304) 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf65c94640 1 -- 172.21.15.7:0/193091115 learned_addr learned my addr 172.21.15.7:0/193091115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.942+0000 7fdf65493640 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65493640 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d55b0 0x55b3506e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65c94640 1 -- 172.21.15.7:0/193091115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 msgr2=0x55b3506d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65c94640 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65c94640 1 -- 172.21.15.7:0/193091115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d55b0 msgr2=0x55b3506e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65c94640 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d55b0 0x55b3506e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65c94640 1 -- 172.21.15.7:0/193091115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3503080f0 con 0x55b3506d0440 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65493640 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d55b0 0x55b3506e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf65c94640 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d1da0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c003ca0 tx=0x7fdf5c0123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a925afbd32652 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf4b7fe640 1 -- 172.21.15.7:0/193091115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c02e070 con 0x55b3506d0440 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf4b7fe640 1 -- 172.21.15.7:0/193091115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5c003aa0 con 0x55b3506d0440 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf4b7fe640 1 -- 172.21.15.7:0/193091115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c01c780 con 0x55b3506d0440 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf6abc3740 1 -- 172.21.15.7:0/193091115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 msgr2=0x55b3506d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d1da0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c003ca0 tx=0x7fdf5c0123a0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.943+0000 7fdf6abc3740 1 -- 172.21.15.7:0/193091115 shutdown_connections 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d55b0 0x55b3506e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d0440 0x55b3506d1da0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/193091115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- 172.21.15.7:0/193091115 >> 172.21.15.7:0/193091115 conn(0x55b3506cafc0 msgr2=0x55b3506cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- 172.21.15.7:0/193091115 shutdown_connections 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- 172.21.15.7:0/193091115 wait complete. 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- start start 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 Processor -- start 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 0x55b3506e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601a88f0 con 0x55b3506d55b0 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601a9d60 con 0x55b3506d0440 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601a8fc0 con 0x55b3506cfa50 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf6abc3740 10 client.0 _ll_register_callbacks cb 0x7fdf5c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf65493640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 0x55b3506e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf65493640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 0x55b3506e0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37312/0 (socket says 172.21.15.7:37312) 2024-03-19T00:48:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf65493640 1 -- 172.21.15.7:0/3955344895 learned_addr learned my addr 172.21.15.7:0/3955344895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf65c94640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.944+0000 7fdf65493640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65c94640 1 -- 172.21.15.7:0/3955344895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 msgr2=0x55b3506deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65c94640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506deb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65c94640 1 -- 172.21.15.7:0/3955344895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 msgr2=0x55b3506e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65c94640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 0x55b3506e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65c94640 1 -- 172.21.15.7:0/3955344895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3506e0650 con 0x55b3506d0440 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65493640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 0x55b3506e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf65c94640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506df050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55b3506d0bf0 tx=0x7fdf5c0053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ae5a29b08c05af server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf497fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf497fa640 1 -- 172.21.15.7:0/3955344895 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c02e070 con 0x55b3506d0440 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5c003aa0 con 0x55b3506d0440 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf497fa640 1 -- 172.21.15.7:0/3955344895 <== mon.2 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdf5c018040 con 0x55b3506d0440 2024-03-19T00:48:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf497fa640 1 -- 172.21.15.7:0/3955344895 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c039050 con 0x55b3506d0440 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf6abc3740 10 client.27656 fetch_fsmap learned FSMap version 27 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b35074c000 con 0x55b3506d0440 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.945+0000 7fdf497fa640 1 -- 172.21.15.7:0/3955344895 <== mon.2 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5c003aa0 con 0x55b3506d0440 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf497fa640 1 -- 172.21.15.7:0/3955344895 <== mon.2 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdf5c00f790 con 0x55b3506d0440 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 10 client.27656 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 10 client.27656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 20 client.27656 populate_metadata read hostname 'smithi007' 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b35074c750 0x55b35074eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 4 client.27656 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b35063bd90 con 0x55b35074c750 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf65493640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b35074c750 0x55b35074eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf497fa640 10 client.27656 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf65493640 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b35074c750 0x55b35074eb70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa602d19f6e41378 server_cookie=72bfee02e24813d1 in_seq=0 out_seq=0 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf497fa640 1 -- 172.21.15.7:0/3955344895 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b35063bd90 con 0x55b35074c750 2024-03-19T00:48:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf497fa640 10 client.27656 handle_command_reply: tid=0 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 1 client.27656 shutdown 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 msgr2=0x55b3506df050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.946+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506df050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55b3506d0bf0 tx=0x7fdf5c0053b0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 shutdown_connections 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55b35074c750 0x55b35074eb70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3506d55b0 0x55b3506e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3506d0440 0x55b3506df050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf497fa640 0 client.27656 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 --2- 172.21.15.7:0/3955344895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3506cfa50 0x55b3506deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 >> 172.21.15.7:0/3955344895 conn(0x55b3506cafc0 msgr2=0x55b3506d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 shutdown_connections 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 1 -- 172.21.15.7:0/3955344895 wait complete. 2024-03-19T00:48:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf6abc3740 20 client.27656 trim_cache size 0 max 16384 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "b9ef34b7-bc51-4d21-9ad6-9f01d26c4a0a": { 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "b9ef34b7-bc51-4d21-9ad6-9f01d26c4a0a", 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdf4c0a2880 msgr2=0x7fdf4c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdf4c0a2880 0x7fdf4c0a4d40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf580c5790 tx=0x7fdf580882e0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 msgr2=0x7fdf6019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf6019efd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdf5802c700 tx=0x7fdf580328a0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 shutdown_connections 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fdf4c0a2880 0x7fdf4c0a4d40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf601033d0 0x7fdf6019efd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf60102a10 0x7fdf60197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 --2- 172.21.15.7:0/3244061843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf60101820 0x7fdf60197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:48:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 >> 172.21.15.7:0/3244061843 conn(0x7fdf600fcd70 msgr2=0x7fdf60105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:48:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 shutdown_connections 2024-03-19T00:48:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:48:44.947+0000 7fdf67ef8640 1 -- 172.21.15.7:0/3244061843 wait complete. 2024-03-19T00:48:44.958 INFO:tasks.cephfs.filesystem:scrub status for tag:b9ef34b7-bc51-4d21-9ad6-9f01d26c4a0a - {'path': '/', 'tag': 'b9ef34b7-bc51-4d21-9ad6-9f01d26c4a0a', 'options': 'recursive,force'} 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: pgmap v675: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: scrub queued for path: / 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: scrub summary: active paths [/] 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:45 smithi007 ceph-mon[62854]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: pgmap v675: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: scrub queued for path: / 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: scrub summary: active paths [/] 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:45 smithi204 ceph-mon[66572]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: pgmap v675: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: scrub queued for path: / 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: scrub summary: active paths [/] 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:45 smithi044 ceph-mon[66366]: scrub summary: idle+waiting paths [/] 2024-03-19T00:48:48.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:47 smithi007 ceph-mon[62854]: pgmap v676: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:48.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:47 smithi204 ceph-mon[66572]: pgmap v676: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:48.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:47 smithi044 ceph-mon[66366]: pgmap v676: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:50.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:49 smithi007 ceph-mon[62854]: pgmap v677: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:50.199 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:49 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:48:50.230 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:49 smithi204 ceph-mon[66572]: pgmap v677: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:50.230 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:49 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:48:50.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:49 smithi044 ceph-mon[66366]: pgmap v677: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:48:50.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:49 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: pgmap v678: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: scrub summary: idle 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:48:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: pgmap v678: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: scrub summary: idle 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:48:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: pgmap v678: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: scrub summary: idle 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:48:52.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:48:52.357 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:48:54.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:53 smithi007 ceph-mon[62854]: pgmap v679: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:48:54.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:53 smithi204 ceph-mon[66572]: pgmap v679: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:48:54.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:53 smithi044 ceph-mon[66366]: pgmap v679: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:48:56.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:55 smithi007 ceph-mon[62854]: pgmap v680: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:48:56.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:55 smithi204 ceph-mon[66572]: pgmap v680: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:48:56.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:55 smithi044 ceph-mon[66366]: pgmap v680: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:48:58.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:57 smithi007 ceph-mon[62854]: pgmap v681: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:48:58.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:57 smithi204 ceph-mon[66572]: pgmap v681: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:48:58.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:57 smithi044 ceph-mon[66366]: pgmap v681: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:49:00.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:48:59 smithi007 ceph-mon[62854]: pgmap v682: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:49:00.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:48:59 smithi204 ceph-mon[66572]: pgmap v682: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:49:00.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:48:59 smithi044 ceph-mon[66366]: pgmap v682: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-03-19T00:49:02.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:01 smithi007 ceph-mon[62854]: pgmap v683: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:49:02.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:01 smithi204 ceph-mon[66572]: pgmap v683: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:49:02.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:01 smithi044 ceph-mon[66366]: pgmap v683: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-03-19T00:49:04.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:03 smithi007 ceph-mon[62854]: pgmap v684: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:04.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:03 smithi204 ceph-mon[66572]: pgmap v684: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:04.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:03 smithi044 ceph-mon[66366]: pgmap v684: 129 pgs: 129 active+clean; 3.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:06.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:05 smithi007 ceph-mon[62854]: pgmap v685: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:49:06.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:05 smithi204 ceph-mon[66572]: pgmap v685: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:49:06.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:05 smithi044 ceph-mon[66366]: pgmap v685: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-03-19T00:49:08.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:07 smithi007 ceph-mon[62854]: pgmap v686: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:49:08.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:07 smithi204 ceph-mon[66572]: pgmap v686: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:49:08.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:07 smithi044 ceph-mon[66366]: pgmap v686: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:49:10.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:09 smithi007 ceph-mon[62854]: pgmap v687: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:10.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:09 smithi204 ceph-mon[66572]: pgmap v687: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:10.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:09 smithi044 ceph-mon[66366]: pgmap v687: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:12.158 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:11 smithi044 ceph-mon[66366]: pgmap v688: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:49:12.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:11 smithi007 ceph-mon[62854]: pgmap v688: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:49:12.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:11 smithi204 ceph-mon[66572]: pgmap v688: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-03-19T00:49:14.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:13 smithi007 ceph-mon[62854]: pgmap v689: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:14.266 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:13 smithi204 ceph-mon[66572]: pgmap v689: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:14.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:13 smithi044 ceph-mon[66366]: pgmap v689: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-03-19T00:49:14.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T00:49:14.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 -- 172.21.15.7:0/3810269532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58107560 msgr2=0x7f0a58109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 --2- 172.21.15.7:0/3810269532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58107560 0x7f0a58109970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009ca0 tx=0x7f0a4802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 -- 172.21.15.7:0/3810269532 shutdown_connections 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 --2- 172.21.15.7:0/3810269532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58107560 0x7f0a58109970 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 --2- 172.21.15.7:0/3810269532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58104c30 0x7f0a58107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 --2- 172.21.15.7:0/3810269532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a58102300 0x7f0a581046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 -- 172.21.15.7:0/3810269532 >> 172.21.15.7:0/3810269532 conn(0x7f0a580fbcb0 msgr2=0x7f0a580fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 -- 172.21.15.7:0/3810269532 shutdown_connections 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.080+0000 7f0a5f668640 1 -- 172.21.15.7:0/3810269532 wait complete. 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 Processor -- start 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 -- start start 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a58102300 0x7f0a58197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58104c30 0x7f0a58197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 0x7f0a5819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5810d390 con 0x7f0a58107560 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5810d210 con 0x7f0a58102300 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5f668640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5810d510 con 0x7f0a58104c30 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 0x7f0a5819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 0x7f0a5819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49652/0 (socket says 172.21.15.7:49652) 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5d404640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a58102300 0x7f0a58197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 -- 172.21.15.7:0/2383626326 learned_addr learned my addr 172.21.15.7:0/2383626326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58104c30 0x7f0a58197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 -- 172.21.15.7:0/2383626326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58104c30 msgr2=0x7f0a58197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58104c30 0x7f0a58197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 -- 172.21.15.7:0/2383626326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a58102300 msgr2=0x7f0a58197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a58102300 0x7f0a58197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 -- 172.21.15.7:0/2383626326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5819f690 con 0x7f0a58107560 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.081+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 0x7f0a5819f020 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0a4802c700 tx=0x7f0a480328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.082+0000 7f0a567fc640 1 -- 172.21.15.7:0/2383626326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48041020 con 0x7f0a58107560 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.082+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5819f920 con 0x7f0a58107560 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.082+0000 7f0a567fc640 1 -- 172.21.15.7:0/2383626326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a48003e20 con 0x7f0a58107560 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.082+0000 7f0a567fc640 1 -- 172.21.15.7:0/2383626326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48039040 con 0x7f0a58107560 2024-03-19T00:49:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.082+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5819fe60 con 0x7f0a58107560 2024-03-19T00:49:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.084+0000 7f0a567fc640 1 -- 172.21.15.7:0/2383626326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0a48044020 con 0x7f0a58107560 2024-03-19T00:49:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.084+0000 7f0a6224e740 1 Processor -- start 2024-03-19T00:49:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.084+0000 7f0a6224e740 1 -- start start 2024-03-19T00:49:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.084+0000 7f0a567fc640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a400a2800 0x7f0a400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc43615e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc4361af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc4361da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a6224e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bc4357a500 con 0x55bc43616440 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a6224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bc43522be0 con 0x55bc4361b5b0 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a6224e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bc435765a0 con 0x55bc43615a50 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a567fc640 1 -- 172.21.15.7:0/2383626326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a480f0fc0 con 0x7f0a58107560 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a400a2800 0x7f0a400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc43615e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5cc03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc4361af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc43615e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:52878/0 (socket says 172.21.15.7:52878) 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 -- 172.21.15.7:0/2940779389 learned_addr learned my addr 172.21.15.7:0/2940779389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5cc03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc4361af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49664/0 (socket says 172.21.15.7:49664) 2024-03-19T00:49:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc4361da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 -- 172.21.15.7:0/2940779389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 msgr2=0x55bc4361da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc4361da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 -- 172.21.15.7:0/2940779389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 msgr2=0x55bc4361af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc4361af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 -- 172.21.15.7:0/2940779389 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43544000 con 0x55bc43615a50 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc4361af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a400a2800 0x7f0a400a4cc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0a58198f80 tx=0x7f0a4c009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a5d404640 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc43615e70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009d20 tx=0x7f0a50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a5876d8c277b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.085+0000 7f0a3f7fe640 1 -- 172.21.15.7:0/2940779389 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5001e070 con 0x55bc43615a50 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a3f7fe640 1 -- 172.21.15.7:0/2940779389 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a50009e10 con 0x55bc43615a50 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a3f7fe640 1 -- 172.21.15.7:0/2940779389 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50018870 con 0x55bc43615a50 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 -- 172.21.15.7:0/2940779389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 msgr2=0x55bc43615e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc43615e70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009d20 tx=0x7f0a50011810 comp rx=0 tx=0).stop 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 -- 172.21.15.7:0/2940779389 shutdown_connections 2024-03-19T00:49:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc4361da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc4361af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2940779389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc43615e70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 -- 172.21.15.7:0/2940779389 >> 172.21.15.7:0/2940779389 conn(0x55bc43610fc0 msgr2=0x55bc436113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 -- 172.21.15.7:0/2940779389 shutdown_connections 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.086+0000 7f0a6224e740 1 -- 172.21.15.7:0/2940779389 wait complete. 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 -- start start 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 Processor -- start 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc4361fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc43620420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43627460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bc4361e560 con 0x55bc43616440 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bc43536160 con 0x55bc4361b5b0 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bc4361e6e0 con 0x55bc43615a50 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a6224e740 10 client.0 _ll_register_callbacks cb 0x55bc43572a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc43620420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc43620420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49676/0 (socket says 172.21.15.7:49676) 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 -- 172.21.15.7:0/2568875826 learned_addr learned my addr 172.21.15.7:0/2568875826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43627460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5d404640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc4361fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 -- 172.21.15.7:0/2568875826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 msgr2=0x55bc4361fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc4361fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 -- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 msgr2=0x55bc43627460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43627460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.087+0000 7f0a5cc03640 1 -- 172.21.15.7:0/2568875826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc433587d0 con 0x55bc43616440 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc43620420 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0058c0 tx=0x7f0a4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60739abe899be1bf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a3cff9640 1 -- 172.21.15.7:0/2568875826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c005020 con 0x55bc43616440 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a50009e10 con 0x55bc43616440 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a3cff9640 1 -- 172.21.15.7:0/2568875826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a4c0051c0 con 0x55bc43616440 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a3cff9640 1 -- 172.21.15.7:0/2568875826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c030050 con 0x55bc43616440 2024-03-19T00:49:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a6224e740 10 client.17826 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc4328e3f0 con 0x55bc43616440 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a3cff9640 1 -- 172.21.15.7:0/2568875826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a50009e10 con 0x55bc43616440 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a3cff9640 1 -- 172.21.15.7:0/2568875826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a4c0047a0 con 0x55bc43616440 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a6224e740 10 client.17826 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a6224e740 10 client.17826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.088+0000 7f0a6224e740 20 client.17826 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.089+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc4362bd70 0x55bc4362e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.089+0000 7f0a6224e740 4 client.17826 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.089+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc43587a40 con 0x55bc4362bd70 2024-03-19T00:49:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.089+0000 7f0a5d404640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc4362bd70 0x55bc4362e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.089+0000 7f0a3cff9640 10 client.17826 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.089+0000 7f0a5d404640 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc4362bd70 0x55bc4362e1b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=844ba4cf733625ba server_cookie=853f90848b55e806 in_seq=0 out_seq=0 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a3cff9640 1 -- 172.21.15.7:0/2568875826 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc43587a40 con 0x55bc4362bd70 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a3cff9640 10 client.17826 handle_command_reply: tid=0 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 client.17826 shutdown 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 msgr2=0x55bc43620420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc43620420 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0058c0 tx=0x7f0a4c005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 shutdown_connections 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43627460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc4362bd70 0x55bc4362e1b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc43620420 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 --2- 172.21.15.7:0/2568875826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43615a50 0x55bc4361fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a3cff9640 0 client.17826 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 >> 172.21.15.7:0/2568875826 conn(0x55bc43610fc0 msgr2=0x55bc436101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 shutdown_connections 2024-03-19T00:49:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 1 -- 172.21.15.7:0/2568875826 wait complete. 2024-03-19T00:49:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.090+0000 7f0a6224e740 20 client.17826 trim_cache size 0 max 16384 2024-03-19T00:49:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 Processor -- start 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 -- start start 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 0x55bc4360bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43616440 0x55bc43627420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43614930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581acb10 con 0x55bc43615a50 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581aa3e0 con 0x55bc4361b5b0 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a6224e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581a8230 con 0x55bc43616440 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 0x55bc4360bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 0x55bc4360bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49686/0 (socket says 172.21.15.7:49686) 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 -- 172.21.15.7:0/3907024943 learned_addr learned my addr 172.21.15.7:0/3907024943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43616440 0x55bc43627420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43614930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 -- 172.21.15.7:0/3907024943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43616440 msgr2=0x55bc43627420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43616440 0x55bc43627420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 -- 172.21.15.7:0/3907024943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 msgr2=0x55bc43614930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43614930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 -- 172.21.15.7:0/3907024943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43617f20 con 0x55bc43615a50 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43614930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.108+0000 7f0a5d404640 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 0x55bc4360bf80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0a500058c0 tx=0x7f0a500097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186780c084ce13d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a3effd640 1 -- 172.21.15.7:0/3907024943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50030070 con 0x55bc43615a50 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a3effd640 1 -- 172.21.15.7:0/3907024943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a500098f0 con 0x55bc43615a50 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a3effd640 1 -- 172.21.15.7:0/3907024943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5001e040 con 0x55bc43615a50 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 -- 172.21.15.7:0/3907024943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 msgr2=0x55bc4360bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 0x55bc4360bf80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0a500058c0 tx=0x7f0a500097a0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 -- 172.21.15.7:0/3907024943 shutdown_connections 2024-03-19T00:49:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc4361b5b0 0x55bc43614930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc43616440 0x55bc43627420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 --2- 172.21.15.7:0/3907024943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43615a50 0x55bc4360bf80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 -- 172.21.15.7:0/3907024943 >> 172.21.15.7:0/3907024943 conn(0x55bc43610fc0 msgr2=0x55bc43614220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 -- 172.21.15.7:0/3907024943 shutdown_connections 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.109+0000 7f0a6224e740 1 -- 172.21.15.7:0/3907024943 wait complete. 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 -- start start 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 Processor -- start 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc43615a50 0x55bc43624f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc436254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc4361b5b0 0x55bc436264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581a88a0 con 0x55bc43616440 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581a9d10 con 0x55bc43615a50 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581a8f70 con 0x55bc4361b5b0 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a6224e740 10 client.0 _ll_register_callbacks cb 0x7f0a4c011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc436254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc436254a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49700/0 (socket says 172.21.15.7:49700) 2024-03-19T00:49:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 -- 172.21.15.7:0/1792512668 learned_addr learned my addr 172.21.15.7:0/1792512668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5dc05640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc4361b5b0 0x55bc436264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5d404640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc43615a50 0x55bc43624f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 -- 172.21.15.7:0/1792512668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc4361b5b0 msgr2=0x55bc436264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc4361b5b0 0x55bc436264b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 -- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc43615a50 msgr2=0x55bc43624f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc43615a50 0x55bc43624f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.110+0000 7f0a5cc03640 1 -- 172.21.15.7:0/1792512668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4324e0f0 con 0x55bc43616440 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a5cc03640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc436254a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0099c0 tx=0x7f0a4c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19c2afe412a3ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a3cff9640 1 -- 172.21.15.7:0/1792512668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c038020 con 0x55bc43616440 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a3cff9640 1 -- 172.21.15.7:0/1792512668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a4c003c10 con 0x55bc43616440 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a500098f0 con 0x55bc43616440 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a3cff9640 1 -- 172.21.15.7:0/1792512668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c043050 con 0x55bc43616440 2024-03-19T00:49:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 10 client.17832 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc436922f0 con 0x55bc43616440 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a3cff9640 1 -- 172.21.15.7:0/1792512668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a500098f0 con 0x55bc43616440 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a3cff9640 1 -- 172.21.15.7:0/1792512668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0a4c005320 con 0x55bc43616440 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 10 client.17832 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 20 client.17832 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc43692cb0 0x55bc436950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 4 client.17832 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.111+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc43581d90 con 0x55bc43692cb0 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a5d404640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc43692cb0 0x55bc436950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a3cff9640 10 client.17832 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a5d404640 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc43692cb0 0x55bc436950d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5ee37fadd9a64c server_cookie=6b9e233307bafc60 in_seq=0 out_seq=0 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a3cff9640 1 -- 172.21.15.7:0/1792512668 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc43581d90 con 0x55bc43692cb0 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a3cff9640 10 client.17832 handle_command_reply: tid=0 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 client.17832 shutdown 2024-03-19T00:49:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 msgr2=0x55bc436254a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc436254a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0099c0 tx=0x7f0a4c0049d0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 shutdown_connections 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc4361b5b0 0x55bc436264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55bc43692cb0 0x55bc436950d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc43616440 0x55bc436254a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a3cff9640 0 client.17832 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 --2- 172.21.15.7:0/1792512668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bc43615a50 0x55bc43624f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 >> 172.21.15.7:0/1792512668 conn(0x55bc43610fc0 msgr2=0x55bc43614ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 shutdown_connections 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.112+0000 7f0a6224e740 1 -- 172.21.15.7:0/1792512668 wait complete. 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a6224e740 20 client.17832 trim_cache size 0 max 16384 2024-03-19T00:49:15.120 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:49:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-19T00:49:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-19T00:49:15.121 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:49:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a400a2800 msgr2=0x7f0a400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a400a2800 0x7f0a400a4cc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0a58198f80 tx=0x7f0a4c009b60 comp rx=0 tx=0).stop 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 msgr2=0x7f0a5819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 0x7f0a5819f020 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0a4802c700 tx=0x7f0a480328a0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 shutdown_connections 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0a400a2800 0x7f0a400a4cc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58107560 0x7f0a5819f020 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58104c30 0x7f0a58197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 --2- 172.21.15.7:0/2383626326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a58102300 0x7f0a58197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 >> 172.21.15.7:0/2383626326 conn(0x7f0a580fbcb0 msgr2=0x7f0a58102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 shutdown_connections 2024-03-19T00:49:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.113+0000 7f0a5f668640 1 -- 172.21.15.7:0/2383626326 wait complete. 2024-03-19T00:49:15.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T00:49:15.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:49:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- 172.21.15.7:0/2023010425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d0104cc0 msgr2=0x7f16d01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- 172.21.15.7:0/2023010425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d0104cc0 0x7f16d01070b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009860 tx=0x7f16c402bb70 comp rx=0 tx=0).stop 2024-03-19T00:49:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- 172.21.15.7:0/2023010425 shutdown_connections 2024-03-19T00:49:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- 172.21.15.7:0/2023010425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d01075f0 0x7f16d01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- 172.21.15.7:0/2023010425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d0104cc0 0x7f16d01070b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- 172.21.15.7:0/2023010425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- 172.21.15.7:0/2023010425 >> 172.21.15.7:0/2023010425 conn(0x7f16d00fbce0 msgr2=0x7f16d00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- 172.21.15.7:0/2023010425 shutdown_connections 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- 172.21.15.7:0/2023010425 wait complete. 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 Processor -- start 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- start start 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d0104cc0 0x7f16d0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d01075f0 0x7f16d019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d360 con 0x7f16d0102390 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.227+0000 7f16d651d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d1e0 con 0x7f16d0104cc0 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16d651d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d4e0 con 0x7f16d01075f0 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49724/0 (socket says 172.21.15.7:49724) 2024-03-19T00:49:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 -- 172.21.15.7:0/3643847787 learned_addr learned my addr 172.21.15.7:0/3643847787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cf7fe640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d0104cc0 0x7f16d0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16d4aba640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d01075f0 0x7f16d019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 -- 172.21.15.7:0/3643847787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d01075f0 msgr2=0x7f16d019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d01075f0 0x7f16d019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 -- 172.21.15.7:0/3643847787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d0104cc0 msgr2=0x7f16d0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d0104cc0 0x7f16d0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 -- 172.21.15.7:0/3643847787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019f6f0 con 0x7f16d0102390 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cf7fe640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d0104cc0 0x7f16d0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cffff640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0197ad0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f16c000e9c0 tx=0x7f16c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c000ccc0 con 0x7f16d0102390 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019f9e0 con 0x7f16d0102390 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c00054d0 con 0x7f16d0102390 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0010430 con 0x7f16d0102390 2024-03-19T00:49:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.228+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d019ff50 con 0x7f16d0102390 2024-03-19T00:49:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.229+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d0068af0 con 0x7f16d0102390 2024-03-19T00:49:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.230+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f16c00046c0 con 0x7f16d0102390 2024-03-19T00:49:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.231+0000 7f16cd7fa640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16b80a2800 0x7f16b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.231+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f16c00cf070 con 0x7f16d0102390 2024-03-19T00:49:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.231+0000 7f16cf7fe640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16b80a2800 0x7f16b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.231+0000 7f16cf7fe640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16b80a2800 0x7f16b80a4cc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f16c402c180 tx=0x7f16c40349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.236+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f16c009bd50 con 0x7f16d0102390 2024-03-19T00:49:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:561/600: open fd = 3 2024-03-19T00:49:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:562/600: open fd = 3 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:563/600: open fd = 3 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:564/600: open fd = 3 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:565/600: open fd = 3 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:566/600: open fd = 3 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:567/600: open fd = 3 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:568/600: open fd = 3 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:569/600: open fd = 3 2024-03-19T00:49:15.375 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:570/600: open fd = 3 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:571/600: open fd = 3 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:572/600: open fd = 3 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:573/600: open fd = 3 2024-03-19T00:49:15.376 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:574/600: open fd = 3 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:575/600: open fd = 3 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:576/600: open fd = 3 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:577/600: open fd = 3 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:578/600: open fd = 3 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:579/600: open fd = 3 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:580/600: open fd = 3 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:581/600: open fd = 3 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:582/600: open fd = 3 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:583/600: open fd = 3 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:584/600: open fd = 3 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:585/600: open fd = 3 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:586/600: open fd = 3 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:587/600: open fd = 3 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:588/600: open fd = 3 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:589/600: open fd = 3 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:590/600: open fd = 3 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.381 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:591/600: open fd = 3 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:592/600: open fd = 3 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:593/600: open fd = 3 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:594/600: open fd = 3 2024-03-19T00:49:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:595/600: open fd = 3 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:596/600: open fd = 3 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:597/600: open fd = 3 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.383 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:598/600: open fd = 3 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:599/600: open fd = 3 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:600/600: open fd = 3 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:write ret = 32 2024-03-19T00:49:15.384 INFO:tasks.workunit.client.0.smithi007.stdout:pread ret = 64 2024-03-19T00:49:15.385 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2024-03-19T00:49:15.385 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2024-03-19T00:49:15.386 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-19T00:49:15.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-19T00:49:15.437 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-03-19T00:49:15.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-19T00:49:15.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.459+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d00fd7d0 con 0x7f16d0102390 2024-03-19T00:49:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.460+0000 7f16cd7fa640 1 -- 172.21.15.7:0/3643847787 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11855 (secure 0 0 0) 0x7f16c009b650 con 0x7f16d0102390 2024-03-19T00:49:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:49:15.461 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:41:21.039340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24497,24490]},"id":1}]} 2024-03-19T00:49:15.461 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16b80a2800 msgr2=0x7f16b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16b80a2800 0x7f16b80a4cc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f16c402c180 tx=0x7f16c40349a0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 msgr2=0x7f16d0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0197ad0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f16c000e9c0 tx=0x7f16c000ee80 comp rx=0 tx=0).stop 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 shutdown_connections 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f16b80a2800 0x7f16b80a4cc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d01075f0 0x7f16d019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16d0104cc0 0x7f16d0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 --2- 172.21.15.7:0/3643847787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d0102390 0x7f16d0197ad0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 >> 172.21.15.7:0/3643847787 conn(0x7f16d00fbce0 msgr2=0x7f16d0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 shutdown_connections 2024-03-19T00:49:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.462+0000 7f16d651d640 1 -- 172.21.15.7:0/3643847787 wait complete. 2024-03-19T00:49:15.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T00:49:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 -- 172.21.15.7:0/1072311323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 msgr2=0x7f6d10101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/1072311323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10101d90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009860 tx=0x7f6d0402bb70 comp rx=0 tx=0).stop 2024-03-19T00:49:15.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 -- 172.21.15.7:0/1072311323 shutdown_connections 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/1072311323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d101022d0 0x7f6d10108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/1072311323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10101d90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/1072311323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d10100720 0x7f6d10100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 -- 172.21.15.7:0/1072311323 >> 172.21.15.7:0/1072311323 conn(0x7f6d100fbc50 msgr2=0x7f6d100fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 -- 172.21.15.7:0/1072311323 shutdown_connections 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.618+0000 7f6d14cdd640 1 -- 172.21.15.7:0/1072311323 wait complete. 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 Processor -- start 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 -- start start 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d10100720 0x7f6d10197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101022d0 0x7f6d1019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010c230 con 0x7f6d10101910 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010c0b0 con 0x7f6d101022d0 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d14cdd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010c3b0 con 0x7f6d10100720 2024-03-19T00:49:15.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49742/0 (socket says 172.21.15.7:49742) 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/3048482087 learned_addr learned my addr 172.21.15.7:0/3048482087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d10100720 0x7f6d10197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.619+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101022d0 0x7f6d1019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/3048482087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d10100720 msgr2=0x7f6d10197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d10100720 0x7f6d10197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/3048482087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101022d0 msgr2=0x7f6d1019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101022d0 0x7f6d1019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/3048482087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1019f620 con 0x7f6d10101910 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101022d0 0x7f6d1019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10197f90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005940 tx=0x7f6d04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6cff7fe640 1 -- 172.21.15.7:0/3048482087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04041020 con 0x7f6d10101910 2024-03-19T00:49:15.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6cff7fe640 1 -- 172.21.15.7:0/3048482087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d0402c9c0 con 0x7f6d10101910 2024-03-19T00:49:15.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1019f8b0 con 0x7f6d10101910 2024-03-19T00:49:15.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6cff7fe640 1 -- 172.21.15.7:0/3048482087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04039040 con 0x7f6d10101910 2024-03-19T00:49:15.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.620+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1019fdf0 con 0x7f6d10101910 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 Processor -- start 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6cff7fe640 1 -- 172.21.15.7:0/3048482087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6d04044020 con 0x7f6d10101910 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 -- start start 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba722f0 0x55a79ba72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba72ce0 0x55a79ba777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 0x55a79ba7a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a79b9d64b0 con 0x55a79ba77e50 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a79b97ed70 con 0x55a79ba722f0 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d1799d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a79b9d2550 con 0x55a79ba72ce0 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 0x55a79ba7a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba722f0 0x55a79ba72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 0x55a79ba7a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49758/0 (socket says 172.21.15.7:49758) 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/169526865 learned_addr learned my addr 172.21.15.7:0/169526865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6cff7fe640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6cf80a2880 0x7f6cf80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.623+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba72ce0 0x55a79ba777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6cf80a2880 0x7f6cf80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6cff7fe640 1 -- 172.21.15.7:0/3048482087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6d040f0690 con 0x7f6d10101910 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/169526865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba72ce0 msgr2=0x55a79ba777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba72ce0 0x55a79ba777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/169526865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba722f0 msgr2=0x55a79ba72710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba722f0 0x55a79ba72710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/169526865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a79b9a1210 con 0x55a79ba77e50 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba722f0 0x55a79ba72710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba72ce0 0x55a79ba777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 0x55a79ba7a2e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6d040bc9d0 tx=0x7f6d0402bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653bf4c171957485 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6cf80a2880 0x7f6cf80a4d40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55a79ba73540 tx=0x7f6d08004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6cfcff9640 1 -- 172.21.15.7:0/169526865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040bc630 con 0x55a79ba77e50 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6cfcff9640 1 -- 172.21.15.7:0/169526865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d040bd3d0 con 0x55a79ba77e50 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.624+0000 7f6cfcff9640 1 -- 172.21.15.7:0/169526865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04003760 con 0x55a79ba77e50 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 -- 172.21.15.7:0/169526865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 msgr2=0x55a79ba7a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 0x55a79ba7a2e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6d040bc9d0 tx=0x7f6d0402bb70 comp rx=0 tx=0).stop 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 -- 172.21.15.7:0/169526865 shutdown_connections 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba77e50 0x55a79ba7a2e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba72ce0 0x55a79ba777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 --2- 172.21.15.7:0/169526865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba722f0 0x55a79ba72710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 -- 172.21.15.7:0/169526865 >> 172.21.15.7:0/169526865 conn(0x55a79ba6d860 msgr2=0x55a79ba6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 -- 172.21.15.7:0/169526865 shutdown_connections 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 -- 172.21.15.7:0/169526865 wait complete. 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 -- start start 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.625+0000 7f6d1799d740 1 Processor -- start 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a79b9d64b0 con 0x55a79ba722f0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a79b992310 con 0x55a79ba72ce0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba7abc0 con 0x55a79ba77e50 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 10 client.0 _ll_register_callbacks cb 0x55a79b9862d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba7c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49766/0 (socket says 172.21.15.7:49766) 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 -- 172.21.15.7:0/2588841770 learned_addr learned my addr 172.21.15.7:0/2588841770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 -- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 msgr2=0x55a79ba839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba839f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 -- 172.21.15.7:0/2588841770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 msgr2=0x55a79ba7c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba7c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 -- 172.21.15.7:0/2588841770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79b6ea3f0 con 0x55a79ba722f0 2024-03-19T00:49:15.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba7c470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005fd0 tx=0x7f6d00012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d75738cbe4ce57 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/2588841770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000f8c0 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/2588841770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6d0000fa60 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.626+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d040bd3d0 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/2588841770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d0000fc60 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 10 client.17850 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a79b7b47d0 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/2588841770 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d040bd3d0 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/2588841770 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6d0001c420 con 0x55a79ba722f0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 10 client.17850 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 10 client.17850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 20 client.17850 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79ba883d0 0x55a79ba8a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 4 client.17850 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a79ba343e0 con 0x55a79ba883d0 2024-03-19T00:49:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.627+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79ba883d0 0x55a79ba8a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6cf5ffb640 10 client.17850 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79ba883d0 0x55a79ba8a840 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a808aa7d81c23a54 server_cookie=af79524b4e312d82 in_seq=0 out_seq=0 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/2588841770 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a79ba343e0 con 0x55a79ba883d0 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6cf5ffb640 10 client.17850 handle_command_reply: tid=0 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 client.17850 shutdown 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 msgr2=0x55a79ba7c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba7c470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005fd0 tx=0x7f6d00012b60 comp rx=0 tx=0).stop 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 shutdown_connections 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79ba883d0 0x55a79ba8a840 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba7c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6cf5ffb640 0 client.17850 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 --2- 172.21.15.7:0/2588841770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba7c470 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 >> 172.21.15.7:0/2588841770 conn(0x55a79ba6d860 msgr2=0x55a79ba6cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.628+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 shutdown_connections 2024-03-19T00:49:15.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.629+0000 7f6d1799d740 1 -- 172.21.15.7:0/2588841770 wait complete. 2024-03-19T00:49:15.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.629+0000 7f6d1799d740 20 client.17850 trim_cache size 0 max 16384 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 Processor -- start 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 -- start start 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba722f0 0x55a79ba6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 0x55a79ba839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba77e50 0x55a79ba8f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101acb40 con 0x55a79ba72ce0 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101aa410 con 0x55a79ba77e50 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d1799d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a8260 con 0x55a79ba722f0 2024-03-19T00:49:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 0x55a79ba839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 0x55a79ba839b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49776/0 (socket says 172.21.15.7:49776) 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/904416096 learned_addr learned my addr 172.21.15.7:0/904416096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba77e50 0x55a79ba8f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba722f0 0x55a79ba6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/904416096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba722f0 msgr2=0x55a79ba6dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba722f0 0x55a79ba6dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/904416096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba77e50 msgr2=0x55a79ba8f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba77e50 0x55a79ba8f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 -- 172.21.15.7:0/904416096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a79ba43880 con 0x55a79ba72ce0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.642+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 0x55a79ba839b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6d040b9130 tx=0x7f6d04005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cb610e9f7847f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6cf7fff640 1 -- 172.21.15.7:0/904416096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040bc630 con 0x55a79ba72ce0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6cf7fff640 1 -- 172.21.15.7:0/904416096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d040bd3d0 con 0x55a79ba72ce0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6cf7fff640 1 -- 172.21.15.7:0/904416096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04003760 con 0x55a79ba72ce0 2024-03-19T00:49:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 -- 172.21.15.7:0/904416096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 msgr2=0x55a79ba839b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 0x55a79ba839b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6d040b9130 tx=0x7f6d04005d40 comp rx=0 tx=0).stop 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 -- 172.21.15.7:0/904416096 shutdown_connections 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba77e50 0x55a79ba8f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba72ce0 0x55a79ba839b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 --2- 172.21.15.7:0/904416096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba722f0 0x55a79ba6dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 -- 172.21.15.7:0/904416096 >> 172.21.15.7:0/904416096 conn(0x55a79ba6d860 msgr2=0x55a79ba71350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 -- 172.21.15.7:0/904416096 shutdown_connections 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.643+0000 7f6d1799d740 1 -- 172.21.15.7:0/904416096 wait complete. 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 -- start start 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 Processor -- start 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba80fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba82520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a88d0 con 0x55a79ba722f0 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a9d40 con 0x55a79ba72ce0 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a8fa0 con 0x55a79ba77e50 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d1799d740 10 client.0 _ll_register_callbacks cb 0x7f6d00009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba80fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba80fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49778/0 (socket says 172.21.15.7:49778) 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 -- 172.21.15.7:0/1590030847 learned_addr learned my addr 172.21.15.7:0/1590030847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0ed9d640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba82520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 -- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 msgr2=0x55a79ba82520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba82520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 -- 172.21.15.7:0/1590030847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 msgr2=0x55a79ba81510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba81510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0e59c640 1 -- 172.21.15.7:0/1590030847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79ba82b10 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.644+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d0e59c640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba80fd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x55a79ba693a0 tx=0x7f6d00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15fb31ffe568e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/1590030847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0002f060 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/1590030847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6d0000f790 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d040bd3d0 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/1590030847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d0001c6f0 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 10 client.17856 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a79baee780 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/1590030847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d040bd3d0 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/1590030847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6d00004760 con 0x55a79ba722f0 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 10 client.17856 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 10 client.17856 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 20 client.17856 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79baef0f0 0x55a79baf1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 4 client.17856 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.645+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 --> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a79b9e39f0 con 0x55a79baef0f0 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79baef0f0 0x55a79baf1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6cf5ffb640 10 client.17856 ms_handle_connect on v2:172.21.15.204:6838/945498116 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6d0dd9b640 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79baef0f0 0x55a79baf1510 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe3e8c157fc755a4 server_cookie=2c8f79874ca9e707 in_seq=0 out_seq=0 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6cf5ffb640 1 -- 172.21.15.7:0/1590030847 <== mds.2 v2:172.21.15.204:6838/945498116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a79b9e39f0 con 0x55a79baef0f0 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6cf5ffb640 10 client.17856 handle_command_reply: tid=0 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6d1799d740 1 client.17856 shutdown 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 msgr2=0x55a79ba80fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.646+0000 7f6d1799d740 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba80fd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x55a79ba693a0 tx=0x7f6d00009e30 comp rx=0 tx=0).stop 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 shutdown_connections 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:6838/945498116,v1:172.21.15.204:6839/945498116] conn(0x55a79baef0f0 0x55a79baf1510 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a79ba77e50 0x55a79ba82520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6cf5ffb640 0 client.17856 ms_handle_reset on v2:172.21.15.204:6838/945498116 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a79ba72ce0 0x55a79ba81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 --2- 172.21.15.7:0/1590030847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a79ba722f0 0x55a79ba80fd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 >> 172.21.15.7:0/1590030847 conn(0x55a79ba6d860 msgr2=0x55a79ba71350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 shutdown_connections 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 1 -- 172.21.15.7:0/1590030847 wait complete. 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d1799d740 20 client.17856 trim_cache size 0 max 16384 2024-03-19T00:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6cf80a2880 msgr2=0x7f6cf80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6cf80a2880 0x7f6cf80a4d40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55a79ba73540 tx=0x7f6d08004ab0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 msgr2=0x7f6d10197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10197f90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005940 tx=0x7f6d04004270 comp rx=0 tx=0).stop 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 shutdown_connections 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6cf80a2880 0x7f6cf80a4d40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d101022d0 0x7f6d1019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d10101910 0x7f6d10197f90 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 --2- 172.21.15.7:0/3048482087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d10100720 0x7f6d10197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 >> 172.21.15.7:0/3048482087 conn(0x7f6d100fbc50 msgr2=0x7f6d100fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 shutdown_connections 2024-03-19T00:49:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.647+0000 7f6d14cdd640 1 -- 172.21.15.7:0/3048482087 wait complete. 2024-03-19T00:49:15.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T00:49:15.729 DEBUG:teuthology.parallel:result is None 2024-03-19T00:49:15.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-19T00:49:15.766 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-19T00:49:15.766 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-19T00:49:15.777 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.765+0000 7f6f9d84b640 1 -- 172.21.15.7:0/3572360176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98101820 msgr2=0x7f6f98101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.765+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/3572360176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98101820 0x7f6f98101c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009860 tx=0x7f6f8802bb70 comp rx=0 tx=0).stop 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- 172.21.15.7:0/3572360176 shutdown_connections 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/3572360176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f981033d0 0x7f6f98109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/3572360176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98102a10 0x7f6f98102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/3572360176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98101820 0x7f6f98101c40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- 172.21.15.7:0/3572360176 >> 172.21.15.7:0/3572360176 conn(0x7f6f980fcd50 msgr2=0x7f6f980ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- 172.21.15.7:0/3572360176 shutdown_connections 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- 172.21.15.7:0/3572360176 wait complete. 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 Processor -- start 2024-03-19T00:49:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- start start 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 0x7f6f98197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98102a10 0x7f6f98198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f981033d0 0x7f6f9819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810d350 con 0x7f6f98101820 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810d1d0 con 0x7f6f981033d0 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f9d84b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810d4d0 con 0x7f6f98102a10 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 0x7f6f98197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 0x7f6f98197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49790/0 (socket says 172.21.15.7:49790) 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.766+0000 7f6f96ffd640 1 -- 172.21.15.7:0/1060746011 learned_addr learned my addr 172.21.15.7:0/1060746011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f977fe640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f981033d0 0x7f6f9819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f96ffd640 1 -- 172.21.15.7:0/1060746011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98102a10 msgr2=0x7f6f98198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98102a10 0x7f6f98198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f96ffd640 1 -- 172.21.15.7:0/1060746011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f981033d0 msgr2=0x7f6f9819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f981033d0 0x7f6f9819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f96ffd640 1 -- 172.21.15.7:0/1060746011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9819f670 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 0x7f6f98197af0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6f88005940 tx=0x7f6f880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f9c849640 1 -- 172.21.15.7:0/1060746011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88041020 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9819f900 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f9c849640 1 -- 172.21.15.7:0/1060746011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f880035f0 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f9c849640 1 -- 172.21.15.7:0/1060746011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88039040 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.767+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9819fe40 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.769+0000 7f6f9c849640 1 -- 172.21.15.7:0/1060746011 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f6f88044020 con 0x7f6f98101820 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 Processor -- start 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 -- start start 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0aace0 0x55b43d0af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0afe50 0x55b43d0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b43d00e4b0 con 0x55b43d0aa2f0 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b43cfb6d70 con 0x55b43d0afe50 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6fa0435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b43d00a550 con 0x55b43d0aace0 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f9c849640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f840a2800 0x7f6f840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0aa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49794/0 (socket says 172.21.15.7:49794) 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 -- 172.21.15.7:0/4142208249 learned_addr learned my addr 172.21.15.7:0/4142208249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0afe50 0x55b43d0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f840a2800 0x7f6f840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0aace0 0x55b43d0af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f9c849640 1 -- 172.21.15.7:0/1060746011 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f880f0810 con 0x7f6f98101820 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 -- 172.21.15.7:0/4142208249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0aace0 msgr2=0x55b43d0af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0aace0 0x55b43d0af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 -- 172.21.15.7:0/4142208249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0afe50 msgr2=0x55b43d0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0afe50 0x55b43d0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f977fe640 1 -- 172.21.15.7:0/4142208249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43cfd9210 con 0x55b43d0aa2f0 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.770+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0afe50 0x55b43d0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6f977fe640 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0aa710 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6f90011e20 tx=0x7f6f9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a9d94ef94e906f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6f977fe640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f840a2800 0x7f6f840a4cc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000fc90 tx=0x7f6f9001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6f8dffb640 1 -- 172.21.15.7:0/4142208249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001dbf0 con 0x55b43d0aa2f0 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6f8dffb640 1 -- 172.21.15.7:0/4142208249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f9001dd90 con 0x55b43d0aa2f0 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6f8dffb640 1 -- 172.21.15.7:0/4142208249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90004280 con 0x55b43d0aa2f0 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 -- 172.21.15.7:0/4142208249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 msgr2=0x55b43d0aa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0aa710 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6f90011e20 tx=0x7f6f9000f790 comp rx=0 tx=0).stop 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 -- 172.21.15.7:0/4142208249 shutdown_connections 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0afe50 0x55b43d0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0aace0 0x55b43d0af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 --2- 172.21.15.7:0/4142208249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0aa710 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.771+0000 7f6fa0435740 1 -- 172.21.15.7:0/4142208249 >> 172.21.15.7:0/4142208249 conn(0x55b43d0a5860 msgr2=0x55b43d0a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 -- 172.21.15.7:0/4142208249 shutdown_connections 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 -- 172.21.15.7:0/4142208249 wait complete. 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 -- start start 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 Processor -- start 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aa2f0 0x55b43d0b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 0x55b43d0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b43d0b2d40 con 0x55b43d0aace0 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b43cfca310 con 0x55b43d0aa2f0 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b43d0b2ec0 con 0x55b43d0afe50 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6fa0435740 10 client.0 _ll_register_callbacks cb 0x55b43cfbe2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6f8ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 0x55b43d0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aa2f0 0x55b43d0b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6f8ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 0x55b43d0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49796/0 (socket says 172.21.15.7:49796) 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6f8ffff640 1 -- 172.21.15.7:0/1159054580 learned_addr learned my addr 172.21.15.7:0/1159054580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.772+0000 7f6f977fe640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f8ffff640 1 -- 172.21.15.7:0/1159054580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 msgr2=0x55b43d0bbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0bbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f8ffff640 1 -- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aa2f0 msgr2=0x55b43d0b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aa2f0 0x55b43d0b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f8ffff640 1 -- 172.21.15.7:0/1159054580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43cd223f0 con 0x55b43d0aace0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aa2f0 0x55b43d0b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 0x55b43d0b4cc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6f800116f0 tx=0x7f6f80011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a53138a9f063106 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f76ffd640 1 -- 172.21.15.7:0/1159054580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80004440 con 0x55b43d0aace0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f76ffd640 1 -- 172.21.15.7:0/1159054580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f80005020 con 0x55b43d0aace0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9001dd90 con 0x55b43d0aace0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f76ffd640 1 -- 172.21.15.7:0/1159054580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f80005220 con 0x55b43d0aace0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6fa0435740 10 client.17868 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b43cdec7d0 con 0x55b43d0aace0 2024-03-19T00:49:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.773+0000 7f6f76ffd640 1 -- 172.21.15.7:0/1159054580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f9001dd90 con 0x55b43d0aace0 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6f76ffd640 1 -- 172.21.15.7:0/1159054580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6f80018070 con 0x55b43d0aace0 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6fa0435740 10 client.17868 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6fa0435740 10 client.17868 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6fa0435740 20 client.17868 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6fa0435740 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d0c0710 0x55b43d0c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6fa0435740 4 client.17868 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b43d06c3e0 con 0x55b43d0c0710 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d0c0710 0x55b43d0c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6f76ffd640 10 client.17868 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.774+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d0c0710 0x55b43d0c2b90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82ebc8567fae7e57 server_cookie=d16417aabf0ebfb4 in_seq=0 out_seq=0 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6f76ffd640 1 -- 172.21.15.7:0/1159054580 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b43d06c3e0 con 0x55b43d0c0710 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6f76ffd640 10 client.17868 handle_command_reply: tid=0 2024-03-19T00:49:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 client.17868 shutdown 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 msgr2=0x55b43d0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 0x55b43d0b4cc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6f800116f0 tx=0x7f6f80011bb0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 shutdown_connections 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d0c0710 0x55b43d0c2b90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0bbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aace0 0x55b43d0b4cc0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 --2- 172.21.15.7:0/1159054580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aa2f0 0x55b43d0b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6f76ffd640 0 client.17868 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 >> 172.21.15.7:0/1159054580 conn(0x55b43d0a5860 msgr2=0x55b43d0a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 shutdown_connections 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 1 -- 172.21.15.7:0/1159054580 wait complete. 2024-03-19T00:49:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.775+0000 7f6fa0435740 20 client.17868 trim_cache size 0 max 16384 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.788+0000 7f6fa0435740 1 Processor -- start 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.788+0000 7f6fa0435740 1 -- start start 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.788+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6fa0435740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981acb40 con 0x55b43d0aa2f0 2024-03-19T00:49:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6fa0435740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981aa410 con 0x55b43d0aace0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6fa0435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981a8260 con 0x55b43d0afe50 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49806/0 (socket says 172.21.15.7:49806) 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 -- 172.21.15.7:0/109243304 learned_addr learned my addr 172.21.15.7:0/109243304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f977fe640 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 -- 172.21.15.7:0/109243304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 msgr2=0x55b43d0c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 -- 172.21.15.7:0/109243304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 msgr2=0x55b43d0bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 -- 172.21.15.7:0/109243304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43d07b880 con 0x55b43d0aa2f0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f977fe640 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0a91d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6f880b97c0 tx=0x7f6f8808fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7797c0f3b99c6573 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f8d7fa640 1 -- 172.21.15.7:0/109243304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880bd4f0 con 0x55b43d0aa2f0 2024-03-19T00:49:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f8d7fa640 1 -- 172.21.15.7:0/109243304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f88088010 con 0x55b43d0aa2f0 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.789+0000 7f6f8d7fa640 1 -- 172.21.15.7:0/109243304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88003b10 con 0x55b43d0aa2f0 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- 172.21.15.7:0/109243304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 msgr2=0x55b43d0a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0a91d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6f880b97c0 tx=0x7f6f8808fb20 comp rx=0 tx=0).stop 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- 172.21.15.7:0/109243304 shutdown_connections 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- 172.21.15.7:0/109243304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0a91d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- 172.21.15.7:0/109243304 >> 172.21.15.7:0/109243304 conn(0x55b43d0a5860 msgr2=0x55b43d0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- 172.21.15.7:0/109243304 shutdown_connections 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- 172.21.15.7:0/109243304 wait complete. 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- start start 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 Processor -- start 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0bb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981a88d0 con 0x55b43d0aa2f0 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981a9d40 con 0x55b43d0aace0 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.790+0000 7f6fa0435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981a8fa0 con 0x55b43d0afe50 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6fa0435740 10 client.0 _ll_register_callbacks cb 0x7f6f80009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0bb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0bb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49816/0 (socket says 172.21.15.7:49816) 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 -- 172.21.15.7:0/726054458 learned_addr learned my addr 172.21.15.7:0/726054458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f977fe640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 -- 172.21.15.7:0/726054458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 msgr2=0x55b43d0b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 -- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 msgr2=0x55b43d0bbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 -- 172.21.15.7:0/726054458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43d0b5030 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f977fe640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f96ffd640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0bb5a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6f880bcaf0 tx=0x7f6f880c5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc92588ea5d582d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f76ffd640 1 -- 172.21.15.7:0/726054458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880c0f40 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f76ffd640 1 -- 172.21.15.7:0/726054458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6f88003b10 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f88088010 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f76ffd640 1 -- 172.21.15.7:0/726054458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f880bd4f0 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6fa0435740 10 client.17874 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b43d0a9f30 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.791+0000 7f6f76ffd640 1 -- 172.21.15.7:0/726054458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f88088010 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6f76ffd640 1 -- 172.21.15.7:0/726054458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6f8803c030 con 0x55b43d0aa2f0 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 10 client.17874 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 10 client.17874 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 20 client.17874 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d127ba0 0x55b43d129fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 4 client.17874 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 --> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b43d01b9f0 con 0x55b43d127ba0 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d127ba0 0x55b43d129fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6f76ffd640 10 client.17874 ms_handle_connect on v2:172.21.15.44:6836/3379136750 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6f8ffff640 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d127ba0 0x55b43d129fc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c6dcb9ea6f100d7 server_cookie=1a01587834a998a9 in_seq=0 out_seq=0 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6f76ffd640 1 -- 172.21.15.7:0/726054458 <== mds.1 v2:172.21.15.44:6836/3379136750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b43d01b9f0 con 0x55b43d127ba0 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6f76ffd640 10 client.17874 handle_command_reply: tid=0 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.792+0000 7f6fa0435740 1 client.17874 shutdown 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 msgr2=0x55b43d0bb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0bb5a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6f880bcaf0 tx=0x7f6f880c5660 comp rx=0 tx=0).stop 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 shutdown_connections 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:6836/3379136750,v1:172.21.15.44:6837/3379136750] conn(0x55b43d127ba0 0x55b43d129fc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b43d0afe50 0x55b43d0b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b43d0aace0 0x55b43d0bbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f76ffd640 0 client.17874 ms_handle_reset on v2:172.21.15.44:6836/3379136750 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 --2- 172.21.15.7:0/726054458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b43d0aa2f0 0x55b43d0bb5a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 >> 172.21.15.7:0/726054458 conn(0x55b43d0a5860 msgr2=0x55b43d0a9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 shutdown_connections 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 1 -- 172.21.15.7:0/726054458 wait complete. 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6fa0435740 20 client.17874 trim_cache size 0 max 16384 2024-03-19T00:49:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:49:15.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f840a2800 msgr2=0x7f6f840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f840a2800 0x7f6f840a4cc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000fc90 tx=0x7f6f9001e000 comp rx=0 tx=0).stop 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 msgr2=0x7f6f98197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 0x7f6f98197af0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6f88005940 tx=0x7f6f880042f0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 shutdown_connections 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f6f840a2800 0x7f6f840a4cc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f981033d0 0x7f6f9819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98102a10 0x7f6f98198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 --2- 172.21.15.7:0/1060746011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f98101820 0x7f6f98197af0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 >> 172.21.15.7:0/1060746011 conn(0x7f6f980fcd50 msgr2=0x7f6f98105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 shutdown_connections 2024-03-19T00:49:15.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.793+0000 7f6f9d84b640 1 -- 172.21.15.7:0/1060746011 wait complete. 2024-03-19T00:49:15.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T00:49:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 -- 172.21.15.7:0/1616957134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820107560 msgr2=0x7ff820109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1616957134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820107560 0x7ff820109950 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff818009ca0 tx=0x7ff81802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 -- 172.21.15.7:0/1616957134 shutdown_connections 2024-03-19T00:49:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1616957134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820107560 0x7ff820109950 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1616957134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff820104c30 0x7ff820107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1616957134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820102300 0x7ff8201046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 -- 172.21.15.7:0/1616957134 >> 172.21.15.7:0/1616957134 conn(0x7ff8200fbc90 msgr2=0x7ff8200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 -- 172.21.15.7:0/1616957134 shutdown_connections 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.952+0000 7ff8280ea640 1 -- 172.21.15.7:0/1616957134 wait complete. 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 Processor -- start 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 -- start start 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff820102300 0x7ff820199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 0x7ff82019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820107560 0x7ff8201a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010d350 con 0x7ff820104c30 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010d1d0 con 0x7ff820107560 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff8280ea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010d4d0 con 0x7ff820102300 2024-03-19T00:49:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 0x7ff82019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 0x7ff82019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49834/0 (socket says 172.21.15.7:49834) 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff825685640 1 -- 172.21.15.7:0/1214848813 learned_addr learned my addr 172.21.15.7:0/1214848813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff826687640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820107560 0x7ff8201a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.953+0000 7ff825e86640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff820102300 0x7ff820199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff825685640 1 -- 172.21.15.7:0/1214848813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff820102300 msgr2=0x7ff820199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff825685640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff820102300 0x7ff820199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff825685640 1 -- 172.21.15.7:0/1214848813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820107560 msgr2=0x7ff8201a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff825685640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820107560 0x7ff8201a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff825685640 1 -- 172.21.15.7:0/1214848813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8201a1850 con 0x7ff820104c30 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff826687640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820107560 0x7ff8201a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff825685640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 0x7ff82019a180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff81000e9c0 tx=0x7ff81000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff816ffd640 1 -- 172.21.15.7:0/1214848813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81000cd80 con 0x7ff820104c30 2024-03-19T00:49:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8201a1b40 con 0x7ff820104c30 2024-03-19T00:49:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff816ffd640 1 -- 172.21.15.7:0/1214848813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8100054d0 con 0x7ff820104c30 2024-03-19T00:49:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff816ffd640 1 -- 172.21.15.7:0/1214848813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810010430 con 0x7ff820104c30 2024-03-19T00:49:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.954+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201a2090 con 0x7ff820104c30 2024-03-19T00:49:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.956+0000 7ff816ffd640 1 -- 172.21.15.7:0/1214848813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7ff810004fd0 con 0x7ff820104c30 2024-03-19T00:49:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.956+0000 7ff82adb2740 1 Processor -- start 2024-03-19T00:49:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.956+0000 7ff82adb2740 1 -- start start 2024-03-19T00:49:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6576710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca6576ce0 0x55aca657b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 0x55aca657e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff82adb2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aca64da4b0 con 0x55aca657be50 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff82adb2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aca6482d70 con 0x55aca6576ce0 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff82adb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aca64d6550 con 0x55aca65762f0 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 0x55aca657e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff826687640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6576710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825e86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca6576ce0 0x55aca657b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff826687640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6576710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:53040/0 (socket says 172.21.15.7:53040) 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 0x55aca657e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49842/0 (socket says 172.21.15.7:49842) 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff826687640 1 -- 172.21.15.7:0/904450751 learned_addr learned my addr 172.21.15.7:0/904450751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff816ffd640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff80c0a2800 0x7ff80c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff826687640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff80c0a2800 0x7ff80c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825e86640 1 -- 172.21.15.7:0/904450751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 msgr2=0x55aca6576710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825e86640 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6576710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff816ffd640 1 -- 172.21.15.7:0/1214848813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff8100cf280 con 0x7ff820104c30 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825e86640 1 -- 172.21.15.7:0/904450751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 msgr2=0x55aca657e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825e86640 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 0x55aca657e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825e86640 1 -- 172.21.15.7:0/904450751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca64a5210 con 0x55aca6576ce0 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff826687640 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6576710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.957+0000 7ff825685640 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 0x55aca657e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff825e86640 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca6576ce0 0x55aca657b7e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009bf0 tx=0x7ff81c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c33e72dd64e0df0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff826687640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff80c0a2800 0x7ff80c0a4cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55aca6577540 tx=0x7ff81802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff80bfff640 1 -- 172.21.15.7:0/904450751 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c01e070 con 0x55aca6576ce0 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff80bfff640 1 -- 172.21.15.7:0/904450751 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff81c0035f0 con 0x55aca6576ce0 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff80bfff640 1 -- 172.21.15.7:0/904450751 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c019690 con 0x55aca6576ce0 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 -- 172.21.15.7:0/904450751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca6576ce0 msgr2=0x55aca657b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca6576ce0 0x55aca657b7e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009bf0 tx=0x7ff81c011810 comp rx=0 tx=0).stop 2024-03-19T00:49:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 -- 172.21.15.7:0/904450751 shutdown_connections 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca657be50 0x55aca657e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca6576ce0 0x55aca657b7e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 --2- 172.21.15.7:0/904450751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6576710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.958+0000 7ff82adb2740 1 -- 172.21.15.7:0/904450751 >> 172.21.15.7:0/904450751 conn(0x55aca6571860 msgr2=0x55aca6571c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 -- 172.21.15.7:0/904450751 shutdown_connections 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 -- 172.21.15.7:0/904450751 wait complete. 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 -- start start 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 Processor -- start 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 0x55aca6580840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca6576ce0 0x55aca6580d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6587dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aca657ee00 con 0x55aca65762f0 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55aca6496310 con 0x55aca657be50 2024-03-19T00:49:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aca657ef80 con 0x55aca6576ce0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff82adb2740 10 client.0 _ll_register_callbacks cb 0x55aca648a2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff825e86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 0x55aca6580840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff825e86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 0x55aca6580840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49854/0 (socket says 172.21.15.7:49854) 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff825e86640 1 -- 172.21.15.7:0/3881010258 learned_addr learned my addr 172.21.15.7:0/3881010258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff826687640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6587dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.959+0000 7ff825685640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca6576ce0 0x55aca6580d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff825e86640 1 -- 172.21.15.7:0/3881010258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca6576ce0 msgr2=0x55aca6580d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff825e86640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca6576ce0 0x55aca6580d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff825e86640 1 -- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 msgr2=0x55aca6587dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff825e86640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6587dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff825e86640 1 -- 172.21.15.7:0/3881010258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca61ee3f0 con 0x55aca65762f0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff825e86640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 0x55aca6580840 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009420 tx=0x7ff81c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94325b421b438aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff8097fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff8097fa640 1 -- 172.21.15.7:0/3881010258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c028020 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff8097fa640 1 -- 172.21.15.7:0/3881010258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff81c0039a0 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c0035f0 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff8097fa640 1 -- 172.21.15.7:0/3881010258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff81c033050 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff82adb2740 10 client.17886 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aca62b87d0 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.960+0000 7ff8097fa640 1 -- 172.21.15.7:0/3881010258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff81c0035f0 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff8097fa640 1 -- 172.21.15.7:0/3881010258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff81c004af0 con 0x55aca65762f0 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff82adb2740 10 client.17886 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff82adb2740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff82adb2740 20 client.17886 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff82adb2740 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca658c400 0x55aca658e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff82adb2740 4 client.17886 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca65383e0 con 0x55aca658c400 2024-03-19T00:49:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff825685640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca658c400 0x55aca658e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff8097fa640 10 client.17886 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.961+0000 7ff825685640 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca658c400 0x55aca658e850 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c333e2abfa00e270 server_cookie=1e0c6f318c7f1aee in_seq=0 out_seq=0 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff8097fa640 1 -- 172.21.15.7:0/3881010258 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aca65383e0 con 0x55aca658c400 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff8097fa640 10 client.17886 handle_command_reply: tid=0 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 client.17886 shutdown 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 msgr2=0x55aca6580840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 0x55aca6580840 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009420 tx=0x7ff81c009fd0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 shutdown_connections 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6587dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca6576ce0 0x55aca6580d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca658c400 0x55aca658e850 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 --2- 172.21.15.7:0/3881010258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca65762f0 0x55aca6580840 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff8097fa640 0 client.17886 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 >> 172.21.15.7:0/3881010258 conn(0x55aca6571860 msgr2=0x55aca6570ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 shutdown_connections 2024-03-19T00:49:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 1 -- 172.21.15.7:0/3881010258 wait complete. 2024-03-19T00:49:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.962+0000 7ff82adb2740 20 client.17886 trim_cache size 0 max 16384 2024-03-19T00:49:15.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:15 smithi007 ceph-mon[62854]: pgmap v690: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:49:15.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:15 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3643847787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.975+0000 7ff82adb2740 1 Processor -- start 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.975+0000 7ff82adb2740 1 -- start start 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca657fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6587d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca65803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff82adb2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201aee40 con 0x55aca6576ce0 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff82adb2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201ac710 con 0x55aca657be50 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff82adb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201aa560 con 0x55aca65762f0 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6587d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6587d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49862/0 (socket says 172.21.15.7:49862) 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 -- 172.21.15.7:0/223851976 learned_addr learned my addr 172.21.15.7:0/223851976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825e86640 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca657fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff826687640 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca65803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 -- 172.21.15.7:0/223851976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 msgr2=0x55aca657fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca657fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 -- 172.21.15.7:0/223851976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 msgr2=0x55aca65803a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca65803a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 -- 172.21.15.7:0/223851976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca6547880 con 0x55aca6576ce0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff825685640 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6587d80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff8100cc9c0 tx=0x7ff810086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a1f296742fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff80b7fe640 1 -- 172.21.15.7:0/223851976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81009b3a0 con 0x55aca6576ce0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff80b7fe640 1 -- 172.21.15.7:0/223851976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff81009bf60 con 0x55aca6576ce0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.976+0000 7ff80b7fe640 1 -- 172.21.15.7:0/223851976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100999a0 con 0x55aca6576ce0 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 -- 172.21.15.7:0/223851976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 msgr2=0x55aca6587d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6587d80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff8100cc9c0 tx=0x7ff810086040 comp rx=0 tx=0).stop 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 -- 172.21.15.7:0/223851976 shutdown_connections 2024-03-19T00:49:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca65803a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6587d80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 --2- 172.21.15.7:0/223851976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca657fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 -- 172.21.15.7:0/223851976 >> 172.21.15.7:0/223851976 conn(0x55aca6571860 msgr2=0x55aca6575350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 -- 172.21.15.7:0/223851976 shutdown_connections 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 -- 172.21.15.7:0/223851976 wait complete. 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 -- start start 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.977+0000 7ff82adb2740 1 Processor -- start 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6585810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6585d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6586d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201aabd0 con 0x55aca6576ce0 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201ac040 con 0x55aca657be50 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201ab2a0 con 0x55aca65762f0 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 10 client.0 _ll_register_callbacks cb 0x7ff81c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T00:49:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6585d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6585d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49874/0 (socket says 172.21.15.7:49874) 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 -- 172.21.15.7:0/2014907031 learned_addr learned my addr 172.21.15.7:0/2014907031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825e86640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6585810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff826687640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6586d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 -- 172.21.15.7:0/2014907031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 msgr2=0x55aca6585810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6585810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 -- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 msgr2=0x55aca6586d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6586d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 -- 172.21.15.7:0/2014907031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca6587350 con 0x55aca6576ce0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff826687640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6586d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff825685640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6585d50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff8100cf1f0 tx=0x7ff810099a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7bf7ea1fb4c556 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff8097fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff8097fa640 1 -- 172.21.15.7:0/2014907031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81009b1c0 con 0x55aca6576ce0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff8097fa640 1 -- 172.21.15.7:0/2014907031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff8100cc190 con 0x55aca6576ce0 2024-03-19T00:49:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81009bf60 con 0x55aca6576ce0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff8097fa640 1 -- 172.21.15.7:0/2014907031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8100a4000 con 0x55aca6576ce0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 10 client.17892 fetch_fsmap learned FSMap version 27 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.978+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aca65f2a20 con 0x55aca6576ce0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff8097fa640 1 -- 172.21.15.7:0/2014907031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff81009bf60 con 0x55aca6576ce0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff8097fa640 1 -- 172.21.15.7:0/2014907031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff810097e40 con 0x55aca6576ce0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 10 client.17892 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 20 client.17892 populate_metadata read hostname 'smithi007' 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca65f33d0 0x55aca65f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 4 client.17892 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 --> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aca64e79f0 con 0x55aca65f33d0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff825e86640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca65f33d0 0x55aca65f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff8097fa640 10 client.17892 ms_handle_connect on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff825e86640 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca65f33d0 0x55aca65f57f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c265e58255e93578 server_cookie=a87a5809ce6d0940 in_seq=0 out_seq=0 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff8097fa640 1 -- 172.21.15.7:0/2014907031 <== mds.0 v2:172.21.15.44:6838/617428839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aca64e79f0 con 0x55aca65f33d0 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff8097fa640 10 client.17892 handle_command_reply: tid=0 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.979+0000 7ff82adb2740 1 client.17892 shutdown 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 msgr2=0x55aca6585d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6585d50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff8100cf1f0 tx=0x7ff810099a20 comp rx=0 tx=0).stop 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 shutdown_connections 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55aca657be50 0x55aca6586d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.44:6838/617428839,v1:172.21.15.44:6839/617428839] conn(0x55aca65f33d0 0x55aca65f57f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aca6576ce0 0x55aca6585d50 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 --2- 172.21.15.7:0/2014907031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aca65762f0 0x55aca6585810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8097fa640 0 client.17892 ms_handle_reset on v2:172.21.15.44:6838/617428839 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 >> 172.21.15.7:0/2014907031 conn(0x55aca6571860 msgr2=0x55aca6570310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 shutdown_connections 2024-03-19T00:49:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 1 -- 172.21.15.7:0/2014907031 wait complete. 2024-03-19T00:49:15.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff82adb2740 20 client.17892 trim_cache size 0 max 16384 2024-03-19T00:49:15.988 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff80c0a2800 msgr2=0x7ff80c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff80c0a2800 0x7ff80c0a4cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55aca6577540 tx=0x7ff81802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 msgr2=0x7ff82019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 0x7ff82019a180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff81000e9c0 tx=0x7ff81000ee80 comp rx=0 tx=0).stop 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 shutdown_connections 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7ff80c0a2800 0x7ff80c0a4cc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff820107560 0x7ff8201a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff820104c30 0x7ff82019a180 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 --2- 172.21.15.7:0/1214848813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff820102300 0x7ff820199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 >> 172.21.15.7:0/1214848813 conn(0x7ff8200fbc90 msgr2=0x7ff820101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 shutdown_connections 2024-03-19T00:49:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:15.980+0000 7ff8280ea640 1 -- 172.21.15.7:0/1214848813 wait complete. 2024-03-19T00:49:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:15 smithi204 ceph-mon[66572]: pgmap v690: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:49:16.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:15 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3643847787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:49:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:15 smithi044 ceph-mon[66366]: pgmap v690: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-03-19T00:49:16.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:15 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3643847787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:49:16.993 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-19T00:49:16.993 INFO:tasks.fwd_scrub:done joining 2024-03-19T00:49:16.993 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-19T00:49:17.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:49:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 -- 172.21.15.7:0/2234317683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24102400 msgr2=0x7fac241047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 --2- 172.21.15.7:0/2234317683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24102400 0x7fac241047f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fac20009900 tx=0x7fac2002bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 -- 172.21.15.7:0/2234317683 shutdown_connections 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 --2- 172.21.15.7:0/2234317683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac24107660 0x7fac24109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 --2- 172.21.15.7:0/2234317683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac24104d30 0x7fac24107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 --2- 172.21.15.7:0/2234317683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24102400 0x7fac241047f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 -- 172.21.15.7:0/2234317683 >> 172.21.15.7:0/2234317683 conn(0x7fac240fbd50 msgr2=0x7fac240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.118+0000 7fac2c580640 1 -- 172.21.15.7:0/2234317683 shutdown_connections 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 -- 172.21.15.7:0/2234317683 wait complete. 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 Processor -- start 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 -- start start 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac24102400 0x7fac24197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 0x7fac24197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac24107660 0x7fac2419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410d380 con 0x7fac24104d30 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410d200 con 0x7fac24107660 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2c580640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410d500 con 0x7fac24102400 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac29b1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 0x7fac24197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac29b1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 0x7fac24197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49890/0 (socket says 172.21.15.7:49890) 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac29b1b640 1 -- 172.21.15.7:0/891625083 learned_addr learned my addr 172.21.15.7:0/891625083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.119+0000 7fac2ab1d640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac24107660 0x7fac2419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac2a31c640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac24102400 0x7fac24197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac29b1b640 1 -- 172.21.15.7:0/891625083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac24102400 msgr2=0x7fac24197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac29b1b640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac24102400 0x7fac24197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac29b1b640 1 -- 172.21.15.7:0/891625083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac24107660 msgr2=0x7fac2419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac29b1b640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac24107660 0x7fac2419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac29b1b640 1 -- 172.21.15.7:0/891625083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2419f6d0 con 0x7fac24104d30 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac29b1b640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 0x7fac24197ff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e9a0 tx=0x7fac1400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1400cd80 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac140035f0 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2419f9c0 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14010690 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.120+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2419ff10 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.121+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac240fd840 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.122+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fac140037a0 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.123+0000 7fac1b7fe640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fac100a2800 0x7fac100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.123+0000 7fac2a31c640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fac100a2800 0x7fac100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.123+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fac140cf0a0 con 0x7fac24104d30 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.124+0000 7fac2a31c640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fac100a2800 0x7fac100a4cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fac20009230 tx=0x7fac20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.127+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fac1409be70 con 0x7fac24104d30 2024-03-19T00:49:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.313+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac2402d440 con 0x7fac24104d30 2024-03-19T00:49:17.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.315+0000 7fac1b7fe640 1 -- 172.21.15.7:0/891625083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7fac1409b770 con 0x7fac24104d30 2024-03-19T00:49:17.316 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:49:17.316 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:49:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fac100a2800 msgr2=0x7fac100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fac100a2800 0x7fac100a4cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fac20009230 tx=0x7fac20005fd0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 msgr2=0x7fac24197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 0x7fac24197ff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e9a0 tx=0x7fac1400ee60 comp rx=0 tx=0).stop 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 shutdown_connections 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fac100a2800 0x7fac100a4cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac24107660 0x7fac2419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac24104d30 0x7fac24197ff0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 --2- 172.21.15.7:0/891625083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac24102400 0x7fac24197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 >> 172.21.15.7:0/891625083 conn(0x7fac240fbd50 msgr2=0x7fac24102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 shutdown_connections 2024-03-19T00:49:17.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.317+0000 7fac2c580640 1 -- 172.21.15.7:0/891625083 wait complete. 2024-03-19T00:49:17.330 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:49:17.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:49:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.434+0000 7f0612bb3640 1 -- 172.21.15.7:0/560142381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c101930 msgr2=0x7f060c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.434+0000 7f0612bb3640 1 --2- 172.21.15.7:0/560142381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c101930 0x7f060c101db0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0600009900 tx=0x7f060002bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.434+0000 7f0612bb3640 1 -- 172.21.15.7:0/560142381 shutdown_connections 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.434+0000 7f0612bb3640 1 --2- 172.21.15.7:0/560142381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c1022f0 0x7f060c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 --2- 172.21.15.7:0/560142381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c101930 0x7f060c101db0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 --2- 172.21.15.7:0/560142381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c100740 0x7f060c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- 172.21.15.7:0/560142381 >> 172.21.15.7:0/560142381 conn(0x7f060c0fbcb0 msgr2=0x7f060c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- 172.21.15.7:0/560142381 shutdown_connections 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- 172.21.15.7:0/560142381 wait complete. 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 Processor -- start 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- start start 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c100740 0x7f060c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c101930 0x7f060c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 0x7f060c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c270 con 0x7f060c1022f0 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c0f0 con 0x7f060c101930 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0612bb3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c3f0 con 0x7f060c100740 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0611150640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 0x7f060c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0611150640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 0x7f060c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49920/0 (socket says 172.21.15.7:49920) 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f0611150640 1 -- 172.21.15.7:0/2617800959 learned_addr learned my addr 172.21.15.7:0/2617800959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f060bfff640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c101930 0x7f060c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.435+0000 7f061094f640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c100740 0x7f060c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0611150640 1 -- 172.21.15.7:0/2617800959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c100740 msgr2=0x7f060c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0611150640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c100740 0x7f060c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0611150640 1 -- 172.21.15.7:0/2617800959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c101930 msgr2=0x7f060c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0611150640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c101930 0x7f060c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0611150640 1 -- 172.21.15.7:0/2617800959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c19f710 con 0x7f060c1022f0 2024-03-19T00:49:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0611150640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 0x7f060c19f0a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00e970 tx=0x7f05fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00ccc0 con 0x7f060c1022f0 2024-03-19T00:49:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05fc00ce60 con 0x7f060c1022f0 2024-03-19T00:49:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c19fa00 con 0x7f060c1022f0 2024-03-19T00:49:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc010430 con 0x7f060c1022f0 2024-03-19T00:49:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.436+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c19ff20 con 0x7f060c1022f0 2024-03-19T00:49:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.437+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c1a0210 con 0x7f060c1022f0 2024-03-19T00:49:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.438+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f05fc004f90 con 0x7f060c1022f0 2024-03-19T00:49:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.439+0000 7f0609ffb640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f40a2800 0x7f05f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.439+0000 7f061094f640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f40a2800 0x7f05f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.439+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f05fc0cf1c0 con 0x7f060c1022f0 2024-03-19T00:49:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.439+0000 7f061094f640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f40a2800 0x7f05f40a4cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f060c101790 tx=0x7f0604009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.443+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05fc09bea0 con 0x7f060c1022f0 2024-03-19T00:49:17.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.629+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f060c1a0430 con 0x7f060c1022f0 2024-03-19T00:49:17.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.629+0000 7f0609ffb640 1 -- 172.21.15.7:0/2617800959 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f05fc09b7a0 con 0x7f060c1022f0 2024-03-19T00:49:17.631 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:49:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f40a2800 msgr2=0x7f05f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f40a2800 0x7f05f40a4cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f060c101790 tx=0x7f0604009500 comp rx=0 tx=0).stop 2024-03-19T00:49:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 msgr2=0x7f060c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 0x7f060c19f0a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00e970 tx=0x7f05fc00ed70 comp rx=0 tx=0).stop 2024-03-19T00:49:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 shutdown_connections 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f05f40a2800 0x7f05f40a4cc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060c1022f0 0x7f060c19f0a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f060c101930 0x7f060c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 --2- 172.21.15.7:0/2617800959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f060c100740 0x7f060c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 >> 172.21.15.7:0/2617800959 conn(0x7f060c0fbcb0 msgr2=0x7f060c069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 shutdown_connections 2024-03-19T00:49:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.631+0000 7f0612bb3640 1 -- 172.21.15.7:0/2617800959 wait complete. 2024-03-19T00:49:17.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:49:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.748+0000 7f63ac014640 1 -- 172.21.15.7:0/2498268845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 msgr2=0x7f63a4108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.748+0000 7f63ac014640 1 --2- 172.21.15.7:0/2498268845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a4108b80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f639c009ca0 tx=0x7f639c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- 172.21.15.7:0/2498268845 shutdown_connections 2024-03-19T00:49:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 --2- 172.21.15.7:0/2498268845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a4108b80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 --2- 172.21.15.7:0/2498268845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63a4101930 0x7f63a4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 --2- 172.21.15.7:0/2498268845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a4100740 0x7f63a4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- 172.21.15.7:0/2498268845 >> 172.21.15.7:0/2498268845 conn(0x7f63a40fbcb0 msgr2=0x7f63a40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- 172.21.15.7:0/2498268845 shutdown_connections 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- 172.21.15.7:0/2498268845 wait complete. 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 Processor -- start 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- start start 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63a4100740 0x7f63a4069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a4101930 0x7f63a419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410c270 con 0x7f63a41022f0 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410c0f0 con 0x7f63a4100740 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63ac014640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410c3f0 con 0x7f63a4101930 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63aa5b1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63a9db0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63a4100740 0x7f63a4069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.749+0000 7f63aa5b1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49932/0 (socket says 172.21.15.7:49932) 2024-03-19T00:49:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 -- 172.21.15.7:0/1570594012 learned_addr learned my addr 172.21.15.7:0/1570594012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63a95af640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a4101930 0x7f63a419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 -- 172.21.15.7:0/1570594012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a4101930 msgr2=0x7f63a419c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a4101930 0x7f63a419c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 -- 172.21.15.7:0/1570594012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63a4100740 msgr2=0x7f63a4069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63a4100740 0x7f63a4069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 -- 172.21.15.7:0/1570594012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a419f640 con 0x7f63a41022f0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63aa5b1640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a419ed80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f639c02c700 tx=0x7f639c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c041020 con 0x7f63a41022f0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a419f8d0 con 0x7f63a41022f0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f639c003e20 con 0x7f63a41022f0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c039040 con 0x7f63a41022f0 2024-03-19T00:49:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.750+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a419fdc0 con 0x7f63a41022f0 2024-03-19T00:49:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.751+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a4107ae0 con 0x7f63a41022f0 2024-03-19T00:49:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.752+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f639c044020 con 0x7f63a41022f0 2024-03-19T00:49:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.753+0000 7f639affd640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f63900a2800 0x7f63900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:17.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.753+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f639c0f1350 con 0x7f63a41022f0 2024-03-19T00:49:17.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.753+0000 7f63a9db0640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f63900a2800 0x7f63900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:17.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.753+0000 7f63a9db0640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f63900a2800 0x7f63900a4cc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63a4101790 tx=0x7f63a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:17.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.757+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f639c04b020 con 0x7f63a41022f0 2024-03-19T00:49:17.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.943+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f63a402d380 con 0x7f63a41022f0 2024-03-19T00:49:17.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.943+0000 7f639affd640 1 -- 172.21.15.7:0/1570594012 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f639c0be090 con 0x7f63a41022f0 2024-03-19T00:49:17.944 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f63900a2800 msgr2=0x7f63900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f63900a2800 0x7f63900a4cc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63a4101790 tx=0x7f63a0005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 msgr2=0x7f63a419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a419ed80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f639c02c700 tx=0x7f639c0328a0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 shutdown_connections 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f63900a2800 0x7f63900a4cc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63a41022f0 0x7f63a419ed80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a4101930 0x7f63a419c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 --2- 172.21.15.7:0/1570594012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63a4100740 0x7f63a4069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 >> 172.21.15.7:0/1570594012 conn(0x7f63a40fbcb0 msgr2=0x7f63a4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.945+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 shutdown_connections 2024-03-19T00:49:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:17.946+0000 7f63ac014640 1 -- 172.21.15.7:0/1570594012 wait complete. 2024-03-19T00:49:17.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- 172.21.15.7:0/1300243598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 msgr2=0x7f0ef0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/1300243598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0100b60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009900 tx=0x7f0eec02bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- 172.21.15.7:0/1300243598 shutdown_connections 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/1300243598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ef01022f0 0x7f0ef0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/1300243598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef0101930 0x7f0ef0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/1300243598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0100b60 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- 172.21.15.7:0/1300243598 >> 172.21.15.7:0/1300243598 conn(0x7f0ef00fbcb0 msgr2=0x7f0ef00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- 172.21.15.7:0/1300243598 shutdown_connections 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- 172.21.15.7:0/1300243598 wait complete. 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 Processor -- start 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- start start 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ef0101930 0x7f0ef0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef01022f0 0x7f0ef019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010c270 con 0x7f0ef0100740 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010c0f0 con 0x7f0ef01022f0 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.062+0000 7f0ef76c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010c3f0 con 0x7f0ef0101930 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49948/0 (socket says 172.21.15.7:49948) 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 -- 172.21.15.7:0/521336113 learned_addr learned my addr 172.21.15.7:0/521336113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef4c5b640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ef0101930 0x7f0ef0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef5c5d640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef01022f0 0x7f0ef019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 -- 172.21.15.7:0/521336113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ef0101930 msgr2=0x7f0ef0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ef0101930 0x7f0ef0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 -- 172.21.15.7:0/521336113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef01022f0 msgr2=0x7f0ef019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef01022f0 0x7f0ef019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 -- 172.21.15.7:0/521336113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019f690 con 0x7f0ef0100740 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef545c640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0197a70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0eec000c00 tx=0x7f0eec005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec043020 con 0x7f0ef0100740 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec004350 con 0x7f0ef0100740 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec003b80 con 0x7f0ef0100740 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef019f920 con 0x7f0ef0100740 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.063+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef019fec0 con 0x7f0ef0100740 2024-03-19T00:49:18.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.064+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef01a0090 con 0x7f0ef0100740 2024-03-19T00:49:18.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.065+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0eec046020 con 0x7f0ef0100740 2024-03-19T00:49:18.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.066+0000 7f0ee67fc640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0edc0a2880 0x7f0edc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.066+0000 7f0ef4c5b640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0edc0a2880 0x7f0edc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.066+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0eec0f0e80 con 0x7f0ef0100740 2024-03-19T00:49:18.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.066+0000 7f0ef4c5b640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0edc0a2880 0x7f0edc0a4d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0198f80 tx=0x7f0ee0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.070+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0eec04d020 con 0x7f0ef0100740 2024-03-19T00:49:18.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:17 smithi007 ceph-mon[62854]: pgmap v691: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:49:18.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:17 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/891625083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:18.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:17 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2617800959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:18.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.255+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0ef01a0260 con 0x7f0ef0100740 2024-03-19T00:49:18.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.256+0000 7f0ee67fc640 1 -- 172.21.15.7:0/521336113 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f0eec0bdbc0 con 0x7f0ef0100740 2024-03-19T00:49:18.256 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:49:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0edc0a2880 msgr2=0x7f0edc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0edc0a2880 0x7f0edc0a4d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0198f80 tx=0x7f0ee0005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 msgr2=0x7f0ef0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0197a70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0eec000c00 tx=0x7f0eec005dc0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 shutdown_connections 2024-03-19T00:49:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0edc0a2880 0x7f0edc0a4d40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ef01022f0 0x7f0ef019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ef0101930 0x7f0ef0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 --2- 172.21.15.7:0/521336113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef0100740 0x7f0ef0197a70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 >> 172.21.15.7:0/521336113 conn(0x7f0ef00fbcb0 msgr2=0x7f0ef0069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 shutdown_connections 2024-03-19T00:49:18.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.258+0000 7f0ef76c0640 1 -- 172.21.15.7:0/521336113 wait complete. 2024-03-19T00:49:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:17 smithi204 ceph-mon[66572]: pgmap v691: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:49:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:17 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/891625083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:18.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:17 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2617800959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:18.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:49:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:17 smithi044 ceph-mon[66366]: pgmap v691: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-03-19T00:49:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:17 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/891625083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:18.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:17 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2617800959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 -- 172.21.15.7:0/3788927934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0100740 msgr2=0x7fc4a0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 --2- 172.21.15.7:0/3788927934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0100740 0x7fc4a0100b60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc490009900 tx=0x7fc49002bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 -- 172.21.15.7:0/3788927934 shutdown_connections 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 --2- 172.21.15.7:0/3788927934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a01022f0 0x7fc4a0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 --2- 172.21.15.7:0/3788927934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0101930 0x7fc4a0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 --2- 172.21.15.7:0/3788927934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0100740 0x7fc4a0100b60 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 -- 172.21.15.7:0/3788927934 >> 172.21.15.7:0/3788927934 conn(0x7fc4a00fbcb0 msgr2=0x7fc4a00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 -- 172.21.15.7:0/3788927934 shutdown_connections 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 -- 172.21.15.7:0/3788927934 wait complete. 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 Processor -- start 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.375+0000 7fc4a6776640 1 -- start start 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a6776640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0100740 0x7fc4a01969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a6776640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 0x7fc4a0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a6776640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a01022f0 0x7fc4a019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a6776640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a010c270 con 0x7fc4a0101930 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a6776640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a010c0f0 con 0x7fc4a0100740 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a6776640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a010c3f0 con 0x7fc4a01022f0 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 0x7fc4a0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 0x7fc4a0196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49974/0 (socket says 172.21.15.7:49974) 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 -- 172.21.15.7:0/2312873451 learned_addr learned my addr 172.21.15.7:0/2312873451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49ffff640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0100740 0x7fc4a01969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc4a4d13640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a01022f0 0x7fc4a019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 -- 172.21.15.7:0/2312873451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a01022f0 msgr2=0x7fc4a019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a01022f0 0x7fc4a019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 -- 172.21.15.7:0/2312873451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0100740 msgr2=0x7fc4a01969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0100740 0x7fc4a01969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 -- 172.21.15.7:0/2312873451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a019e5e0 con 0x7fc4a0101930 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49ffff640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0100740 0x7fc4a01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.376+0000 7fc49f7fe640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 0x7fc4a0196f00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc48800ea60 tx=0x7fc48800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.377+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48800cde0 con 0x7fc4a0101930 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.377+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a019e8d0 con 0x7fc4a0101930 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.377+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4880035f0 con 0x7fc4a0101930 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.377+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488010690 con 0x7fc4a0101930 2024-03-19T00:49:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.377+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a019ee20 con 0x7fc4a0101930 2024-03-19T00:49:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.378+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a0107b30 con 0x7fc4a0101930 2024-03-19T00:49:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.379+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fc4880037a0 con 0x7fc4a0101930 2024-03-19T00:49:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.379+0000 7fc49d7fa640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc48c0a2800 0x7fc48c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.379+0000 7fc49ffff640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc48c0a2800 0x7fc48c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.379+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc4880cf050 con 0x7fc4a0101930 2024-03-19T00:49:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.380+0000 7fc49ffff640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc48c0a2800 0x7fc48c0a4cc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc490005ab0 tx=0x7fc490005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.384+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc48809bd90 con 0x7fc4a0101930 2024-03-19T00:49:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.569+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc4a0062420 con 0x7fc4a0101930 2024-03-19T00:49:18.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.569+0000 7fc49d7fa640 1 -- 172.21.15.7:0/2312873451 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fc48809b690 con 0x7fc4a0101930 2024-03-19T00:49:18.570 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:49:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc48c0a2800 msgr2=0x7fc48c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc48c0a2800 0x7fc48c0a4cc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc490005ab0 tx=0x7fc490005fd0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 msgr2=0x7fc4a0196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 0x7fc4a0196f00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc48800ea60 tx=0x7fc48800ef20 comp rx=0 tx=0).stop 2024-03-19T00:49:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 shutdown_connections 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fc48c0a2800 0x7fc48c0a4cc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a01022f0 0x7fc4a019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4a0101930 0x7fc4a0196f00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 --2- 172.21.15.7:0/2312873451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a0100740 0x7fc4a01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 >> 172.21.15.7:0/2312873451 conn(0x7fc4a00fbcb0 msgr2=0x7fc4a0104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 shutdown_connections 2024-03-19T00:49:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.571+0000 7fc4a6776640 1 -- 172.21.15.7:0/2312873451 wait complete. 2024-03-19T00:49:18.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-19T00:49:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 -- 172.21.15.7:0/238109152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c102aa0 msgr2=0x7f614c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 --2- 172.21.15.7:0/238109152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c102aa0 0x7f614c102f20 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6134009900 tx=0x7f613402bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 -- 172.21.15.7:0/238109152 shutdown_connections 2024-03-19T00:49:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 --2- 172.21.15.7:0/238109152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614c103460 0x7f614c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 --2- 172.21.15.7:0/238109152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c102aa0 0x7f614c102f20 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 --2- 172.21.15.7:0/238109152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c1018b0 0x7f614c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 -- 172.21.15.7:0/238109152 >> 172.21.15.7:0/238109152 conn(0x7f614c0fce40 msgr2=0x7f614c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 -- 172.21.15.7:0/238109152 shutdown_connections 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.685+0000 7f615144b640 1 -- 172.21.15.7:0/238109152 wait complete. 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 Processor -- start 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 -- start start 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 0x7f614c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c102aa0 0x7f614c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614c103460 0x7f614c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f614c10b200 con 0x7f614c1018b0 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f614c10b080 con 0x7f614c102aa0 2024-03-19T00:49:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f615144b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f614c10b380 con 0x7f614c103460 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f614affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 0x7f614c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f614affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 0x7f614c195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49996/0 (socket says 172.21.15.7:49996) 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f614affd640 1 -- 172.21.15.7:0/1533514308 learned_addr learned my addr 172.21.15.7:0/1533514308 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f61427fc640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c102aa0 0x7f614c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.686+0000 7f614b7fe640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614c103460 0x7f614c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f614affd640 1 -- 172.21.15.7:0/1533514308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614c103460 msgr2=0x7f614c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f614affd640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614c103460 0x7f614c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f614affd640 1 -- 172.21.15.7:0/1533514308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c102aa0 msgr2=0x7f614c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f614affd640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c102aa0 0x7f614c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f614affd640 1 -- 172.21.15.7:0/1533514308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c19d440 con 0x7f614c1018b0 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f61427fc640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c102aa0 0x7f614c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f614affd640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 0x7f614c195820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f613c009be0 tx=0x7f613c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c007c40 con 0x7f614c1018b0 2024-03-19T00:49:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f613c007de0 con 0x7f614c1018b0 2024-03-19T00:49:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c19d730 con 0x7f614c1018b0 2024-03-19T00:49:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c0193f0 con 0x7f614c1018b0 2024-03-19T00:49:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.687+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c19dc50 con 0x7f614c1018b0 2024-03-19T00:49:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.688+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c108bd0 con 0x7f614c1018b0 2024-03-19T00:49:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.689+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f613c020030 con 0x7f614c1018b0 2024-03-19T00:49:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.690+0000 7f6148ff9640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61380a2800 0x7f61380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.690+0000 7f61427fc640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61380a2800 0x7f61380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.690+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f613c0d2600 con 0x7f614c1018b0 2024-03-19T00:49:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.690+0000 7f61427fc640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61380a2800 0x7f61380a4cc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6134000c00 tx=0x7f613402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:18.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.693+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f613c027060 con 0x7f614c1018b0 2024-03-19T00:49:18.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.928+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f614c062420 con 0x7f614c1018b0 2024-03-19T00:49:18.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.937+0000 7f6148ff9640 1 -- 172.21.15.7:0/1533514308 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+739822 (secure 0 0 0) 0x7f613c016070 con 0x7f614c1018b0 2024-03-19T00:49:18.948 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:49:18.948 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 20, 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 14152, 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1790452807 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-03-19T00:49:18.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1790452807 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/1790452807", 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-03-19T00:28:55.916832+0000", 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24109, 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-19T00:49:18.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T00:49:18.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:18.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:18.955 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-19T00:49:18.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-19T00:49:18.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-19T00:49:18.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:18.964 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T00:49:18.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-19T00:49:18.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:18.976 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:18.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-19T00:49:18.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:18.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T00:49:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.983 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:18.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T00:49:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-19T00:49:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-19T00:49:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-19T00:49:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T00:49:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-19T00:49:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-19T00:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-19T00:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-19T00:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-19T00:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-19T00:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-19T00:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-19T00:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-19T00:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T00:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T00:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-19T00:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-19T00:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-19T00:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-19T00:49:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-19T00:49:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-19T00:49:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T00:49:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-19T00:49:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T00:49:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T00:49:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T00:49:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T00:49:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-19T00:49:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-19T00:49:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.114 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-19T00:49:19.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.117 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.119 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.122 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-19T00:49:19.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.130 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.131 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-19T00:49:19.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-19T00:49:19.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.146 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-19T00:49:19.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T00:49:19.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T00:49:19.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T00:49:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.157 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.160 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.162 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.164 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.169 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.171 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.173 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.176 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-19T00:49:19.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-19T00:49:19.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-19T00:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.206 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.207 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.218 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.220 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-19T00:49:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-19T00:49:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-19T00:49:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-19T00:49:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-19T00:49:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T00:49:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-19T00:49:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-19T00:49:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-19T00:49:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-19T00:49:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.293 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.303 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.306 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.314 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.316 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.319 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-19T00:49:19.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.325 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T00:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.367 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24110, 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-03-19T00:49:19.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.372 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-19T00:49:19.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-19T00:49:19.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.393 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T00:49:19.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T00:49:19.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-19T00:49:19.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-19T00:49:19.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-19T00:49:19.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.408 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T00:49:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.421 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.423 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-19T00:49:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.428 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.437 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.438 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T00:49:19.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.454 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-19T00:49:19.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.461 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.462 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.464 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.469 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T00:49:19.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.478 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.485 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.487 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T00:49:19.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T00:49:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T00:49:19.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.494 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-19T00:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-19T00:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-19T00:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-19T00:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-19T00:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-19T00:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-19T00:49:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T00:49:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-19T00:49:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-19T00:49:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-19T00:49:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T00:49:19.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.668 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.671 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-19T00:49:19.675 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.678 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.688 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.693 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.697 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.699 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.700 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.708 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-19T00:49:19.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-19T00:49:19.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-19T00:49:19.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.713 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.719 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.724 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.726 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.727 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.729 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.733 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.734 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.743 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-19T00:49:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-19T00:49:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-19T00:49:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-19T00:49:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-19T00:49:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "restful" 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-19T00:49:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-19T00:49:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T00:49:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-19T00:49:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-19T00:49:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.842 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T00:49:19.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T00:49:19.848 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T00:49:19.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.853 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-19T00:49:19.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-19T00:49:19.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-19T00:49:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.859 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.861 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T00:49:19.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-19T00:49:19.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-19T00:49:19.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-19T00:49:19.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.872 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-19T00:49:19.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-19T00:49:19.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.885 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.886 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.888 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-19T00:49:19.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.890 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T00:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-19T00:49:19.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T00:49:19.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.897 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.898 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.899 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.908 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T00:49:19.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T00:49:19.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T00:49:19.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.916 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T00:49:19.917 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T00:49:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.920 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T00:49:19.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-19T00:49:19.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-19T00:49:19.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-19T00:49:19.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-19T00:49:19.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.951 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-19T00:49:19.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.954 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.957 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-19T00:49:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-19T00:49:19.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:19.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-19T00:49:19.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.965 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.966 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.968 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-19T00:49:19.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:19.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-19T00:49:19.973 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.975 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:19.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.980 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-19T00:49:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-19T00:49:19.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-19T00:49:19.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.988 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.993 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:19.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:19.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.003 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-19T00:49:20.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-19T00:49:20.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.014 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.021 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.023 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.024 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.027 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.032 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-19T00:49:20.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.038 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.042 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.050 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-19T00:49:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-19T00:49:20.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.063 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.089 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.093 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.105 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.108 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-19T00:49:20.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-19T00:49:20.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-19T00:49:20.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-19T00:49:20.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-19T00:49:20.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.128 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.129 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.131 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.133 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.138 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.139 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.141 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.142 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.145 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.149 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-19T00:49:20.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-19T00:49:20.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.175 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.177 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.179 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.180 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-19T00:49:20.190 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-19T00:49:20.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-19T00:49:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-19T00:49:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.194 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.197 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-19T00:49:20.202 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-19T00:49:20.203 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-19T00:49:20.204 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-19T00:49:20.205 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 3, 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-19T00:49:20.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1214490391 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 829005753 2024-03-19T00:49:20.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1833517360 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-19T00:49:20.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3984607080 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-19T00:49:20.209 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:49:20.210 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:18 smithi044 ceph-mon[66366]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:49:20.210 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:18 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1570594012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.210 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:18 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/521336113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.210 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:18 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2312873451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:18 smithi204 ceph-mon[66572]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:49:20.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:18 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1570594012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:18 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/521336113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:18 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2312873451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61380a2800 msgr2=0x7f61380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61380a2800 0x7f61380a4cc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6134000c00 tx=0x7f613402c7e0 comp rx=0 tx=0).stop 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 msgr2=0x7f614c195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 0x7f614c195820 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f613c009be0 tx=0x7f613c007590 comp rx=0 tx=0).stop 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 shutdown_connections 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61380a2800 0x7f61380a4cc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614c103460 0x7f614c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c102aa0 0x7f614c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 --2- 172.21.15.7:0/1533514308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1018b0 0x7f614c195820 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 >> 172.21.15.7:0/1533514308 conn(0x7f614c0fce40 msgr2=0x7f614c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 shutdown_connections 2024-03-19T00:49:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:18.942+0000 7f615144b640 1 -- 172.21.15.7:0/1533514308 wait complete. 2024-03-19T00:49:20.219 DEBUG:tasks.check_counter:Getting stats from a 2024-03-19T00:49:20.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-19T00:49:20.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:18 smithi007 ceph-mon[62854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T00:49:20.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1570594012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.219 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/521336113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:18 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2312873451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:20.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:19 smithi007 ceph-mon[62854]: pgmap v692: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:49:20.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:19 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1533514308' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T00:49:20.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:19 smithi204 ceph-mon[66572]: pgmap v692: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:49:20.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:19 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1533514308' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T00:49:20.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:19 smithi044 ceph-mon[66366]: pgmap v692: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-19T00:49:20.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:19 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1533514308' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T00:49:20.480 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:49:22.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:21 smithi007 ceph-mon[62854]: pgmap v693: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:49:22.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:21 smithi204 ceph-mon[66572]: pgmap v693: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:49:22.331 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:49:22.331 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:22.331 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 223, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 218, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 752096, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 133824, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.037308591, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.014908794, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.018620905, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.004546603, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 217, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.010649663, 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000049076 2024-03-19T00:49:22.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 752096, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 133824 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 5, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 167, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3200, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 107232, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.013492148, 2024-03-19T00:49:22.333 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.010177916, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001723061, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 166, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.008642404, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000052062 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-19T00:49:22.334 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 107232 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000258717, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000189587, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:22.337 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:22.338 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149024, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 31, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:22.340 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 224, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 720597, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 224, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 224, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 720597, 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-19T00:49:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:22.342 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:49:22.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:21 smithi044 ceph-mon[66366]: pgmap v693: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-03-19T00:49:23.050 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-19T00:49:23.050 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-19T00:49:23.050 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-19T00:49:23.050 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-19T00:49:23.050 DEBUG:tasks.check_counter:Getting stats from d 2024-03-19T00:49:23.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-19T00:49:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:49:24.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:23 smithi007 ceph-mon[62854]: pgmap v694: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-03-19T00:49:24.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:23 smithi204 ceph-mon[66572]: pgmap v694: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-03-19T00:49:24.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:23 smithi044 ceph-mon[66366]: pgmap v694: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 233, 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 226, 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 756768, 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 138096, 2024-03-19T00:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.038665701, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.015144107, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.019755982, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.004528710, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 226, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.012201135, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000053987 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 756768, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 138096 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 1, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 170, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 160, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 107664, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.013274099, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.010338280, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001332506, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 168, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.007121591, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000042390 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 107664 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000476119, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000406698, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-19T00:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149001, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 31, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 230, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 721377, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 230, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 230, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 721377, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:49:25.863 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-19T00:49:25.863 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-19T00:49:25.863 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-19T00:49:25.863 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-19T00:49:25.863 DEBUG:tasks.check_counter:Getting stats from g 2024-03-19T00:49:25.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-19T00:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:49:26.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:25 smithi007 ceph-mon[62854]: pgmap v695: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 14 op/s 2024-03-19T00:49:26.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:25 smithi204 ceph-mon[66572]: pgmap v695: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 14 op/s 2024-03-19T00:49:26.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:25 smithi044 ceph-mon[66366]: pgmap v695: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 14 op/s 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-19T00:49:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001135159, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000191849, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 225, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 220, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 752640, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 135104, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-19T00:49:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.036391402, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.014704387, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.019326867, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.004457961, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 219, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.011197541, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000051130 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 752640, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 135104 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 5, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 169, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3200, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 107584, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.014122868, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.010492663, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001953049, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 168, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.008976037, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000053428 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 107584 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148967, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 29, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 226, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 720857, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 226, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-19T00:49:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 226, 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 720857, 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:27.950 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:49:28.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:27 smithi007 ceph-mon[62854]: pgmap v696: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-19T00:49:28.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:27 smithi204 ceph-mon[66572]: pgmap v696: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-19T00:49:28.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:27 smithi044 ceph-mon[66366]: pgmap v696: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-19T00:49:28.625 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-19T00:49:28.625 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-19T00:49:28.625 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-19T00:49:28.625 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-19T00:49:28.625 DEBUG:tasks.check_counter:Getting stats from j 2024-03-19T00:49:28.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-19T00:49:28.872 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:49:29.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:29 smithi007 ceph-mon[62854]: pgmap v697: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-19T00:49:30.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:29 smithi204 ceph-mon[66572]: pgmap v697: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-19T00:49:30.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:29 smithi044 ceph-mon[66366]: pgmap v697: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-19T00:49:30.636 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 231, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 226, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 754272, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 138944, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.039629109, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.015372399, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.020396681, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.004741295, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 225, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.011137421, 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000049499 2024-03-19T00:49:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 754272, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 138944 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 1, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 172, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 160, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 108016, 2024-03-19T00:49:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.013801552, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.010801468, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001188269, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 170, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.009482561, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000055779 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 108016 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3040, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 448, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000660352, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000041394, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000610563, 2024-03-19T00:49:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000793886, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000264628 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-19T00:49:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148967, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 29, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-19T00:49:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 232, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 721637, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 232, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 232, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 721637, 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-19T00:49:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-19T00:49:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-19T00:49:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-19T00:49:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-19T00:49:30.647 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-19T00:49:31.369 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-19T00:49:31.369 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-19T00:49:31.369 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-19T00:49:31.369 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-19T00:49:31.369 DEBUG:tasks.check_counter:Getting stats from b 2024-03-19T00:49:31.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-19T00:49:31.638 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:49:32.022 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:31 smithi044 ceph-mon[66366]: pgmap v698: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-03-19T00:49:32.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:31 smithi007 ceph-mon[62854]: pgmap v698: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-03-19T00:49:32.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:31 smithi204 ceph-mon[66572]: pgmap v698: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-03-19T00:49:33.661 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1394, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1270, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 453090, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 2671593, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 159, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.122382233, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.027159591, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.064622175, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.021254347, 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:33.662 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1268, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.416898620, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000328784 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1589, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000284516, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000179 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 29568, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 1456 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1153, 2024-03-19T00:49:33.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 604, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 427659, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 779597, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 24, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.090980622, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.019742614, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.031926449, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.036968610, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 602, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.739057133, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001227669 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:33.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 998, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000132786, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000133 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 61056, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 492112 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1307, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1281, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1155160, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 492542, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 12, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-03-19T00:49:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.104480366, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.040902700, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.052339601, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.018934992, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1280, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.572767646, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000447474 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1320, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000235718, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000178 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 817728, 2024-03-19T00:49:33.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 166624 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 483, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.192978216, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000399540 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.667 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 23, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000449325, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000019535 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 67, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 64, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 64, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.095908829, 2024-03-19T00:49:33.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001498575 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 3, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 12, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 23, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 23, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 23, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 1, 2024-03-19T00:49:33.669 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 2, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 2, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 2, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 119, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 107, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 2, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 1, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 1, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 1, 2024-03-19T00:49:33.670 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 258, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 4, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 258, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 178, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 20, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 158, 2024-03-19T00:49:33.671 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 80, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 19, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 19, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 5, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 26, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 8, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 1, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 24 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 4, 2024-03-19T00:49:33.672 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 17, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 2, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 2, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-03-19T00:49:33.673 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 20, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 1, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 148, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-03-19T00:49:33.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 148, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4815757, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-03-19T00:49:33.675 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 131, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.214946650, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001640814 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 23, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 46, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 23, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 18, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 20, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 2, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 23, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 27, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 4, 2024-03-19T00:49:33.676 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 2, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 18, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 16, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 55312, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 231804 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 105, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 2, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 67, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 24, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 8, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5, 2024-03-19T00:49:33.677 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.005033340, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001006668 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 29, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.008083667, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000278747 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-03-19T00:49:33.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.006191166, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.006191166 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-03-19T00:49:33.679 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-03-19T00:49:33.680 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 8, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013213841, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001651730 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.682 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.025509730, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005101946 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001380004, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001380004 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-19T00:49:33.683 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.003807943, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001903971 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.012316518, 2024-03-19T00:49:33.684 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.006158259 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001032180, 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000516090 2024-03-19T00:49:33.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.686 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 9, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.019340440, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002148937 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 4, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 3, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 9, 2024-03-19T00:49:33.688 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 9, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 667, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:33.689 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:33.690 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 295909, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 107, 2024-03-19T00:49:33.691 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8696, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 166, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 94762, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 657, 2024-03-19T00:49:33.692 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 1042, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 641124, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 37, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 1042, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1042, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 18.775731676, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.018018936 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.693 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1042, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 1084 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 1042, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 300, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 742, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 11, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 22, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 133, 2024-03-19T00:49:33.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 49, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 514, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 278, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 22, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-03-19T00:49:33.695 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 5, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-03-19T00:49:33.696 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 91, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 7, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 12, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 12, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 42, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 8 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-03-19T00:49:33.697 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 1, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 1, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 2, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 2, 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-03-19T00:49:33.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 3850, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1660910, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 3850, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3850, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1660910, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.699 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 1042, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 641326, 2024-03-19T00:49:33.700 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1042, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 641326, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:33.701 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 1042, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 1042, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1042, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1042, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:33.702 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 202, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 202, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55eaef2d43c0": { 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:33.703 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 148, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 621453, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 148, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 131, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 621453, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:33.704 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-19T00:49:34.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:33 smithi007 ceph-mon[62854]: pgmap v699: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-19T00:49:34.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:33 smithi204 ceph-mon[66572]: pgmap v699: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-19T00:49:34.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:33 smithi044 ceph-mon[66366]: pgmap v699: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-19T00:49:34.490 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-03-19T00:49:34.490 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-03-19T00:49:34.490 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=4 2024-03-19T00:49:34.491 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-19T00:49:34.491 DEBUG:tasks.check_counter:Getting stats from e 2024-03-19T00:49:34.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-19T00:49:34.763 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:49:36.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:35 smithi007 ceph-mon[62854]: pgmap v700: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-19T00:49:36.234 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:35 smithi044 ceph-mon[66366]: pgmap v700: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-19T00:49:36.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:35 smithi204 ceph-mon[66572]: pgmap v700: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-19T00:49:36.644 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-19T00:49:36.644 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:36.644 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2288, 2024-03-19T00:49:36.644 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2290, 2024-03-19T00:49:36.644 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 649357, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 727591, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 6, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.127289315, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.037529985, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.032159352, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.048983369, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2288, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.040721707, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000017797 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.645 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 248, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 237, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 790236, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 141274, 2024-03-19T00:49:36.646 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.027396786, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.008399127, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.019350633, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.003953121, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 237, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.007068412, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000029824 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:36.647 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 789680, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 140576 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 173, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 428, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 374558, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:36.648 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.006975167, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.004859141, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001080835, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000017721, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 171, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.005655394, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000033072 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.649 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 374240 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-03-19T00:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3875, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.075589493, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000019506 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.651 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-03-19T00:49:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-03-19T00:49:36.653 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 1, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-03-19T00:49:36.654 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T00:49:36.655 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.656 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-03-19T00:49:36.657 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-03-19T00:49:36.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-03-19T00:49:36.659 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4196450, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 4196309, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.660 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 7 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 3, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 3, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 0, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 1, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 1, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 0, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 0, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 0, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 0, 2024-03-19T00:49:36.661 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 50088, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 215420 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-03-19T00:49:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.664 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.666 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.670 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.671 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.673 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.674 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 3, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2024-03-19T00:49:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 6104, 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-19T00:49:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 181321, 2024-03-19T00:49:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 47, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8696, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 166, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 8464, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 13, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-03-19T00:49:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 2291, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 2291, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2291, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 1.032985445, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000450888 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2291, 2024-03-19T00:49:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2292 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 2291, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 2291, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 1490, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 799, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-03-19T00:49:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-03-19T00:49:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-03-19T00:49:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 91, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 2, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 4, 2024-03-19T00:49:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 4, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-03-19T00:49:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2534, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1196509, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2534, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2534, 2024-03-19T00:49:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1196509, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2291, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 2151, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2291, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2151, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2291, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 2291, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2291, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2291, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x5603464606e0": { 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-19T00:49:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:36.696 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-19T00:49:37.301 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-19T00:49:37.301 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-19T00:49:37.301 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-19T00:49:37.301 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-19T00:49:37.301 DEBUG:tasks.check_counter:Getting stats from h 2024-03-19T00:49:37.301 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-19T00:49:37.567 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:49:38.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:37 smithi007 ceph-mon[62854]: pgmap v701: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-19T00:49:38.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:37 smithi204 ceph-mon[66572]: pgmap v701: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-19T00:49:38.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:37 smithi044 ceph-mon[66366]: pgmap v701: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 59, 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 65, 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 22755, 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 73772, 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 15, 2024-03-19T00:49:39.244 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.008976172, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.001035775, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.004669278, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000514726, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 59, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.015376129, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000260612 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 61, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000008129, 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000133 2024-03-19T00:49:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1936, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 1584 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 657, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 828, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 171359, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 631255, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 7, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.056553210, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.027512346, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.028396171, 2024-03-19T00:49:39.246 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.002633475, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 831, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.034153743, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000041099 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1274, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000207821, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000163 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 374336 2024-03-19T00:49:39.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 326, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 313, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 809768, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 619438, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 46, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 8, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.038721069, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.009543668, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.028972423, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.003911619, 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:39.248 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 312, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013210482, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000042341 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 55, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000005177, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000094 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 801472, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 146176 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-19T00:49:39.249 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 58, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001529150, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000026364 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-19T00:49:39.250 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000007199, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000003599 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 11, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 7, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 7, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002181878, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000311696 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-03-19T00:49:39.251 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 4, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 11, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 16, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 16, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 13, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 4, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 3, 2024-03-19T00:49:39.252 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 3, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 3, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 25, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 18, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 1, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 4, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-03-19T00:49:39.253 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 5, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T00:49:39.254 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 2 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 15, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 9, 2024-03-19T00:49:39.255 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 15, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T00:49:39.256 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 7, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-03-19T00:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4196450, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.009552793, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001592132 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-03-19T00:49:39.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 20, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 22, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 2, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 19, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 25, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 6, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 16, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 20, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 4, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 3, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 7, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 4, 2024-03-19T00:49:39.259 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 51988, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 223612 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 12, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 1, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 11, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 17, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 2, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001589417, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000317883 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000592461, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000296230 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-03-19T00:49:39.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.264 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.265 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-03-19T00:49:39.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.269 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-03-19T00:49:39.270 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 3, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 6, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 3, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 671, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:39.271 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:39.272 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 250623, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 86, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:39.273 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8696, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 166, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 81797, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 339, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 44, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 12757, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 28, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 44, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 44, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.217996863, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.004954474 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-03-19T00:49:39.275 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 44, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 67 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 44, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 44, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 10, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 6, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 8, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-03-19T00:49:39.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 6, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 20, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-03-19T00:49:39.277 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-03-19T00:49:39.278 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 91, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 2, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 9, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 9, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 16, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 1 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-03-19T00:49:39.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:39.280 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 1038, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 893117, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 1038, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 1038, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 893117, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:39.281 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 44, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 12757, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 44, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 12757, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-03-19T00:49:39.282 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 44, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 44, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 44, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 44, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.283 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55f92960e460": { 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 7, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 2146, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 7, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 6, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2146, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:39.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:39.286 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-19T00:49:40.020 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:39 smithi044 ceph-mon[66366]: pgmap v702: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:40.045 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-19T00:49:40.045 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-19T00:49:40.045 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-19T00:49:40.045 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=15 2024-03-19T00:49:40.045 DEBUG:tasks.check_counter:Getting stats from k 2024-03-19T00:49:40.045 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-19T00:49:40.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:39 smithi007 ceph-mon[62854]: pgmap v702: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:40.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:39 smithi204 ceph-mon[66572]: pgmap v702: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:40.300 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T00:49:42.058 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:41 smithi044 ceph-mon[66366]: pgmap v703: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 238, 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 231, 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 758128, 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 141312, 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:42.148 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.028521536, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.008723861, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.018334569, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.004078680, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 231, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.006951012, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000030090 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:42.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 758128, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 141312 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 174, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 160, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 108368, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.006678980, 2024-03-19T00:49:42.150 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.004648974, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000925037, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 172, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.004831895, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000028092 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 108368 2024-03-19T00:49:42.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.000420931, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000350636, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:42.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:42.155 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148967, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 29, 2024-03-19T00:49:42.156 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-19T00:49:42.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 235, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 722027, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 235, 2024-03-19T00:49:42.158 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 235, 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 722027, 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-19T00:49:42.159 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-19T00:49:42.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:41 smithi007 ceph-mon[62854]: pgmap v703: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:49:42.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:41 smithi204 ceph-mon[66572]: pgmap v703: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:49:42.806 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-19T00:49:42.806 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-19T00:49:42.806 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-19T00:49:42.806 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-19T00:49:42.806 DEBUG:tasks.check_counter:Getting stats from c 2024-03-19T00:49:42.806 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-19T00:49:43.064 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:49:44.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:43 smithi007 ceph-mon[62854]: pgmap v704: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:44.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:43 smithi204 ceph-mon[66572]: pgmap v704: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:44.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:43 smithi044 ceph-mon[66366]: pgmap v704: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:44.961 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-19T00:49:44.961 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:44.961 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1937, 2024-03-19T00:49:44.961 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1928, 2024-03-19T00:49:44.961 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1910151, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 678984, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.123491948, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.036748892, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.043025547, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.037767479, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1927, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.045205550, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000023459 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:44.962 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 786800, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 140288 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 9, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 178, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 4856, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 377642, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-03-19T00:49:44.963 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.007948109, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.004842980, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001892888, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000135976, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 177, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.011354584, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000064150 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.964 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 376304 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 801, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 803, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 197152, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 254725, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.054039275, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.018009479, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.011817695, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.019817755, 2024-03-19T00:49:44.965 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 801, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.018054681, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000022540 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.966 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4157, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.080759218, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000019427 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-19T00:49:44.967 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-03-19T00:49:44.968 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 16, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 16, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 16, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-03-19T00:49:44.969 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-03-19T00:49:44.970 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-03-19T00:49:44.971 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-03-19T00:49:44.972 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-03-19T00:49:44.973 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 148, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-03-19T00:49:44.974 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4815757, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4811456, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 148 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 14, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 17, 2024-03-19T00:49:44.975 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 3, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 9, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 11, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 2, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 16, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 18, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 2, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 53368, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215420 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-03-19T00:49:44.976 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.977 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-03-19T00:49:44.978 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.979 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.982 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.984 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.985 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-03-19T00:49:44.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-03-19T00:49:44.987 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 4, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 6052, 2024-03-19T00:49:44.988 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:44.989 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:44.990 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 189383, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 50, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:44.991 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8696, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 166, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 54050, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 257, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:44.992 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2499, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2499, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2499, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.887988084, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000355337 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.993 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2499, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2500 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2499, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2499, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1596, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 900, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-03-19T00:49:44.994 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-03-19T00:49:44.995 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-03-19T00:49:44.996 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 91, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-03-19T00:49:44.997 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T00:49:44.998 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2743, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1849791, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2743, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:44.999 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2743, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1849791, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:45.000 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2499, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 621463, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2499, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 621463, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:45.001 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2499, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2499, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2499, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2499, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:45.002 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:45.003 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55b630d4a6e0": { 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:45.004 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:45.005 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-19T00:49:45.829 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-19T00:49:45.829 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-19T00:49:45.829 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-19T00:49:45.829 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-03-19T00:49:45.829 DEBUG:tasks.check_counter:Getting stats from f 2024-03-19T00:49:45.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-19T00:49:46.104 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:45 smithi204 ceph-mon[66572]: pgmap v705: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:49:46.129 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:49:46.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:45 smithi044 ceph-mon[66366]: pgmap v705: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:49:46.465 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:45 smithi007 ceph-mon[62854]: pgmap v705: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1414, 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1414, 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2438622, 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448892, 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-03-19T00:49:47.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.077562265, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.022260731, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.022698500, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.025285739, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1413, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.028383938, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000020087 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-19T00:49:47.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 714, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 886, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 175580, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 602771, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.052911394, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016602265, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.012779280, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.018690693, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 884, 2024-03-19T00:49:47.922 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.023248699, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000026299 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 376032 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1045, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1036, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1056320, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 396126, 2024-03-19T00:49:47.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.082240758, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.024344457, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.033268276, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.022942288, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1035, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.034681643, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000033508 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 787184, 2024-03-19T00:49:47.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 141408 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4986, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.084653685, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000016978 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-19T00:49:47.925 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-03-19T00:49:47.926 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 2, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 2, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 2, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-03-19T00:49:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-03-19T00:49:47.928 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-03-19T00:49:47.929 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-03-19T00:49:47.930 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-03-19T00:49:47.931 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 1874, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 3, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-03-19T00:49:47.932 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 6269523, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 6267872, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 1874 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 4, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 13, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 9, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 2, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 11, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 9, 2024-03-19T00:49:47.933 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 2, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 11, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 9, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 50124, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215420 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-03-19T00:49:47.934 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.935 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-03-19T00:49:47.936 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-03-19T00:49:47.937 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-03-19T00:49:47.938 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.939 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.940 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.941 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.942 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-03-19T00:49:47.943 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-03-19T00:49:47.944 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2024-03-19T00:49:47.945 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 6054, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:47.946 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:47.947 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 189383, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 50, 2024-03-19T00:49:47.948 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8696, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 166, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-19T00:49:47.949 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 13392, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 36, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2924, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2924, 2024-03-19T00:49:47.950 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2924, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.378981552, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000471607 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2924, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2925 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2924, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2924, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-03-19T00:49:47.951 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1512, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1410, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-03-19T00:49:47.952 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-03-19T00:49:47.953 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 91, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-03-19T00:49:47.954 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-03-19T00:49:47.955 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 3169, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3369756, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:47.956 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 3169, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3169, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3369756, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:47.957 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2924, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2075224, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2924, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2075224, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:47.958 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2924, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2924, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2924, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2924, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.959 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.960 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5574bfe4a6e0": { 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-19T00:49:47.961 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:47.962 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:47.962 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:47.962 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:47.962 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.962 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:47.962 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-19T00:49:48.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:47 smithi204 ceph-mon[66572]: pgmap v706: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:49:48.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:47 smithi044 ceph-mon[66366]: pgmap v706: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:49:48.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:47 smithi007 ceph-mon[62854]: pgmap v706: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:49:48.915 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-19T00:49:48.915 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-19T00:49:48.915 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-19T00:49:48.915 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-19T00:49:48.915 DEBUG:tasks.check_counter:Getting stats from i 2024-03-19T00:49:48.916 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-19T00:49:49.176 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:49:50.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:49 smithi204 ceph-mon[66572]: pgmap v707: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:50.356 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:49 smithi044 ceph-mon[66366]: pgmap v707: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:50.464 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:49 smithi007 ceph-mon[62854]: pgmap v707: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:50.881 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-19T00:49:50.881 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:50.881 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 3072, 2024-03-19T00:49:50.881 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 6045, 2024-03-19T00:49:50.881 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 845498, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 4162144, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 11, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.226571534, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.091520183, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.095122476, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.034451805, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6044, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.093844040, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000015526 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.882 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4791, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000927131, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000193 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3968, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 1184 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 680, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 851, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 168509, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1058471, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 48, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 8, 2024-03-19T00:49:50.883 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.056556767, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.019306401, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.020476174, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.011926546, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 849, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033657301, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000039643 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 47, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000003726, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000079 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.884 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 378880 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 988, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1050, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 982980, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 477242, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 8, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.070984892, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.026314155, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.045344914, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.006263899, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1049, 2024-03-19T00:49:50.885 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033603381, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000032033 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1395, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000179418, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000128 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 801728, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 159760 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-19T00:49:50.886 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4274, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.835597464, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000195507 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 15, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000313993, 2024-03-19T00:49:50.887 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000020932 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 599, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 599, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 599, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.340349403, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003907094 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-03-19T00:49:50.888 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 16, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 1, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 15, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 13, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 4, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 611, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 605, 2024-03-19T00:49:50.889 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 3, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 1, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 1, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 1, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 18, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 5, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-03-19T00:49:50.890 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 18, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 18, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 596, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 1189, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 594, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T00:49:50.891 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 596, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 595, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 1, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 2 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 4, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 6, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 4, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 6, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 4, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 1, 2024-03-19T00:49:50.892 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 1, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 1, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-03-19T00:49:50.893 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 1874, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 1874, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 3, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 3, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-03-19T00:49:50.894 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 6269523, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1796, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.093666849, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001722531 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 20, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 23, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 3, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 19, 2024-03-19T00:49:50.895 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 21, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 2, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 16, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 22, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 6, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 2, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 2, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 54768, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223612 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 1790, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 1, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 599, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 17, 2024-03-19T00:49:50.896 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 4, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.897 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001716069, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000572023 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-03-19T00:49:50.898 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.899 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.900 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 594, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.334645993, 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003930380 2024-03-19T00:49:50.901 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000524981, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000524981 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.902 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.903 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.904 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.905 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.003462360, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003462360 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-03-19T00:49:50.906 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 87, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 87, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 684, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:50.907 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:50.908 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 275055, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 111, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:50.909 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8696, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 166, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 80677, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 331, 2024-03-19T00:49:50.910 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2501, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 2090633, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 27, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2501, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2501, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 4.274943121, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001709293 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.911 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2501, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2527 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2501, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 19, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 2482, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 1797, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 50, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-03-19T00:49:50.912 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 14, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 18, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-03-19T00:49:50.913 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 594, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-03-19T00:49:50.914 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 91, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 9, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 9, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 20, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 3 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-03-19T00:49:50.915 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 2, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 1, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 2, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 1, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:50.916 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 4736, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1552187, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 4736, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4736, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1552187, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:50.917 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2501, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2090742, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2501, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2090742, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.918 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2501, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2501, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2501, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2501, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.919 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 1, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 109, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 1, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 109, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.920 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55808947e460": { 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 1874, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2075219, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 1874, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1796, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2075219, 2024-03-19T00:49:50.921 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:50.922 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:50.922 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:50.922 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:50.922 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.922 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:50.922 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-19T00:49:51.668 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-19T00:49:51.668 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-03-19T00:49:51.668 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=4 2024-03-19T00:49:51.668 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=6 2024-03-19T00:49:51.668 DEBUG:tasks.check_counter:Getting stats from l 2024-03-19T00:49:51.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 60839b00-e587-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-19T00:49:51.921 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T00:49:52.260 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:51 smithi044 ceph-mon[66366]: pgmap v708: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:49:52.260 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:51 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:49:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:51 smithi204 ceph-mon[66572]: pgmap v708: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:49:52.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:51 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:49:52.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:51 smithi007 ceph-mon[62854]: pgmap v708: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-19T00:49:52.267 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:51 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: pgmap v709: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:49:53.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:53 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: pgmap v709: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:49:53.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:53 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: pgmap v709: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:49:53.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:49:53.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:53 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' 2024-03-19T00:49:53.857 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 0, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 0, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 0, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001150614, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000326823, 2024-03-19T00:49:53.858 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.859 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 5, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 179, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3200, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 109344, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.007083934, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.004937394, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001013178, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 178, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.008521477, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000047873 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.860 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 109344 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 238, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 233, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 756176, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 143424, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-03-19T00:49:53.861 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.028398678, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008535828, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.019360926, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004052628, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 232, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.008269333, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000035643 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 756176, 2024-03-19T00:49:53.862 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 143424 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T00:49:53.863 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-19T00:49:53.864 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 149007, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 30, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-19T00:49:53.865 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-19T00:49:53.866 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 239, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 722547, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 239, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 239, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 722547, 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-19T00:49:53.867 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-19T00:49:53.868 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-19T00:49:53.868 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-19T00:49:53.868 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:53.868 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-19T00:49:53.868 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-19T00:49:54.604 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-19T00:49:54.604 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-19T00:49:54.604 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-19T00:49:54.604 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-19T00:49:54.604 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-19T00:49:54.614 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-19T00:49:54.614 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:49:54.614 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T00:49:54.634 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:49:54.634 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T00:49:54.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 -- 172.21.15.7:0/1230632674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 msgr2=0x7fa6341070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1230632674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa6341070e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa624009900 tx=0x7fa62402bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 -- 172.21.15.7:0/1230632674 shutdown_connections 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1230632674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa634107620 0x7fa634109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1230632674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa6341070e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1230632674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6341023c0 0x7fa6341047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 -- 172.21.15.7:0/1230632674 >> 172.21.15.7:0/1230632674 conn(0x7fa6340fbd30 msgr2=0x7fa6340fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 -- 172.21.15.7:0/1230632674 shutdown_connections 2024-03-19T00:49:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.847+0000 7fa63a29a640 1 -- 172.21.15.7:0/1230632674 wait complete. 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 Processor -- start 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 -- start start 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6341023c0 0x7fa634101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa634100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa634107620 0x7fa6341008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410d380 con 0x7fa634104cf0 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410d200 con 0x7fa634107620 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.848+0000 7fa63a29a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410d500 con 0x7fa6341023c0 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa634100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa633fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6341023c0 0x7fa634101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa634100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57482/0 (socket says 172.21.15.7:57482) 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 -- 172.21.15.7:0/1305398103 learned_addr learned my addr 172.21.15.7:0/1305398103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa638837640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa634107620 0x7fa6341008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 -- 172.21.15.7:0/1305398103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6341023c0 msgr2=0x7fa634101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6341023c0 0x7fa634101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 -- 172.21.15.7:0/1305398103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa634107620 msgr2=0x7fa6341008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa634107620 0x7fa6341008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 -- 172.21.15.7:0/1305398103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa634101140 con 0x7fa634104cf0 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa633fff640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6341023c0 0x7fa634101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6337fe640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa634100370 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa624000c00 tx=0x7fa624005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624043020 con 0x7fa634104cf0 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa624004350 con 0x7fa634104cf0 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a1ba0 con 0x7fa634104cf0 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624003b80 con 0x7fa634104cf0 2024-03-19T00:49:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.849+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a20f0 con 0x7fa634104cf0 2024-03-19T00:49:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.851+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6340fd820 con 0x7fa634104cf0 2024-03-19T00:49:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.852+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fa624046020 con 0x7fa634104cf0 2024-03-19T00:49:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.852+0000 7fa6317fa640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6200a2800 0x7fa6200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.852+0000 7fa633fff640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6200a2800 0x7fa6200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.853+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa6240f0ee0 con 0x7fa634104cf0 2024-03-19T00:49:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.853+0000 7fa633fff640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6200a2800 0x7fa6200a4cc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0095f0 tx=0x7fa61c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:54.857+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa62404d020 con 0x7fa634104cf0 2024-03-19T00:49:55.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.041+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa634062420 con 0x7fa634104cf0 2024-03-19T00:49:55.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.042+0000 7fa6317fa640 1 -- 172.21.15.7:0/1305398103 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7fa6240bdc20 con 0x7fa634104cf0 2024-03-19T00:49:55.044 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:49:55.044 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6200a2800 msgr2=0x7fa6200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6200a2800 0x7fa6200a4cc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0095f0 tx=0x7fa61c005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 msgr2=0x7fa634100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa634100370 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa624000c00 tx=0x7fa624005dc0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 shutdown_connections 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fa6200a2800 0x7fa6200a4cc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa634107620 0x7fa6341008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa634104cf0 0x7fa634100370 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 --2- 172.21.15.7:0/1305398103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6341023c0 0x7fa634101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 >> 172.21.15.7:0/1305398103 conn(0x7fa6340fbd30 msgr2=0x7fa634102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 shutdown_connections 2024-03-19T00:49:55.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.045+0000 7fa63a29a640 1 -- 172.21.15.7:0/1305398103 wait complete. 2024-03-19T00:49:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:49:55.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:49:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 -- 172.21.15.7:0/1162629910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 msgr2=0x7f74bc100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 --2- 172.21.15.7:0/1162629910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 0x7f74bc100290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00aa10 tx=0x7f74ac02cf70 comp rx=0 tx=0).stop 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 -- 172.21.15.7:0/1162629910 shutdown_connections 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 --2- 172.21.15.7:0/1162629910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74bc101a10 0x7f74bc108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 --2- 172.21.15.7:0/1162629910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 --2- 172.21.15.7:0/1162629910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 0x7f74bc100290 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 -- 172.21.15.7:0/1162629910 >> 172.21.15.7:0/1162629910 conn(0x7f74bc0fb430 msgr2=0x7f74bc0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 -- 172.21.15.7:0/1162629910 shutdown_connections 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 -- 172.21.15.7:0/1162629910 wait complete. 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 Processor -- start 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.163+0000 7f74c29b3640 1 -- start start 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c29b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 0x7f74bc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c29b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c29b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74bc101a10 0x7f74bc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c29b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc10b160 con 0x7f74bc101050 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c29b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc10afe0 con 0x7f74bc101a10 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c29b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc10b2e0 con 0x7f74bc0ffe70 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57496/0 (socket says 172.21.15.7:57496) 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 -- 172.21.15.7:0/3161775649 learned_addr learned my addr 172.21.15.7:0/3161775649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74c0f50640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74bc101a10 0x7f74bc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bbfff640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 0x7f74bc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 -- 172.21.15.7:0/3161775649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 msgr2=0x7f74bc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 0x7f74bc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 -- 172.21.15.7:0/3161775649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74bc101a10 msgr2=0x7f74bc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74bc101a10 0x7f74bc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 -- 172.21.15.7:0/3161775649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc19f6c0 con 0x7f74bc101050 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.164+0000 7f74bb7fe640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc197fe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e9f0 tx=0x7f74b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.165+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b000cd80 con 0x7f74bc101050 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.165+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc19f9b0 con 0x7f74bc101050 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.165+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74b00108e0 con 0x7f74bc101050 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.165+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0018920 con 0x7f74bc101050 2024-03-19T00:49:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.165+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc19ff50 con 0x7f74bc101050 2024-03-19T00:49:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.166+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc1a01e0 con 0x7f74bc101050 2024-03-19T00:49:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.167+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f74b0010a90 con 0x7f74bc101050 2024-03-19T00:49:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.167+0000 7f74b97fa640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74a40a2800 0x7f74a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.167+0000 7f74bbfff640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74a40a2800 0x7f74a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.167+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f74b00cf550 con 0x7f74bc101050 2024-03-19T00:49:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.168+0000 7f74bbfff640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74a40a2800 0x7f74a40a4cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005ab0 tx=0x7f74ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:55.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.171+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f74b009bc40 con 0x7f74bc101050 2024-03-19T00:49:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.353+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74bc1a03b0 con 0x7f74bc101050 2024-03-19T00:49:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.353+0000 7f74b97fa640 1 -- 172.21.15.7:0/3161775649 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f74b00ce020 con 0x7f74bc101050 2024-03-19T00:49:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74a40a2800 msgr2=0x7f74a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74a40a2800 0x7f74a40a4cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005ab0 tx=0x7f74ac005fd0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 msgr2=0x7f74bc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc197fe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e9f0 tx=0x7f74b000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 shutdown_connections 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f74a40a2800 0x7f74a40a4cc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74bc101a10 0x7f74bc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc101050 0x7f74bc197fe0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 --2- 172.21.15.7:0/3161775649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74bc0ffe70 0x7f74bc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 >> 172.21.15.7:0/3161775649 conn(0x7f74bc0fb430 msgr2=0x7f74bc106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 shutdown_connections 2024-03-19T00:49:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.355+0000 7f74c29b3640 1 -- 172.21.15.7:0/3161775649 wait complete. 2024-03-19T00:49:55.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:49:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 -- 172.21.15.7:0/1093653806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454101760 msgr2=0x7fe454101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 --2- 172.21.15.7:0/1093653806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454101760 0x7fe454101be0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009900 tx=0x7fe43c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 -- 172.21.15.7:0/1093653806 shutdown_connections 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 --2- 172.21.15.7:0/1093653806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe454102120 0x7fe4541089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 --2- 172.21.15.7:0/1093653806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454101760 0x7fe454101be0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 --2- 172.21.15.7:0/1093653806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 0x7fe454100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 -- 172.21.15.7:0/1093653806 >> 172.21.15.7:0/1093653806 conn(0x7fe4540fbb00 msgr2=0x7fe4540fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 -- 172.21.15.7:0/1093653806 shutdown_connections 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.471+0000 7fe45359e640 1 -- 172.21.15.7:0/1093653806 wait complete. 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 Processor -- start 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 -- start start 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 0x7fe454069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe454101760 0x7fe45419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 0x7fe45419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe45410c0b0 con 0x7fe454102120 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe45410bf30 con 0x7fe454100570 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.472+0000 7fe45359e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe45410c230 con 0x7fe454101760 2024-03-19T00:49:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 0x7fe45419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 0x7fe45419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57506/0 (socket says 172.21.15.7:57506) 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 -- 172.21.15.7:0/3499507526 learned_addr learned my addr 172.21.15.7:0/3499507526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe45259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 0x7fe454069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe449d9b640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe454101760 0x7fe45419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 -- 172.21.15.7:0/3499507526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe454101760 msgr2=0x7fe45419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe454101760 0x7fe45419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 -- 172.21.15.7:0/3499507526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 msgr2=0x7fe454069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 0x7fe454069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 -- 172.21.15.7:0/3499507526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45419f4c0 con 0x7fe454102120 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe45259c640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 0x7fe454069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe452d9d640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 0x7fe45419ec00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00eb10 tx=0x7fe44c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c004b20 con 0x7fe454102120 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45419f7b0 con 0x7fe454102120 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe44c004cc0 con 0x7fe454102120 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c010690 con 0x7fe454102120 2024-03-19T00:49:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.473+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45419fc80 con 0x7fe454102120 2024-03-19T00:49:55.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.474+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4541079b0 con 0x7fe454102120 2024-03-19T00:49:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.475+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe44c005020 con 0x7fe454102120 2024-03-19T00:49:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.476+0000 7fe44bfff640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4400a2800 0x7fe4400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.476+0000 7fe45259c640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4400a2800 0x7fe4400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.476+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe44c0cf3b0 con 0x7fe454102120 2024-03-19T00:49:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.476+0000 7fe45259c640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4400a2800 0x7fe4400a4cc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe4541015c0 tx=0x7fe444005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.480+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe44c09c090 con 0x7fe454102120 2024-03-19T00:49:55.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.662+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe454062420 con 0x7fe454102120 2024-03-19T00:49:55.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.662+0000 7fe44bfff640 1 -- 172.21.15.7:0/3499507526 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fe44c09b990 con 0x7fe454102120 2024-03-19T00:49:55.663 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:49:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4400a2800 msgr2=0x7fe4400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4400a2800 0x7fe4400a4cc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe4541015c0 tx=0x7fe444005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 msgr2=0x7fe45419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 0x7fe45419ec00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00eb10 tx=0x7fe44c00c2d0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 shutdown_connections 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe4400a2800 0x7fe4400a4cc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.664+0000 7fe45359e640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe454102120 0x7fe45419ec00 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.665+0000 7fe45359e640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe454101760 0x7fe45419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.665+0000 7fe45359e640 1 --2- 172.21.15.7:0/3499507526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe454100570 0x7fe454069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.665+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 >> 172.21.15.7:0/3499507526 conn(0x7fe4540fbb00 msgr2=0x7fe454104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.665+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 shutdown_connections 2024-03-19T00:49:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.665+0000 7fe45359e640 1 -- 172.21.15.7:0/3499507526 wait complete. 2024-03-19T00:49:55.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:55 smithi007 ceph-mon[62854]: pgmap v710: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:49:55.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:55 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1305398103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:55.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:55 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3161775649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:55.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:49:55.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:55 smithi204 ceph-mon[66572]: pgmap v710: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:49:55.772 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:55 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1305398103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:55.772 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:55 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3161775649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:55.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 -- 172.21.15.7:0/3784918862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe914101830 msgr2=0x7fe914101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 --2- 172.21.15.7:0/3784918862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe914101830 0x7fe914101c50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009900 tx=0x7fe90c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:55.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 -- 172.21.15.7:0/3784918862 shutdown_connections 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 --2- 172.21.15.7:0/3784918862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9141033e0 0x7fe914109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 --2- 172.21.15.7:0/3784918862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102a20 0x7fe914102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 --2- 172.21.15.7:0/3784918862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe914101830 0x7fe914101c50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 -- 172.21.15.7:0/3784918862 >> 172.21.15.7:0/3784918862 conn(0x7fe9140fcda0 msgr2=0x7fe9140ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 -- 172.21.15.7:0/3784918862 shutdown_connections 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 -- 172.21.15.7:0/3784918862 wait complete. 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 Processor -- start 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.781+0000 7fe91b7df640 1 -- start start 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914101830 0x7fe914197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914102a20 0x7fe914197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 0x7fe91419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d330 con 0x7fe9141033e0 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d1b0 con 0x7fe914102a20 2024-03-19T00:49:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d4b0 con 0x7fe914101830 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 0x7fe91419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 0x7fe91419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57520/0 (socket says 172.21.15.7:57520) 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 -- 172.21.15.7:0/1867212705 learned_addr learned my addr 172.21.15.7:0/1867212705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91957b640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914101830 0x7fe914197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe918d7a640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914102a20 0x7fe914197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 -- 172.21.15.7:0/1867212705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914101830 msgr2=0x7fe914197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914101830 0x7fe914197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 -- 172.21.15.7:0/1867212705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914102a20 msgr2=0x7fe914197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914102a20 0x7fe914197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 -- 172.21.15.7:0/1867212705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91419f680 con 0x7fe9141033e0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe919d7c640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 0x7fe91419f010 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe91000eb10 tx=0x7fe91000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910004b20 con 0x7fe9141033e0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe910004cc0 con 0x7fe9141033e0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91419f970 con 0x7fe9141033e0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.782+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910010690 con 0x7fe9141033e0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.783+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91419fe70 con 0x7fe9141033e0 2024-03-19T00:49:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.783+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe914108bc0 con 0x7fe9141033e0 2024-03-19T00:49:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.785+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fe910005000 con 0x7fe9141033e0 2024-03-19T00:49:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.785+0000 7fe90a7fc640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9000a2800 0x7fe9000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.785+0000 7fe91957b640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9000a2800 0x7fe9000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.785+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe9100cf550 con 0x7fe9141033e0 2024-03-19T00:49:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.786+0000 7fe91957b640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9000a2800 0x7fe9000a4cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe90c005ab0 tx=0x7fe90c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.789+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe910053740 con 0x7fe9141033e0 2024-03-19T00:49:55.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:55 smithi044 ceph-mon[66366]: pgmap v710: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-19T00:49:55.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:55 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1305398103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:55.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:55 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3161775649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.972+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe914062420 con 0x7fe9141033e0 2024-03-19T00:49:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.973+0000 7fe90a7fc640 1 -- 172.21.15.7:0/1867212705 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fe91009c290 con 0x7fe9141033e0 2024-03-19T00:49:55.974 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9000a2800 msgr2=0x7fe9000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9000a2800 0x7fe9000a4cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe90c005ab0 tx=0x7fe90c005fd0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 msgr2=0x7fe91419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 0x7fe91419f010 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe91000eb10 tx=0x7fe91000c2d0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 shutdown_connections 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fe9000a2800 0x7fe9000a4cc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9141033e0 0x7fe91419f010 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe914102a20 0x7fe914197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 --2- 172.21.15.7:0/1867212705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914101830 0x7fe914197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 >> 172.21.15.7:0/1867212705 conn(0x7fe9140fcda0 msgr2=0x7fe914105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 shutdown_connections 2024-03-19T00:49:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:55.975+0000 7fe91b7df640 1 -- 172.21.15.7:0/1867212705 wait complete. 2024-03-19T00:49:55.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:49:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2567432008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6281022f0 msgr2=0x7fd628108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2567432008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6281022f0 0x7fd628108b80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd620009900 tx=0x7fd62002bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2567432008 shutdown_connections 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2567432008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6281022f0 0x7fd628108b80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2567432008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd628101930 0x7fd628101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2567432008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628100740 0x7fd628100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2567432008 >> 172.21.15.7:0/2567432008 conn(0x7fd6280fbcb0 msgr2=0x7fd6280fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2567432008 shutdown_connections 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.090+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2567432008 wait complete. 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 Processor -- start 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 -- start start 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 0x7fd628197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd628101930 0x7fd628197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6281022f0 0x7fd62819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810c250 con 0x7fd628100740 2024-03-19T00:49:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810c0d0 con 0x7fd6281022f0 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd62e1e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810c3d0 con 0x7fd628101930 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 0x7fd628197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 0x7fd628197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57534/0 (socket says 172.21.15.7:57534) 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd6277fe640 1 -- 172.21.15.7:0/2760158015 learned_addr learned my addr 172.21.15.7:0/2760158015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd627fff640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6281022f0 0x7fd62819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.091+0000 7fd626ffd640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd628101930 0x7fd628197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd6277fe640 1 -- 172.21.15.7:0/2760158015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd628101930 msgr2=0x7fd628197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd6277fe640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd628101930 0x7fd628197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd6277fe640 1 -- 172.21.15.7:0/2760158015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6281022f0 msgr2=0x7fd62819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd6277fe640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6281022f0 0x7fd62819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd6277fe640 1 -- 172.21.15.7:0/2760158015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62819f630 con 0x7fd628100740 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd627fff640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6281022f0 0x7fd62819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd6277fe640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 0x7fd628197a10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d20 tx=0x7fd61800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618010630 con 0x7fd628100740 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd618010c80 con 0x7fd628100740 2024-03-19T00:49:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62819f920 con 0x7fd628100740 2024-03-19T00:49:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618018d70 con 0x7fd628100740 2024-03-19T00:49:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.092+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62819fe70 con 0x7fd628100740 2024-03-19T00:49:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.093+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6281a00a0 con 0x7fd628100740 2024-03-19T00:49:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.094+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fd61801f020 con 0x7fd628100740 2024-03-19T00:49:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.095+0000 7fd624ff9640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd6100a2800 0x7fd6100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.095+0000 7fd626ffd640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd6100a2800 0x7fd6100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.095+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd6180cf780 con 0x7fd628100740 2024-03-19T00:49:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.095+0000 7fd626ffd640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd6100a2800 0x7fd6100a4cc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd628198f20 tx=0x7fd61c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:56.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.098+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd61809c460 con 0x7fd628100740 2024-03-19T00:49:56.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.285+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd6281a0430 con 0x7fd628100740 2024-03-19T00:49:56.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.286+0000 7fd624ff9640 1 -- 172.21.15.7:0/2760158015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fd61809bd60 con 0x7fd628100740 2024-03-19T00:49:56.287 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd6100a2800 msgr2=0x7fd6100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd6100a2800 0x7fd6100a4cc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd628198f20 tx=0x7fd61c005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 msgr2=0x7fd628197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 0x7fd628197a10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d20 tx=0x7fd61800c6a0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 shutdown_connections 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fd6100a2800 0x7fd6100a4cc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6281022f0 0x7fd62819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd628101930 0x7fd628197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 --2- 172.21.15.7:0/2760158015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd628100740 0x7fd628197a10 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 >> 172.21.15.7:0/2760158015 conn(0x7fd6280fbcb0 msgr2=0x7fd6280fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 shutdown_connections 2024-03-19T00:49:56.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.288+0000 7fd62e1e8640 1 -- 172.21.15.7:0/2760158015 wait complete. 2024-03-19T00:49:56.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T00:49:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 -- 172.21.15.7:0/3291698144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 msgr2=0x7f7d981099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 --2- 172.21.15.7:0/3291698144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d981099b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009860 tx=0x7f7d9002bb70 comp rx=0 tx=0).stop 2024-03-19T00:49:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 -- 172.21.15.7:0/3291698144 shutdown_connections 2024-03-19T00:49:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 --2- 172.21.15.7:0/3291698144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d981099b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 --2- 172.21.15.7:0/3291698144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 --2- 172.21.15.7:0/3291698144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d98102360 0x7f7d98104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 -- 172.21.15.7:0/3291698144 >> 172.21.15.7:0/3291698144 conn(0x7f7d980fbcb0 msgr2=0x7f7d980fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 -- 172.21.15.7:0/3291698144 shutdown_connections 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.403+0000 7f7da03f3640 1 -- 172.21.15.7:0/3291698144 wait complete. 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 Processor -- start 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 -- start start 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d98102360 0x7f7d98101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d98100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9810d390 con 0x7f7d98104c90 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9810d210 con 0x7f7d981075c0 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7da03f3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9810d510 con 0x7f7d98102360 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57548/0 (socket says 172.21.15.7:57548) 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 -- 172.21.15.7:0/2860089697 learned_addr learned my addr 172.21.15.7:0/2860089697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9e990640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d98100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9e18f640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d98102360 0x7f7d98101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 -- 172.21.15.7:0/2860089697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d98102360 msgr2=0x7f7d98101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d98102360 0x7f7d98101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 -- 172.21.15.7:0/2860089697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 msgr2=0x7f7d98100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d98100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9d98e640 1 -- 172.21.15.7:0/2860089697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d981010e0 con 0x7f7d98104c90 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.404+0000 7f7d9e990640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d98100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.405+0000 7f7d9d98e640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98100310 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800e9a0 tx=0x7f7d8800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.405+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800cd80 con 0x7f7d98104c90 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.405+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d880035f0 con 0x7f7d98104c90 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.405+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d981a1bb0 con 0x7f7d98104c90 2024-03-19T00:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.405+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88010690 con 0x7f7d98104c90 2024-03-19T00:49:56.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.405+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d981a2100 con 0x7f7d98104c90 2024-03-19T00:49:56.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.406+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d980fd7a0 con 0x7f7d98104c90 2024-03-19T00:49:56.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.407+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7d880037a0 con 0x7f7d98104c90 2024-03-19T00:49:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.407+0000 7f7d8f7fe640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d840a2800 0x7f7d840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.408+0000 7f7d9e18f640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d840a2800 0x7f7d840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.408+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7d880cf100 con 0x7f7d98104c90 2024-03-19T00:49:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.408+0000 7f7d9e18f640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d840a2800 0x7f7d840a4cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005fd0 tx=0x7f7d94009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.412+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d8809bde0 con 0x7f7d98104c90 2024-03-19T00:49:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.607+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d98062420 con 0x7f7d98104c90 2024-03-19T00:49:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.607+0000 7f7d8f7fe640 1 -- 172.21.15.7:0/2860089697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+577 (secure 0 0 0) 0x7f7d8809b6e0 con 0x7f7d98104c90 2024-03-19T00:49:56.608 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1277095143 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:56.608 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4032873653 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1320706322 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/237878025 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3660465780 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/97837638 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-03-19T00:49:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d840a2800 msgr2=0x7f7d840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d840a2800 0x7f7d840a4cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005fd0 tx=0x7f7d94009500 comp rx=0 tx=0).stop 2024-03-19T00:49:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 msgr2=0x7f7d98100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98100310 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800e9a0 tx=0x7f7d8800ee60 comp rx=0 tx=0).stop 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 shutdown_connections 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f7d840a2800 0x7f7d840a4cc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d981075c0 0x7f7d98100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d98104c90 0x7f7d98100310 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 --2- 172.21.15.7:0/2860089697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d98102360 0x7f7d98101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 >> 172.21.15.7:0/2860089697 conn(0x7f7d980fbcb0 msgr2=0x7f7d98102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 shutdown_connections 2024-03-19T00:49:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.609+0000 7f7da03f3640 1 -- 172.21.15.7:0/2860089697 wait complete. 2024-03-19T00:49:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:56 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3499507526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:56 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1867212705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:56 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2760158015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.621 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-19T00:49:56.621 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T00:49:56.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:49:56.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:56 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3499507526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:56 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1867212705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:56 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2760158015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 -- 172.21.15.7:0/2020584783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548102300 msgr2=0x7f1548108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 --2- 172.21.15.7:0/2020584783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548102300 0x7f1548108b90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1538009900 tx=0x7f153802bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:56.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 -- 172.21.15.7:0/2020584783 shutdown_connections 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 --2- 172.21.15.7:0/2020584783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548102300 0x7f1548108b90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 --2- 172.21.15.7:0/2020584783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548101940 0x7f1548101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 --2- 172.21.15.7:0/2020584783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1548100750 0x7f1548100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 -- 172.21.15.7:0/2020584783 >> 172.21.15.7:0/2020584783 conn(0x7f15480fbce0 msgr2=0x7f15480fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 -- 172.21.15.7:0/2020584783 shutdown_connections 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 -- 172.21.15.7:0/2020584783 wait complete. 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 Processor -- start 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.779+0000 7f154f005640 1 -- start start 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548100750 0x7f1548197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 0x7f1548197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1548102300 0x7f154819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f154810c280 con 0x7f1548101940 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f154810c100 con 0x7f1548102300 2024-03-19T00:49:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f154810c400 con 0x7f1548100750 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 0x7f1548197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 0x7f1548197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57558/0 (socket says 172.21.15.7:57558) 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154cda1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548100750 0x7f1548197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 -- 172.21.15.7:0/436088845 learned_addr learned my addr 172.21.15.7:0/436088845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154d5a2640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1548102300 0x7f154819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 -- 172.21.15.7:0/436088845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548100750 msgr2=0x7f1548197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548100750 0x7f1548197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 -- 172.21.15.7:0/436088845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1548102300 msgr2=0x7f154819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1548102300 0x7f154819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 -- 172.21.15.7:0/436088845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154819f670 con 0x7f1548101940 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1547fff640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 0x7f1548197f90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9f0 tx=0x7f153c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00cd80 con 0x7f1548101940 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153c0054d0 con 0x7f1548101940 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154819f960 con 0x7f1548101940 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.780+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c010490 con 0x7f1548101940 2024-03-19T00:49:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.781+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154819fe60 con 0x7f1548101940 2024-03-19T00:49:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.781+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15481a00f0 con 0x7f1548101940 2024-03-19T00:49:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.783+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f153c005000 con 0x7f1548101940 2024-03-19T00:49:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.783+0000 7f1545ffb640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15300a2800 0x7f15300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.783+0000 7f154cda1640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15300a2800 0x7f15300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.783+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f153c0cf320 con 0x7f1548101940 2024-03-19T00:49:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.784+0000 7f154cda1640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15300a2800 0x7f15300a4cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f15400099a0 tx=0x7f1540009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.787+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f153c053740 con 0x7f1548101940 2024-03-19T00:49:56.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:56 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3499507526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:56 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1867212705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:56 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2760158015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:56.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.972+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15481a02c0 con 0x7f1548101940 2024-03-19T00:49:56.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.974+0000 7f1545ffb640 1 -- 172.21.15.7:0/436088845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f153c0052f0 con 0x7f1548101940 2024-03-19T00:49:56.975 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:49:56.975 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:49:56.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15300a2800 msgr2=0x7f15300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15300a2800 0x7f15300a4cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f15400099a0 tx=0x7f1540009500 comp rx=0 tx=0).stop 2024-03-19T00:49:56.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 msgr2=0x7f1548197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 0x7f1548197f90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9f0 tx=0x7f153c00edf0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 shutdown_connections 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f15300a2800 0x7f15300a4cc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1548102300 0x7f154819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1548101940 0x7f1548197f90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 --2- 172.21.15.7:0/436088845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1548100750 0x7f1548197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 >> 172.21.15.7:0/436088845 conn(0x7f15480fbce0 msgr2=0x7f15480fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 shutdown_connections 2024-03-19T00:49:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:56.976+0000 7f154f005640 1 -- 172.21.15.7:0/436088845 wait complete. 2024-03-19T00:49:56.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T00:31:18.272582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T00:35:58.138748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T00:35:58.754565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T00:36:02.466956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T00:49:56.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 -- 172.21.15.7:0/2170552150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838101930 msgr2=0x7f7838101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 --2- 172.21.15.7:0/2170552150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838101930 0x7f7838101db0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f782802bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 -- 172.21.15.7:0/2170552150 shutdown_connections 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 --2- 172.21.15.7:0/2170552150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78381022f0 0x7f7838108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 --2- 172.21.15.7:0/2170552150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7838101930 0x7f7838101db0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 --2- 172.21.15.7:0/2170552150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7838100740 0x7f7838100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 -- 172.21.15.7:0/2170552150 >> 172.21.15.7:0/2170552150 conn(0x7f78380fbcb0 msgr2=0x7f78380fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 -- 172.21.15.7:0/2170552150 shutdown_connections 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.093+0000 7f783fd28640 1 -- 172.21.15.7:0/2170552150 wait complete. 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 Processor -- start 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 -- start start 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7838100740 0x7f7838197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838101930 0x7f7838198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 0x7f783819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d390 con 0x7f78381022f0 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d210 con 0x7f7838101930 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783fd28640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d510 con 0x7f7838100740 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783e2c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 0x7f783819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783e2c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 0x7f783819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57570/0 (socket says 172.21.15.7:57570) 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783e2c5640 1 -- 172.21.15.7:0/897521291 learned_addr learned my addr 172.21.15.7:0/897521291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783dac4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7838100740 0x7f7838197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.094+0000 7f783d2c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838101930 0x7f7838198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783e2c5640 1 -- 172.21.15.7:0/897521291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7838100740 msgr2=0x7f7838197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783e2c5640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7838100740 0x7f7838197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783e2c5640 1 -- 172.21.15.7:0/897521291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838101930 msgr2=0x7f7838198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783e2c5640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838101930 0x7f7838198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783e2c5640 1 -- 172.21.15.7:0/897521291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783819f6f0 con 0x7f78381022f0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783e2c5640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 0x7f783819f080 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f783400efd0 tx=0x7f783400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834003b70 con 0x7f78381022f0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783819f9e0 con 0x7f78381022f0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7834003d10 con 0x7f78381022f0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834010980 con 0x7f78381022f0 2024-03-19T00:49:57.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.095+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783819ff00 con 0x7f78381022f0 2024-03-19T00:49:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.096+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838107b80 con 0x7f78381022f0 2024-03-19T00:49:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.097+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7834004020 con 0x7f78381022f0 2024-03-19T00:49:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.098+0000 7f782effd640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f78240a2800 0x7f78240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.098+0000 7f783dac4640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f78240a2800 0x7f78240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.098+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f78340cf220 con 0x7f78381022f0 2024-03-19T00:49:57.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.098+0000 7f783dac4640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f78240a2800 0x7f78240a4cc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7838101790 tx=0x7f7830005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:57.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.103+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f783409bf60 con 0x7f78381022f0 2024-03-19T00:49:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.285+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7838062420 con 0x7f78381022f0 2024-03-19T00:49:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.285+0000 7f782effd640 1 -- 172.21.15.7:0/897521291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f783409b860 con 0x7f78381022f0 2024-03-19T00:49:57.286 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-19T00:49:57.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f78240a2800 msgr2=0x7f78240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f78240a2800 0x7f78240a4cc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7838101790 tx=0x7f7830005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 msgr2=0x7f783819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 0x7f783819f080 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f783400efd0 tx=0x7f783400c6a0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 shutdown_connections 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f78240a2800 0x7f78240a4cc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78381022f0 0x7f783819f080 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838101930 0x7f7838198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 --2- 172.21.15.7:0/897521291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7838100740 0x7f7838197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 >> 172.21.15.7:0/897521291 conn(0x7f78380fbcb0 msgr2=0x7f7838104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 shutdown_connections 2024-03-19T00:49:57.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.287+0000 7f783fd28640 1 -- 172.21.15.7:0/897521291 wait complete. 2024-03-19T00:49:57.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T00:49:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 -- 172.21.15.7:0/3543396816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 msgr2=0x7f2700102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 --2- 172.21.15.7:0/3543396816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 0x7f2700102e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009860 tx=0x7f26f002bb70 comp rx=0 tx=0).stop 2024-03-19T00:49:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 -- 172.21.15.7:0/3543396816 shutdown_connections 2024-03-19T00:49:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 --2- 172.21.15.7:0/3543396816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27001033d0 0x7f2700109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 --2- 172.21.15.7:0/3543396816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 0x7f2700102e90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 --2- 172.21.15.7:0/3543396816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 -- 172.21.15.7:0/3543396816 >> 172.21.15.7:0/3543396816 conn(0x7f27000fcd70 msgr2=0x7f27000ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 -- 172.21.15.7:0/3543396816 shutdown_connections 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.403+0000 7f2707737640 1 -- 172.21.15.7:0/3543396816 wait complete. 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 Processor -- start 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 -- start start 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 0x7f2700198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27001033d0 0x7f270019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f270010d350 con 0x7f2700101820 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f270010d1d0 con 0x7f2700102a10 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f270010d4d0 con 0x7f27001033d0 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57578/0 (socket says 172.21.15.7:57578) 2024-03-19T00:49:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 -- 172.21.15.7:0/3808975196 learned_addr learned my addr 172.21.15.7:0/3808975196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2705cd4640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27001033d0 0x7f270019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2704cd2640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 0x7f2700198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 -- 172.21.15.7:0/3808975196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27001033d0 msgr2=0x7f270019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27001033d0 0x7f270019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 -- 172.21.15.7:0/3808975196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 msgr2=0x7f2700198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 0x7f2700198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 -- 172.21.15.7:0/3808975196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270019f6d0 con 0x7f2700101820 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f27054d3640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700197b00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e9a0 tx=0x7f26fc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00cd80 con 0x7f2700101820 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270019f9c0 con 0x7f2700101820 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.404+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26fc0108e0 con 0x7f2700101820 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.405+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc018950 con 0x7f2700101820 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.405+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270019fee0 con 0x7f2700101820 2024-03-19T00:49:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.405+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2700068af0 con 0x7f2700101820 2024-03-19T00:49:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.406+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f26fc010a90 con 0x7f2700101820 2024-03-19T00:49:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.407+0000 7f26f67fc640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2800 0x7f26ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.407+0000 7f2704cd2640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2800 0x7f26ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.407+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f26fc0cf120 con 0x7f2700101820 2024-03-19T00:49:57.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.407+0000 7f2704cd2640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2800 0x7f26ec0a4cc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26f0000c00 tx=0x7f26f00349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.411+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26fc053740 con 0x7f2700101820 2024-03-19T00:49:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.594+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2700062420 con 0x7f2700101820 2024-03-19T00:49:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.595+0000 7f26f67fc640 1 -- 172.21.15.7:0/3808975196 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f26fc09be60 con 0x7f2700101820 2024-03-19T00:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2800 msgr2=0x7f26ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2800 0x7f26ec0a4cc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26f0000c00 tx=0x7f26f00349a0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 msgr2=0x7f2700197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700197b00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e9a0 tx=0x7f26fc00ee60 comp rx=0 tx=0).stop 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 shutdown_connections 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f26ec0a2800 0x7f26ec0a4cc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27001033d0 0x7f270019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2700102a10 0x7f2700198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 --2- 172.21.15.7:0/3808975196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2700101820 0x7f2700197b00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 >> 172.21.15.7:0/3808975196 conn(0x7f27000fcd70 msgr2=0x7f2700105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 shutdown_connections 2024-03-19T00:49:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.597+0000 7f2707737640 1 -- 172.21.15.7:0/3808975196 wait complete. 2024-03-19T00:49:57.607 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:57 smithi007 ceph-mon[62854]: pgmap v711: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:49:57.607 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2860089697' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:49:57.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/436088845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:57.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:57 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/897521291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:57.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T00:49:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 -- 172.21.15.7:0/3462285508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c0ff8d0 msgr2=0x7fcd8c0ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 --2- 172.21.15.7:0/3462285508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c0ffd50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009900 tx=0x7fcd7c02bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 -- 172.21.15.7:0/3462285508 shutdown_connections 2024-03-19T00:49:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 --2- 172.21.15.7:0/3462285508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd8c100290 0x7fcd8c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 --2- 172.21.15.7:0/3462285508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c0ffd50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 --2- 172.21.15.7:0/3462285508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c101d50 0x7fcd8c102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 -- 172.21.15.7:0/3462285508 >> 172.21.15.7:0/3462285508 conn(0x7fcd8c0fb430 msgr2=0x7fcd8c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 -- 172.21.15.7:0/3462285508 shutdown_connections 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.713+0000 7fcd942bf640 1 -- 172.21.15.7:0/3462285508 wait complete. 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 Processor -- start 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 -- start start 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 0x7fcd8c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd8c101d50 0x7fcd8c1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c10f580 con 0x7fcd8c100290 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c10f400 con 0x7fcd8c0ff8d0 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd942bf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c10f700 con 0x7fcd8c101d50 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd9185a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 0x7fcd8c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd9205b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd9185a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 0x7fcd8c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57592/0 (socket says 172.21.15.7:57592) 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd9205b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44096/0 (socket says 172.21.15.7:44096) 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd9185a640 1 -- 172.21.15.7:0/1056295126 learned_addr learned my addr 172.21.15.7:0/1056295126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.714+0000 7fcd9285c640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd8c101d50 0x7fcd8c1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9185a640 1 -- 172.21.15.7:0/1056295126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd8c101d50 msgr2=0x7fcd8c1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9185a640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd8c101d50 0x7fcd8c1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9185a640 1 -- 172.21.15.7:0/1056295126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 msgr2=0x7fcd8c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9185a640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9185a640 1 -- 172.21.15.7:0/1056295126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c1a18d0 con 0x7fcd8c100290 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9205b640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd9185a640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 0x7fcd8c19a1d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c006010 tx=0x7fcd7c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c043020 con 0x7fcd8c100290 2024-03-19T00:49:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c1a1b60 con 0x7fcd8c100290 2024-03-19T00:49:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd7c0043d0 con 0x7fcd8c100290 2024-03-19T00:49:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c003b80 con 0x7fcd8c100290 2024-03-19T00:49:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.715+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c1a20b0 con 0x7fcd8c100290 2024-03-19T00:49:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.716+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c10b930 con 0x7fcd8c100290 2024-03-19T00:49:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.717+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7fcd7c046020 con 0x7fcd8c100290 2024-03-19T00:49:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.718+0000 7fcd837fe640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcd780a2800 0x7fcd780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.718+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcd7c0f0fd0 con 0x7fcd8c100290 2024-03-19T00:49:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.718+0000 7fcd9205b640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcd780a2800 0x7fcd780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.718+0000 7fcd9205b640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcd780a2800 0x7fcd780a4cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcd880095f0 tx=0x7fcd88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.722+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd7c04d020 con 0x7fcd8c100290 2024-03-19T00:49:57.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:57 smithi204 ceph-mon[66572]: pgmap v711: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:49:57.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2860089697' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:49:57.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/436088845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:57.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:57 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/897521291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:57 smithi044 ceph-mon[66366]: pgmap v711: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-19T00:49:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2860089697' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:49:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/436088845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:49:57.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:57 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/897521291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T00:49:57.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.904+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd8c062420 con 0x7fcd8c100290 2024-03-19T00:49:57.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.905+0000 7fcd837fe640 1 -- 172.21.15.7:0/1056295126 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fcd7c0bdd10 con 0x7fcd8c100290 2024-03-19T00:49:57.906 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcd780a2800 msgr2=0x7fcd780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcd780a2800 0x7fcd780a4cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcd880095f0 tx=0x7fcd88005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 msgr2=0x7fcd8c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 0x7fcd8c19a1d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c006010 tx=0x7fcd7c005e40 comp rx=0 tx=0).stop 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 shutdown_connections 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7fcd780a2800 0x7fcd780a4cc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd8c101d50 0x7fcd8c1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd8c100290 0x7fcd8c19a1d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 --2- 172.21.15.7:0/1056295126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0ff8d0 0x7fcd8c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 >> 172.21.15.7:0/1056295126 conn(0x7fcd8c0fb430 msgr2=0x7fcd8c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 shutdown_connections 2024-03-19T00:49:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:57.907+0000 7fcd942bf640 1 -- 172.21.15.7:0/1056295126 wait complete. 2024-03-19T00:49:57.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T00:49:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 -- 172.21.15.7:0/3325882334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc107520 msgr2=0x7f62cc109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 --2- 172.21.15.7:0/3325882334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc107520 0x7f62cc109970 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009ca0 tx=0x7f62bc02c100 comp rx=0 tx=0).stop 2024-03-19T00:49:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 -- 172.21.15.7:0/3325882334 shutdown_connections 2024-03-19T00:49:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 --2- 172.21.15.7:0/3325882334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc107520 0x7f62cc109970 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 --2- 172.21.15.7:0/3325882334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc104bf0 0x7f62cc106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 --2- 172.21.15.7:0/3325882334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc1022c0 0x7f62cc1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 -- 172.21.15.7:0/3325882334 >> 172.21.15.7:0/3325882334 conn(0x7f62cc0fbcb0 msgr2=0x7f62cc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 -- 172.21.15.7:0/3325882334 shutdown_connections 2024-03-19T00:49:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.023+0000 7f62d1227640 1 -- 172.21.15.7:0/3325882334 wait complete. 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 Processor -- start 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 -- start start 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc1022c0 0x7f62cc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 0x7f62cc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc107520 0x7f62cc1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc10d390 con 0x7f62cc104bf0 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc10d210 con 0x7f62cc107520 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62d1227640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc10d510 con 0x7f62cc1022c0 2024-03-19T00:49:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 0x7f62cc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 0x7f62cc19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57602/0 (socket says 172.21.15.7:57602) 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62ca59c640 1 -- 172.21.15.7:0/591850491 learned_addr learned my addr 172.21.15.7:0/591850491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62cad9d640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc1022c0 0x7f62cc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.024+0000 7f62cb59e640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc107520 0x7f62cc1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62ca59c640 1 -- 172.21.15.7:0/591850491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc1022c0 msgr2=0x7f62cc199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62ca59c640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc1022c0 0x7f62cc199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62ca59c640 1 -- 172.21.15.7:0/591850491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc107520 msgr2=0x7f62cc1a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62ca59c640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc107520 0x7f62cc1a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62ca59c640 1 -- 172.21.15.7:0/591850491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1a18c0 con 0x7f62cc104bf0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62ca59c640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 0x7f62cc19a1c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e9a0 tx=0x7f62b400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b400cd80 con 0x7f62cc104bf0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b40035f0 con 0x7f62cc104bf0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4010690 con 0x7f62cc104bf0 2024-03-19T00:49:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc1a1bb0 con 0x7f62cc104bf0 2024-03-19T00:49:58.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.025+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1a2100 con 0x7f62cc104bf0 2024-03-19T00:49:58.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.026+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc109010 con 0x7f62cc104bf0 2024-03-19T00:49:58.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.027+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f62b40037a0 con 0x7f62cc104bf0 2024-03-19T00:49:58.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.028+0000 7f62c3fff640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62b80a2800 0x7f62b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.028+0000 7f62cad9d640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62b80a2800 0x7f62b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.028+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f62b40cf080 con 0x7f62cc104bf0 2024-03-19T00:49:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.028+0000 7f62cad9d640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62b80a2800 0x7f62b80a4cc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f62cc19ab50 tx=0x7f62c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.032+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f62b409bd60 con 0x7f62cc104bf0 2024-03-19T00:49:58.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.214+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f62cc0624c0 con 0x7f62cc104bf0 2024-03-19T00:49:58.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.214+0000 7f62c3fff640 1 -- 172.21.15.7:0/591850491 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f62b409b660 con 0x7f62cc104bf0 2024-03-19T00:49:58.215 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-19T00:49:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62b80a2800 msgr2=0x7f62b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62b80a2800 0x7f62b80a4cc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f62cc19ab50 tx=0x7f62c4005f90 comp rx=0 tx=0).stop 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 msgr2=0x7f62cc19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 0x7f62cc19a1c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e9a0 tx=0x7f62b400ee60 comp rx=0 tx=0).stop 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 shutdown_connections 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f62b80a2800 0x7f62b80a4cc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc107520 0x7f62cc1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62cc104bf0 0x7f62cc19a1c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 --2- 172.21.15.7:0/591850491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc1022c0 0x7f62cc199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.216+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 >> 172.21.15.7:0/591850491 conn(0x7f62cc0fbcb0 msgr2=0x7f62cc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.217+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 shutdown_connections 2024-03-19T00:49:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.217+0000 7f62d1227640 1 -- 172.21.15.7:0/591850491 wait complete. 2024-03-19T00:49:58.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T00:49:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 -- 172.21.15.7:0/2032163419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 msgr2=0x7f8e44073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 --2- 172.21.15.7:0/2032163419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e44073c70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009900 tx=0x7f8e3402bc30 comp rx=0 tx=0).stop 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 -- 172.21.15.7:0/2032163419 shutdown_connections 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 --2- 172.21.15.7:0/2032163419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e441096d0 0x7f8e4410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 --2- 172.21.15.7:0/2032163419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e44074240 0x7f8e440746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 --2- 172.21.15.7:0/2032163419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e44073c70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 -- 172.21.15.7:0/2032163419 >> 172.21.15.7:0/2032163419 conn(0x7f8e440fcd70 msgr2=0x7f8e440ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 -- 172.21.15.7:0/2032163419 shutdown_connections 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.331+0000 7f8e4b854640 1 -- 172.21.15.7:0/2032163419 wait complete. 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 Processor -- start 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 -- start start 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 0x7f8e44199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e4419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e441096d0 0x7f8e441a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4410f510 con 0x7f8e44075820 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4410f390 con 0x7f8e441096d0 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e4b854640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4410f690 con 0x7f8e44074240 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e4419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e495f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 0x7f8e44199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e4419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57628/0 (socket says 172.21.15.7:57628) 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e495f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 0x7f8e44199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.7:45840/0 (socket says 172.21.15.7:45840) 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e49df1640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e441096d0 0x7f8e441a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 -- 172.21.15.7:0/1454314708 learned_addr learned my addr 172.21.15.7:0/1454314708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 -- 172.21.15.7:0/1454314708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 msgr2=0x7f8e44199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 0x7f8e44199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 -- 172.21.15.7:0/1454314708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e441096d0 msgr2=0x7f8e441a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e441096d0 0x7f8e441a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 -- 172.21.15.7:0/1454314708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e441a18d0 con 0x7f8e44075820 2024-03-19T00:49:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e495f0640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 0x7f8e44199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e49df1640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e441096d0 0x7f8e441a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e48def640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e4419a1d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9f0 tx=0x7f8e4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.332+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000cd80 con 0x7f8e44075820 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.333+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e400108e0 con 0x7f8e44075820 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.333+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e441a1bc0 con 0x7f8e44075820 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.333+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40018920 con 0x7f8e44075820 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.333+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a2160 con 0x7f8e44075820 2024-03-19T00:49:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.334+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44108450 con 0x7f8e44075820 2024-03-19T00:49:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.335+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f8e40010a90 con 0x7f8e44075820 2024-03-19T00:49:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.335+0000 7f8e3a7fc640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8e300a2880 0x7f8e300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:49:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.335+0000 7f8e495f0640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8e300a2880 0x7f8e300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:49:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.335+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8e400cf030 con 0x7f8e44075820 2024-03-19T00:49:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.336+0000 7f8e495f0640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8e300a2880 0x7f8e300a4d40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e4419ab60 tx=0x7f8e34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:49:58.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.340+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e4009b720 con 0x7f8e44075820 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.537+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e4402d1f0 con 0x7f8e44075820 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.538+0000 7f8e3a7fc640 1 -- 172.21.15.7:0/1454314708 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+577 (secure 0 0 0) 0x7f8e4009f7c0 con 0x7f8e44075820 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1277095143 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4032873653 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/501265234 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1320706322 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:58.539 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/237878025 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:58.540 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:58.540 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1237634613 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:58.540 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3660465780 2024-03-20T00:28:30.106926+0000 2024-03-19T00:49:58.540 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/97837638 2024-03-20T00:28:55.916728+0000 2024-03-19T00:49:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-03-19T00:49:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8e300a2880 msgr2=0x7f8e300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8e300a2880 0x7f8e300a4d40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e4419ab60 tx=0x7f8e34005fd0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 msgr2=0x7f8e4419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:49:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e4419a1d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9f0 tx=0x7f8e4000eeb0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 shutdown_connections 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8e300a2880 0x7f8e300a4d40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e441096d0 0x7f8e441a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e44075820 0x7f8e4419a1d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 --2- 172.21.15.7:0/1454314708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e44074240 0x7f8e44199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 >> 172.21.15.7:0/1454314708 conn(0x7f8e440fcd70 msgr2=0x7f8e44105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 shutdown_connections 2024-03-19T00:49:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:49:58.540+0000 7f8e4b854640 1 -- 172.21.15.7:0/1454314708 wait complete. 2024-03-19T00:49:58.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3808975196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1056295126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:58 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/591850491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.552 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-03-19T00:49:58.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:49:58.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-19T00:49:58.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[113479]: fuse finished with error 0 and tester_r 0 2024-03-19T00:49:58.578 INFO:teuthology.orchestra.run:waiting for 300 2024-03-19T00:49:58.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3808975196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1056295126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:58 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/591850491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3808975196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1056295126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T00:49:58.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:58 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/591850491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T00:49:59.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:59 smithi007 ceph-mon[62854]: pgmap v712: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:59.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:49:59 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1454314708' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:49:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:59 smithi204 ceph-mon[66572]: pgmap v712: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:59.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:49:59 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1454314708' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:49:59.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:59 smithi044 ceph-mon[66366]: pgmap v712: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-19T00:49:59.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:49:59 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1454314708' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T00:50:00.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:00 smithi204 ceph-mon[66572]: overall HEALTH_OK 2024-03-19T00:50:00.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:00 smithi044 ceph-mon[66366]: overall HEALTH_OK 2024-03-19T00:50:00.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:00 smithi007 ceph-mon[62854]: overall HEALTH_OK 2024-03-19T00:50:01.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:01 smithi204 ceph-mon[66572]: pgmap v713: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-19T00:50:01.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:01 smithi044 ceph-mon[66366]: pgmap v713: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-19T00:50:01.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:01 smithi007 ceph-mon[62854]: pgmap v713: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-19T00:50:03.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:03 smithi204 ceph-mon[66572]: pgmap v714: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-19T00:50:03.807 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:03 smithi007 ceph-mon[62854]: pgmap v714: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-19T00:50:03.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:03 smithi044 ceph-mon[66366]: pgmap v714: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-19T00:50:04.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-03-19T00:50:04.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:50:04.680 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-19T00:50:04.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:50:04.701 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-19T00:50:04.758 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-19T00:50:04.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:50:04.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T00:50:04.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T00:50:04.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-19T00:50:04.927 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-19T00:50:04.952 INFO:tasks.cephadm:Teardown begin 2024-03-19T00:50:04.952 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-19T00:50:04.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 -- 172.21.15.7:0/3270312735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 msgr2=0x7f30c4109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 --2- 172.21.15.7:0/3270312735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c4109980 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009ca0 tx=0x7f30b402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 -- 172.21.15.7:0/3270312735 shutdown_connections 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 --2- 172.21.15.7:0/3270312735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c4109980 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 --2- 172.21.15.7:0/3270312735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c4104c60 0x7f30c4107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 --2- 172.21.15.7:0/3270312735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30c4102330 0x7f30c4104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 -- 172.21.15.7:0/3270312735 >> 172.21.15.7:0/3270312735 conn(0x7f30c40fbce0 msgr2=0x7f30c40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 -- 172.21.15.7:0/3270312735 shutdown_connections 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 -- 172.21.15.7:0/3270312735 wait complete. 2024-03-19T00:50:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 Processor -- start 2024-03-19T00:50:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.059+0000 7f30c959e640 1 -- start start 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30c4102330 0x7f30c4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c4104c60 0x7f30c4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30c410d380 con 0x7f30c4107590 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30c410d200 con 0x7f30c4102330 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30c410d500 con 0x7f30c4104c60 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35714/0 (socket says 172.21.15.7:35714) 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 -- 172.21.15.7:0/3310086964 learned_addr learned my addr 172.21.15.7:0/3310086964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c2ffd640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30c4102330 0x7f30c4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c27fc640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c4104c60 0x7f30c4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 -- 172.21.15.7:0/3310086964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c4104c60 msgr2=0x7f30c4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c4104c60 0x7f30c4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 -- 172.21.15.7:0/3310086964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30c4102330 msgr2=0x7f30c4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30c4102330 0x7f30c4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 -- 172.21.15.7:0/3310086964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c419f650 con 0x7f30c4107590 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c37fe640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c419efe0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f30b402c700 tx=0x7f30b40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4041020 con 0x7f30c4107590 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c419f8e0 con 0x7f30c4107590 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30b4003e20 con 0x7f30c4107590 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4039040 con 0x7f30c4107590 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.060+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c419fe20 con 0x7f30c4107590 2024-03-19T00:50:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.061+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c40fd7d0 con 0x7f30c4107590 2024-03-19T00:50:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.063+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f30b4044020 con 0x7f30c4107590 2024-03-19T00:50:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.063+0000 7f30abfff640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f30ac0a2800 0x7f30ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.063+0000 7f30c2ffd640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f30ac0a2800 0x7f30ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.063+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f30b40f1350 con 0x7f30c4107590 2024-03-19T00:50:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.064+0000 7f30c2ffd640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f30ac0a2800 0x7f30ac0a4cc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0095f0 tx=0x7f30bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.067+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f30b404b020 con 0x7f30c4107590 2024-03-19T00:50:05.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.290+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c402d1f0 con 0x7f30c4107590 2024-03-19T00:50:05.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.291+0000 7f30abfff640 1 -- 172.21.15.7:0/3310086964 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11856 (secure 0 0 0) 0x7f30b40be090 con 0x7f30c4107590 2024-03-19T00:50:05.292 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:50:05.292 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:49:17.370027+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:50:05.293 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-03-19T00:50:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f30ac0a2800 msgr2=0x7f30ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f30ac0a2800 0x7f30ac0a4cc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0095f0 tx=0x7f30bc005f90 comp rx=0 tx=0).stop 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 msgr2=0x7f30c419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c419efe0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f30b402c700 tx=0x7f30b40328a0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 shutdown_connections 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f30ac0a2800 0x7f30ac0a4cc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c4107590 0x7f30c419efe0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c4104c60 0x7f30c4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 --2- 172.21.15.7:0/3310086964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30c4102330 0x7f30c4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.293+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 >> 172.21.15.7:0/3310086964 conn(0x7f30c40fbce0 msgr2=0x7f30c4102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.294+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 shutdown_connections 2024-03-19T00:50:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.294+0000 7f30c959e640 1 -- 172.21.15.7:0/3310086964 wait complete. 2024-03-19T00:50:05.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T00:50:05.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 -- 172.21.15.7:0/3180668697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 msgr2=0x7f8d5c109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 --2- 172.21.15.7:0/3180668697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c109eb0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009ca0 tx=0x7f8d5402c0f0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 -- 172.21.15.7:0/3180668697 shutdown_connections 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 --2- 172.21.15.7:0/3180668697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c109eb0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 --2- 172.21.15.7:0/3180668697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c0ff940 0x7f8d5c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 --2- 172.21.15.7:0/3180668697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c102ed0 0x7f8d5c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 -- 172.21.15.7:0/3180668697 >> 172.21.15.7:0/3180668697 conn(0x7f8d5c0fb4a0 msgr2=0x7f8d5c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 -- 172.21.15.7:0/3180668697 shutdown_connections 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 -- 172.21.15.7:0/3180668697 wait complete. 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.408+0000 7f8d645d4640 1 Processor -- start 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 -- start start 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c0ff940 0x7f8d5c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c102ed0 0x7f8d5c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c10d370 con 0x7f8d5c100300 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c10d1f0 con 0x7f8d5c0ff940 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d645d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c10d4f0 con 0x7f8d5c102ed0 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35738/0 (socket says 172.21.15.7:35738) 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d62370640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c0ff940 0x7f8d5c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 -- 172.21.15.7:0/2632282851 learned_addr learned my addr 172.21.15.7:0/2632282851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d62b71640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c102ed0 0x7f8d5c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 -- 172.21.15.7:0/2632282851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c102ed0 msgr2=0x7f8d5c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c102ed0 0x7f8d5c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 -- 172.21.15.7:0/2632282851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c0ff940 msgr2=0x7f8d5c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c0ff940 0x7f8d5c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 -- 172.21.15.7:0/2632282851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c19f5e0 con 0x7f8d5c100300 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d62370640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c0ff940 0x7f8d5c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:50:05.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d61b6f640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c197f00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00ea60 tx=0x7f8d4c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:05.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00cde0 con 0x7f8d5c100300 2024-03-19T00:50:05.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.409+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d4c0035f0 con 0x7f8d5c100300 2024-03-19T00:50:05.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.410+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c19f8d0 con 0x7f8d5c100300 2024-03-19T00:50:05.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.410+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c010690 con 0x7f8d5c100300 2024-03-19T00:50:05.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.410+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c19fe20 con 0x7f8d5c100300 2024-03-19T00:50:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.410+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c108d90 con 0x7f8d5c100300 2024-03-19T00:50:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.412+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f8d4c0037a0 con 0x7f8d5c100300 2024-03-19T00:50:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.412+0000 7f8d537fe640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d480a2800 0x7f8d480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.412+0000 7f8d62370640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d480a2800 0x7f8d480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.412+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8d4c0cf2d0 con 0x7f8d5c100300 2024-03-19T00:50:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.413+0000 7f8d62370640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d480a2800 0x7f8d480a4cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c069c80 tx=0x7f8d58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.416+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8d4c053740 con 0x7f8d5c100300 2024-03-19T00:50:05.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.640+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d5c02d420 con 0x7f8d5c100300 2024-03-19T00:50:05.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.641+0000 7f8d537fe640 1 -- 172.21.15.7:0/2632282851 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11856 (secure 0 0 0) 0x7f8d4c09c010 con 0x7f8d5c100300 2024-03-19T00:50:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:50:05.642 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2506887195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2506887195},{"type":"v1","addr":"172.21.15.7:6835","nonce":2506887195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/520010551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":520010551},{"type":"v1","addr":"172.21.15.7:6837","nonce":520010551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1114335301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1114335301},{"type":"v1","addr":"172.21.15.7:6839","nonce":1114335301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3178899604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3178899604},{"type":"v1","addr":"172.21.15.7:6841","nonce":3178899604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1644274763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1644274763},{"type":"v1","addr":"172.21.15.204:6833","nonce":1644274763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1889634498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889634498},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889634498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T00:35:59.576911+0000","modified":"2024-03-19T00:49:17.370027+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24497,"mds_2":24490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3301743833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3301743833},{"type":"v1","addr":"172.21.15.204:6835","nonce":3301743833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/1036080774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1036080774},{"type":"v1","addr":"172.21.15.44:6835","nonce":1036080774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2630641732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2630641732},{"type":"v1","addr":"172.21.15.204:6837","nonce":2630641732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/945498116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":945498116},{"type":"v1","addr":"172.21.15.204:6839","nonce":945498116}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/3379136750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3379136750},{"type":"v1","addr":"172.21.15.44:6837","nonce":3379136750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/617428839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":617428839},{"type":"v1","addr":"172.21.15.44:6839","nonce":617428839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24490,24497]},"id":1}]} 2024-03-19T00:50:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d480a2800 msgr2=0x7f8d480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d480a2800 0x7f8d480a4cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c069c80 tx=0x7f8d58005f90 comp rx=0 tx=0).stop 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 msgr2=0x7f8d5c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c197f00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00ea60 tx=0x7f8d4c00ef20 comp rx=0 tx=0).stop 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 shutdown_connections 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f8d480a2800 0x7f8d480a4cc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5c102ed0 0x7f8d5c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d5c100300 0x7f8d5c197f00 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 --2- 172.21.15.7:0/2632282851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d5c0ff940 0x7f8d5c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 >> 172.21.15.7:0/2632282851 conn(0x7f8d5c0fb4a0 msgr2=0x7f8d5c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 shutdown_connections 2024-03-19T00:50:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.643+0000 7f8d645d4640 1 -- 172.21.15.7:0/2632282851 wait complete. 2024-03-19T00:50:05.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 -- 172.21.15.7:0/196497284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 msgr2=0x7f61d8106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/196497284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8106ef0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc02bc30 comp rx=0 tx=0).stop 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 -- 172.21.15.7:0/196497284 shutdown_connections 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/196497284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d8107430 0x7f61d8109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/196497284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8106ef0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/196497284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61d81021d0 0x7f61d81045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 -- 172.21.15.7:0/196497284 >> 172.21.15.7:0/196497284 conn(0x7f61d80fbbe0 msgr2=0x7f61d80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 -- 172.21.15.7:0/196497284 shutdown_connections 2024-03-19T00:50:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.757+0000 7f61ddb9d640 1 -- 172.21.15.7:0/196497284 wait complete. 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 Processor -- start 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 -- start start 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d81021d0 0x7f61d81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61d8107430 0x7f61d819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61d810d290 con 0x7f61d8104b00 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61d810d110 con 0x7f61d81021d0 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61ddb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61d810d410 con 0x7f61d8107430 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35764/0 (socket says 172.21.15.7:35764) 2024-03-19T00:50:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 -- 172.21.15.7:0/1737834695 learned_addr learned my addr 172.21.15.7:0/1737834695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61dcb9b640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d81021d0 0x7f61d81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61dd39c640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61d8107430 0x7f61d819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 -- 172.21.15.7:0/1737834695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61d8107430 msgr2=0x7f61d819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61d8107430 0x7f61d819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 -- 172.21.15.7:0/1737834695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d81021d0 msgr2=0x7f61d81979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d81021d0 0x7f61d81979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.758+0000 7f61d7fff640 1 -- 172.21.15.7:0/1737834695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d819f5e0 con 0x7f61d8104b00 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.759+0000 7f61d7fff640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8197f00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f61cc000c00 tx=0x7f61cc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.759+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc043020 con 0x7f61d8104b00 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.759+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61cc004350 con 0x7f61d8104b00 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.759+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d819f870 con 0x7f61d8104b00 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.759+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc003b10 con 0x7f61d8104b00 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.759+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d819fdc0 con 0x7f61d8104b00 2024-03-19T00:50:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.760+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d80fd6d0 con 0x7f61d8104b00 2024-03-19T00:50:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.761+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f61cc046020 con 0x7f61d8104b00 2024-03-19T00:50:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.762+0000 7f61d5ffb640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61c00a27f0 0x7f61c00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.762+0000 7f61dcb9b640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61c00a27f0 0x7f61c00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.762+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f61cc0f0f80 con 0x7f61d8104b00 2024-03-19T00:50:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.762+0000 7f61dcb9b640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61c00a27f0 0x7f61c00a4cb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005fd0 tx=0x7f61d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.766+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f61cc04d020 con 0x7f61d8104b00 2024-03-19T00:50:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:05 smithi044 ceph-mon[66366]: pgmap v715: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:50:05.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:05 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3310086964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:50:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.953+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61d8062420 con 0x7f61d8104b00 2024-03-19T00:50:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.954+0000 7f61d5ffb640 1 -- 172.21.15.7:0/1737834695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24444 (secure 0 0 0) 0x7f61cc0bdc60 con 0x7f61d8104b00 2024-03-19T00:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-19T00:50:05.956 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"60839b00-e587-11ee-95c9-87774f69a715","created":"2024-03-19T00:28:02.904625+0000","modified":"2024-03-19T00:36:57.940744+0000","last_up_change":"2024-03-19T00:35:09.616255+0000","last_in_change":"2024-03-19T00:34:49.193658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T00:31:18.272582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T00:35:58.138748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T00:35:58.754565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T00:36:02.466956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d5ef20b-dcde-4e0d-8a75-d7f4369aaeea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6803","nonce":2757458659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6805","nonce":2757458659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6809","nonce":2757458659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2757458659},{"type":"v1","addr":"172.21.15.7:6807","nonce":2757458659}]},"public_addr":"172.21.15.7:6803/2757458659","cluster_addr":"172.21.15.7:6805/2757458659","heartbeat_back_addr":"172.21.15.7:6809/2757458659","heartbeat_front_addr":"172.21.15.7:6807/2757458659","state":["exists","up"]},{"osd":1,"uuid":"baa358f6-e033-488f-9078-2a0c3b3c38f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6801","nonce":274970163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6803","nonce":274970163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6807","nonce":274970163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":274970163},{"type":"v1","addr":"172.21.15.44:6805","nonce":274970163}]},"public_addr":"172.21.15.44:6801/274970163","cluster_addr":"172.21.15.44:6803/274970163","heartbeat_back_addr":"172.21.15.44:6807/274970163","heartbeat_front_addr":"172.21.15.44:6805/274970163","state":["exists","up"]},{"osd":2,"uuid":"c4bd1927-870b-4e32-ad51-aff87143cb46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6801","nonce":281046450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6803","nonce":281046450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6807","nonce":281046450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":281046450},{"type":"v1","addr":"172.21.15.204:6805","nonce":281046450}]},"public_addr":"172.21.15.204:6801/281046450","cluster_addr":"172.21.15.204:6803/281046450","heartbeat_back_addr":"172.21.15.204:6807/281046450","heartbeat_front_addr":"172.21.15.204:6805/281046450","state":["exists","up"]},{"osd":3,"uuid":"0e3811a3-7436-489a-ac0a-857023269d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6811","nonce":375379923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6813","nonce":375379923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6817","nonce":375379923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":375379923},{"type":"v1","addr":"172.21.15.7:6815","nonce":375379923}]},"public_addr":"172.21.15.7:6811/375379923","cluster_addr":"172.21.15.7:6813/375379923","heartbeat_back_addr":"172.21.15.7:6817/375379923","heartbeat_front_addr":"172.21.15.7:6815/375379923","state":["exists","up"]},{"osd":4,"uuid":"993175bf-d87f-4369-9ded-8b968d4b9160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6809","nonce":3332813314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6811","nonce":3332813314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6815","nonce":3332813314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3332813314},{"type":"v1","addr":"172.21.15.44:6813","nonce":3332813314}]},"public_addr":"172.21.15.44:6809/3332813314","cluster_addr":"172.21.15.44:6811/3332813314","heartbeat_back_addr":"172.21.15.44:6815/3332813314","heartbeat_front_addr":"172.21.15.44:6813/3332813314","state":["exists","up"]},{"osd":5,"uuid":"3a893438-b29c-4c46-bbe8-ba8ac823e8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6809","nonce":3713019537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6811","nonce":3713019537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6815","nonce":3713019537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3713019537},{"type":"v1","addr":"172.21.15.204:6813","nonce":3713019537}]},"public_addr":"172.21.15.204:6809/3713019537","cluster_addr":"172.21.15.204:6811/3713019537","heartbeat_back_addr":"172.21.15.204:6815/3713019537","heartbeat_front_addr":"172.21.15.204:6813/3713019537","state":["exists","up"]},{"osd":6,"uuid":"9dfd817f-972c-4c18-8d94-76aebea473aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6819","nonce":3051876259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6821","nonce":3051876259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6825","nonce":3051876259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3051876259},{"type":"v1","addr":"172.21.15.7:6823","nonce":3051876259}]},"public_addr":"172.21.15.7:6819/3051876259","cluster_addr":"172.21.15.7:6821/3051876259","heartbeat_back_addr":"172.21.15.7:6825/3051876259","heartbeat_front_addr":"172.21.15.7:6823/3051876259","state":["exists","up"]},{"osd":7,"uuid":"f17cf883-f837-421c-8306-e1091b9b5f73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6817","nonce":48320224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6819","nonce":48320224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6823","nonce":48320224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":48320224},{"type":"v1","addr":"172.21.15.44:6821","nonce":48320224}]},"public_addr":"172.21.15.44:6817/48320224","cluster_addr":"172.21.15.44:6819/48320224","heartbeat_back_addr":"172.21.15.44:6823/48320224","heartbeat_front_addr":"172.21.15.44:6821/48320224","state":["exists","up"]},{"osd":8,"uuid":"295a9edc-0de0-4302-8887-8717d45ba545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6817","nonce":1654194341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6819","nonce":1654194341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6823","nonce":1654194341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1654194341},{"type":"v1","addr":"172.21.15.204:6821","nonce":1654194341}]},"public_addr":"172.21.15.204:6817/1654194341","cluster_addr":"172.21.15.204:6819/1654194341","heartbeat_back_addr":"172.21.15.204:6823/1654194341","heartbeat_front_addr":"172.21.15.204:6821/1654194341","state":["exists","up"]},{"osd":9,"uuid":"3844148f-1d07-4f72-a3c0-e249888e8416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6827","nonce":1022710322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6829","nonce":1022710322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6833","nonce":1022710322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1022710322},{"type":"v1","addr":"172.21.15.7:6831","nonce":1022710322}]},"public_addr":"172.21.15.7:6827/1022710322","cluster_addr":"172.21.15.7:6829/1022710322","heartbeat_back_addr":"172.21.15.7:6833/1022710322","heartbeat_front_addr":"172.21.15.7:6831/1022710322","state":["exists","up"]},{"osd":10,"uuid":"88ad1139-05b0-45d2-938e-bae6421c1a4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6825","nonce":443402515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6827","nonce":443402515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6831","nonce":443402515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":443402515},{"type":"v1","addr":"172.21.15.44:6829","nonce":443402515}]},"public_addr":"172.21.15.44:6825/443402515","cluster_addr":"172.21.15.44:6827/443402515","heartbeat_back_addr":"172.21.15.44:6831/443402515","heartbeat_front_addr":"172.21.15.44:6829/443402515","state":["exists","up"]},{"osd":11,"uuid":"1b004ee9-0ccb-4258-bc42-4b24baf7931f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6825","nonce":849720251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6827","nonce":849720251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6831","nonce":849720251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":849720251},{"type":"v1","addr":"172.21.15.204:6829","nonce":849720251}]},"public_addr":"172.21.15.204:6825/849720251","cluster_addr":"172.21.15.204:6827/849720251","heartbeat_back_addr":"172.21.15.204:6831/849720251","heartbeat_front_addr":"172.21.15.204:6829/849720251","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:28.028061+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:30:52.301060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:15.288950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:31:41.102749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:06.477790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:31.560355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:32:57.126551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:23.148699+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:33:48.730196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:15.137301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:34:41.253461+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T00:35:07.160499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1277095143":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/4032873653":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6801/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:6800/501265234":"2024-03-20T00:28:55.916728+0000","172.21.15.7:0/1320706322":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/237878025":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6800/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:6801/1237634613":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/3660465780":"2024-03-20T00:28:30.106926+0000","172.21.15.7:0/97837638":"2024-03-20T00:28:55.916728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T00:50:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61c00a27f0 msgr2=0x7f61c00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61c00a27f0 0x7f61c00a4cb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005fd0 tx=0x7f61d0009500 comp rx=0 tx=0).stop 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 msgr2=0x7f61d8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8197f00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f61cc000c00 tx=0x7f61cc005dc0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 shutdown_connections 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f61c00a27f0 0x7f61c00a4cb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61d8107430 0x7f61d819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61d8104b00 0x7f61d8197f00 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 --2- 172.21.15.7:0/1737834695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61d81021d0 0x7f61d81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.956+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 >> 172.21.15.7:0/1737834695 conn(0x7f61d80fbbe0 msgr2=0x7f61d8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.957+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 shutdown_connections 2024-03-19T00:50:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:05.957+0000 7f61ddb9d640 1 -- 172.21.15.7:0/1737834695 wait complete. 2024-03-19T00:50:05.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:05 smithi007 ceph-mon[62854]: pgmap v715: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:50:05.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:05 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3310086964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:50:05.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-19T00:50:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:05 smithi204 ceph-mon[66572]: pgmap v715: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-19T00:50:06.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:05 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3310086964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- 172.21.15.7:0/1771518658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101390 msgr2=0x7f53d8108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- 172.21.15.7:0/1771518658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101390 0x7f53d8108c30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009cd0 tx=0x7f53d002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- 172.21.15.7:0/1771518658 shutdown_connections 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- 172.21.15.7:0/1771518658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101390 0x7f53d8108c30 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- 172.21.15.7:0/1771518658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d81009d0 0x7f53d8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- 172.21.15.7:0/1771518658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d8101d30 0x7f53d8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- 172.21.15.7:0/1771518658 >> 172.21.15.7:0/1771518658 conn(0x7f53d80fb440 msgr2=0x7f53d80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- 172.21.15.7:0/1771518658 shutdown_connections 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- 172.21.15.7:0/1771518658 wait complete. 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 Processor -- start 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- start start 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d81009d0 0x7f53d8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8101390 0x7f53d8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 0x7f53d819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.072+0000 7f53e0390640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53d810d3c0 con 0x7f53d8101d30 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53e0390640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53d810d240 con 0x7f53d8101390 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53e0390640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53d810d540 con 0x7f53d81009d0 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 0x7f53d819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53dd92b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8101390 0x7f53d8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 0x7f53d819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35772/0 (socket says 172.21.15.7:35772) 2024-03-19T00:50:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 -- 172.21.15.7:0/1453292945 learned_addr learned my addr 172.21.15.7:0/1453292945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de12c640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d81009d0 0x7f53d8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 -- 172.21.15.7:0/1453292945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d81009d0 msgr2=0x7f53d8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d81009d0 0x7f53d8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 -- 172.21.15.7:0/1453292945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8101390 msgr2=0x7f53d8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8101390 0x7f53d8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 -- 172.21.15.7:0/1453292945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d819f640 con 0x7f53d8101d30 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53de92d640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 0x7f53d819efd0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f53d002c630 tx=0x7f53d00328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0041020 con 0x7f53d8101d30 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d0003970 con 0x7f53d8101d30 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d819f8d0 con 0x7f53d8101d30 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0039040 con 0x7f53d8101d30 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.073+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d819fdc0 con 0x7f53d8101d30 2024-03-19T00:50:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.074+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d8107b30 con 0x7f53d8101d30 2024-03-19T00:50:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.075+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f53d0044020 con 0x7f53d8101d30 2024-03-19T00:50:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.076+0000 7f53cf7fe640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f53c40a2800 0x7f53c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.076+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f53d00f1280 con 0x7f53d8101d30 2024-03-19T00:50:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.076+0000 7f53de12c640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f53c40a2800 0x7f53c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.077+0000 7f53de12c640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f53c40a2800 0x7f53c40a4cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f53d8069cc0 tx=0x7f53d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.080+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f53d004b020 con 0x7f53d8101d30 2024-03-19T00:50:06.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.302+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f53d8062420 con 0x7f53d8101d30 2024-03-19T00:50:06.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.304+0000 7f53cf7fe640 1 -- 172.21.15.7:0/1453292945 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f53d00098b0 con 0x7f53d8101d30 2024-03-19T00:50:06.305 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f53c40a2800 msgr2=0x7f53c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f53c40a2800 0x7f53c40a4cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f53d8069cc0 tx=0x7f53d4005f90 comp rx=0 tx=0).stop 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 msgr2=0x7f53d819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 0x7f53d819efd0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f53d002c630 tx=0x7f53d00328a0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 shutdown_connections 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f53c40a2800 0x7f53c40a4cc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d8101d30 0x7f53d819efd0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53d8101390 0x7f53d8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 --2- 172.21.15.7:0/1453292945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53d81009d0 0x7f53d8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 >> 172.21.15.7:0/1453292945 conn(0x7f53d80fb440 msgr2=0x7f53d8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:06.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 shutdown_connections 2024-03-19T00:50:06.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.307+0000 7f53e0390640 1 -- 172.21.15.7:0/1453292945 wait complete. 2024-03-19T00:50:06.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 -- 172.21.15.7:0/3784932131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64107540 msgr2=0x7f4d64109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 --2- 172.21.15.7:0/3784932131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64107540 0x7f4d64109970 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009ca0 tx=0x7f4d5c02c0f0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 -- 172.21.15.7:0/3784932131 shutdown_connections 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 --2- 172.21.15.7:0/3784932131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64107540 0x7f4d64109970 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 --2- 172.21.15.7:0/3784932131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d64104c10 0x7f4d64107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 --2- 172.21.15.7:0/3784932131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d641022e0 0x7f4d641046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 -- 172.21.15.7:0/3784932131 >> 172.21.15.7:0/3784932131 conn(0x7f4d640fbcb0 msgr2=0x7f4d640fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 -- 172.21.15.7:0/3784932131 shutdown_connections 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.427+0000 7f4d68e12640 1 -- 172.21.15.7:0/3784932131 wait complete. 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 Processor -- start 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 -- start start 2024-03-19T00:50:06.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d641022e0 0x7f4d64197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 0x7f4d64197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d64107540 0x7f4d6419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410d370 con 0x7f4d64104c10 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410d1f0 con 0x7f4d641022e0 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d68e12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410d4f0 con 0x7f4d64107540 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d61d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 0x7f4d64197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.428+0000 7f4d61d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 0x7f4d64197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35786/0 (socket says 172.21.15.7:35786) 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 -- 172.21.15.7:0/613635158 learned_addr learned my addr 172.21.15.7:0/613635158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d6259c640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d641022e0 0x7f4d64197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d62d9d640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d64107540 0x7f4d6419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 -- 172.21.15.7:0/613635158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d64107540 msgr2=0x7f4d6419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d64107540 0x7f4d6419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 -- 172.21.15.7:0/613635158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d641022e0 msgr2=0x7f4d64197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d641022e0 0x7f4d64197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 -- 172.21.15.7:0/613635158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6419f640 con 0x7f4d64104c10 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d6259c640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d641022e0 0x7f4d64197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d61d9b640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 0x7f4d64197f60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e9c0 tx=0x7f4d5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400cd80 con 0x7f4d64104c10 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6419f930 con 0x7f4d64104c10 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d540054d0 con 0x7f4d64104c10 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54010430 con 0x7f4d64104c10 2024-03-19T00:50:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.429+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6419fed0 con 0x7f4d64104c10 2024-03-19T00:50:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.430+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d640686a0 con 0x7f4d64104c10 2024-03-19T00:50:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.431+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f4d54004fb0 con 0x7f4d64104c10 2024-03-19T00:50:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.432+0000 7f4d537fe640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d4c0a2800 0x7f4d4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.432+0000 7f4d6259c640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d4c0a2800 0x7f4d4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.432+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f4d540cf610 con 0x7f4d64104c10 2024-03-19T00:50:06.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.432+0000 7f4d6259c640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d4c0a2800 0x7f4d4c0a4cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4d580095f0 tx=0x7f4d58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.440+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4d5409c250 con 0x7f4d64104c10 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a[62850]: 2024-03-19T00:50:06.303+0000 7ff149fec700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/2632282851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1737834695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1453292945' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/1453292945' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: osdmap e92: 12 total, 12 up, 12 in 2024-03-19T00:50:06.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:06 smithi007 ceph-mon[62854]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-19T00:50:06.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: ignoring --setuser ceph since I am not root 2024-03-19T00:50:06.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: ignoring --setgroup ceph since I am not root 2024-03-19T00:50:06.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mds[96561]: -- 172.21.15.44:0/234877409 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c14356fa40 con 0x55c143542800 2024-03-19T00:50:06.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: starting mds.b at 2024-03-19T00:50:06.606 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e[95937]: ignoring --setuser ceph since I am not root 2024-03-19T00:50:06.606 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e[95937]: ignoring --setgroup ceph since I am not root 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mds[95941]: -- 172.21.15.44:0/1976938363 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a590241a40 con 0x55a590215000 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e[95937]: starting mds.e at 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/2632282851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1737834695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1453292945' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/1453292945' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: osdmap e92: 12 total, 12 up, 12 in 2024-03-19T00:50:06.607 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mon[66366]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-19T00:50:06.608 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h[96250]: ignoring --setuser ceph since I am not root 2024-03-19T00:50:06.608 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h[96250]: ignoring --setgroup ceph since I am not root 2024-03-19T00:50:06.608 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-mds[96254]: -- 172.21.15.44:0/1521708712 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d52420ba40 con 0x55d5241de800 2024-03-19T00:50:06.608 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:50:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h[96250]: starting mds.h at 2024-03-19T00:50:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:06.668+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4d640fd7a0 con 0x7f4d64104c10 2024-03-19T00:50:06.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i[97471]: ignoring --setuser ceph since I am not root 2024-03-19T00:50:06.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i[97471]: ignoring --setgroup ceph since I am not root 2024-03-19T00:50:06.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mds[97475]: -- 172.21.15.204:0/3616772624 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b761231a40 con 0x55b761204800 2024-03-19T00:50:06.765 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i[97471]: starting mds.i at 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f[96854]: ignoring --setuser ceph since I am not root 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f[96854]: ignoring --setgroup ceph since I am not root 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mds[96858]: -- 172.21.15.204:0/1040502038 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c839ae3a40 con 0x55c839ab6c00 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f[96854]: starting mds.f at 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c[97163]: ignoring --setuser ceph since I am not root 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c[97163]: ignoring --setgroup ceph since I am not root 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mds[97167]: -- 172.21.15.204:0/1399702575 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be6d15fa40 con 0x55be6d132800 2024-03-19T00:50:06.766 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c[97163]: starting mds.c at 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/2632282851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1737834695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1453292945' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/1453292945' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: osdmap e92: 12 total, 12 up, 12 in 2024-03-19T00:50:06.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:06 smithi204 ceph-mon[66572]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-19T00:50:07.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.305+0000 7f4d537fe640 1 -- 172.21.15.7:0/613635158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f4d5409bb50 con 0x7f4d64104c10 2024-03-19T00:50:07.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.307+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d4c0a2800 msgr2=0x7f4d4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.307+0000 7f4d68e12640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d4c0a2800 0x7f4d4c0a4cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4d580095f0 tx=0x7f4d58005f90 comp rx=0 tx=0).stop 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 msgr2=0x7f4d64197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 0x7f4d64197f60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e9c0 tx=0x7f4d5400ee80 comp rx=0 tx=0).stop 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 shutdown_connections 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f4d4c0a2800 0x7f4d4c0a4cc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d64107540 0x7f4d6419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d64104c10 0x7f4d64197f60 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 --2- 172.21.15.7:0/613635158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d641022e0 0x7f4d64197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 >> 172.21.15.7:0/613635158 conn(0x7f4d640fbcb0 msgr2=0x7f4d64102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 shutdown_connections 2024-03-19T00:50:07.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.308+0000 7f4d68e12640 1 -- 172.21.15.7:0/613635158 wait complete. 2024-03-19T00:50:07.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-19T00:50:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 -- 172.21.15.7:0/2636651611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce01075c0 msgr2=0x7f5ce01099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 --2- 172.21.15.7:0/2636651611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce01075c0 0x7f5ce01099b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009ca0 tx=0x7f5cd802c0f0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 -- 172.21.15.7:0/2636651611 shutdown_connections 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 --2- 172.21.15.7:0/2636651611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce01075c0 0x7f5ce01099b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 --2- 172.21.15.7:0/2636651611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce0104c90 0x7f5ce0107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 --2- 172.21.15.7:0/2636651611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce0102360 0x7f5ce0104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.428+0000 7f5ce77df640 1 -- 172.21.15.7:0/2636651611 >> 172.21.15.7:0/2636651611 conn(0x7f5ce00fbcb0 msgr2=0x7f5ce00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 -- 172.21.15.7:0/2636651611 shutdown_connections 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 -- 172.21.15.7:0/2636651611 wait complete. 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 Processor -- start 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 -- start start 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce0102360 0x7f5ce0101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 0x7f5ce0100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce01075c0 0x7f5ce0100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce010d390 con 0x7f5ce0104c90 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce010d210 con 0x7f5ce01075c0 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce77df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce010d510 con 0x7f5ce0102360 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce4d7a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 0x7f5ce0100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce4d7a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 0x7f5ce0100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35790/0 (socket says 172.21.15.7:35790) 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.429+0000 7f5ce4d7a640 1 -- 172.21.15.7:0/456369775 learned_addr learned my addr 172.21.15.7:0/456369775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce5d7c640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce01075c0 0x7f5ce0100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce557b640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce0102360 0x7f5ce0101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce4d7a640 1 -- 172.21.15.7:0/456369775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce0102360 msgr2=0x7f5ce0101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce4d7a640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce0102360 0x7f5ce0101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce4d7a640 1 -- 172.21.15.7:0/456369775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce01075c0 msgr2=0x7f5ce0100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce4d7a640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce01075c0 0x7f5ce0100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce4d7a640 1 -- 172.21.15.7:0/456369775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce01010e0 con 0x7f5ce0104c90 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce4d7a640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 0x7f5ce0100310 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400d740 tx=0x7f5cd400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4017070 con 0x7f5ce0104c90 2024-03-19T00:50:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cd400de50 con 0x7f5ce0104c90 2024-03-19T00:50:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01a1bb0 con 0x7f5ce0104c90 2024-03-19T00:50:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4004d60 con 0x7f5ce0104c90 2024-03-19T00:50:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.430+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a2100 con 0x7f5ce0104c90 2024-03-19T00:50:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.431+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce00fd7a0 con 0x7f5ce0104c90 2024-03-19T00:50:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.432+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f5cd4004f00 con 0x7f5ce0104c90 2024-03-19T00:50:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.433+0000 7f5cd27fc640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5ccc0a2880 0x7f5ccc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.433+0000 7f5ce557b640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5ccc0a2880 0x7f5ccc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.433+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9350+0+0 (secure 0 0 0) 0x7f5cd40ce770 con 0x7f5ce0104c90 2024-03-19T00:50:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.433+0000 7f5ce557b640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5ccc0a2880 0x7f5ccc0a4d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00bbe0 tx=0x7f5cdc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:07.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.438+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5cd409ad60 con 0x7f5ce0104c90 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: pgmap v716: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/613635158' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: Cluster is now healthy 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/613635158' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T00:50:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6836/3103327125,v1:172.21.15.44:6837/3103327125] up:boot 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6838/1032747252,v1:172.21.15.44:6839/1032747252] up:boot 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6837/3817583664,v1:172.21.15.204:6839/3817583664] up:boot 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6834/3016113595,v1:172.21.15.204:6835/3016113595] up:boot 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.44:6834/2814487487,v1:172.21.15.44:6835/2814487487] up:boot 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: mds.? [v2:172.21.15.204:6836/2384541144,v1:172.21.15.204:6838/2384541144] up:boot 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: fsmap 12 up:standby 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T00:50:07.596 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:07 smithi007 ceph-mon[62854]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T00:50:07.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:07.644+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ce0062420 con 0x7f5ce0104c90 2024-03-19T00:50:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: pgmap v716: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:50:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/613635158' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:50:07.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: Cluster is now healthy 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/613635158' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6836/3103327125,v1:172.21.15.44:6837/3103327125] up:boot 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6838/1032747252,v1:172.21.15.44:6839/1032747252] up:boot 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6837/3817583664,v1:172.21.15.204:6839/3817583664] up:boot 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6834/3016113595,v1:172.21.15.204:6835/3016113595] up:boot 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.44:6834/2814487487,v1:172.21.15.44:6835/2814487487] up:boot 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: mds.? [v2:172.21.15.204:6836/2384541144,v1:172.21.15.204:6838/2384541144] up:boot 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: fsmap 12 up:standby 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T00:50:07.857 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T00:50:07.858 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:07 smithi044 ceph-mon[66366]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T00:50:08.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: pgmap v716: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-19T00:50:08.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/613635158' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T00:50:08.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T00:50:08.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T00:50:08.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: Cluster is now healthy 2024-03-19T00:50:08.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/613635158' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6836/3103327125,v1:172.21.15.44:6837/3103327125] up:boot 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6838/1032747252,v1:172.21.15.44:6839/1032747252] up:boot 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6837/3817583664,v1:172.21.15.204:6839/3817583664] up:boot 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6834/3016113595,v1:172.21.15.204:6835/3016113595] up:boot 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.44:6834/2814487487,v1:172.21.15.44:6835/2814487487] up:boot 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: mds.? [v2:172.21.15.204:6836/2384541144,v1:172.21.15.204:6838/2384541144] up:boot 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: fsmap 12 up:standby 2024-03-19T00:50:08.018 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T00:50:08.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T00:50:08.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T00:50:08.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T00:50:08.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T00:50:08.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:07 smithi204 ceph-mon[66572]: from='mgr.14152 172.21.15.7:0/817177496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T00:50:08.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.529+0000 7f5cd27fc640 1 -- 172.21.15.7:0/456369775 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f5cd409ee00 con 0x7f5ce0104c90 2024-03-19T00:50:08.530 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-03-19T00:50:08.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.531+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5ccc0a2880 msgr2=0x7f5ccc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5ccc0a2880 0x7f5ccc0a4d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00bbe0 tx=0x7f5cdc00b390 comp rx=0 tx=0).stop 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 msgr2=0x7f5ce0100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 0x7f5ce0100310 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400d740 tx=0x7f5cd400dc00 comp rx=0 tx=0).stop 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 shutdown_connections 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f5ccc0a2880 0x7f5ccc0a4d40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce01075c0 0x7f5ce0100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ce0104c90 0x7f5ce0100310 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 --2- 172.21.15.7:0/456369775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ce0102360 0x7f5ce0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 >> 172.21.15.7:0/456369775 conn(0x7f5ce00fbcb0 msgr2=0x7f5ce0102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 shutdown_connections 2024-03-19T00:50:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.532+0000 7f5ce77df640 1 -- 172.21.15.7:0/456369775 wait complete. 2024-03-19T00:50:08.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 -- 172.21.15.7:0/959960142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628101920 msgr2=0x7f7628101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 --2- 172.21.15.7:0/959960142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628101920 0x7f7628101da0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7618009900 tx=0x7f761802bc30 comp rx=0 tx=0).stop 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 -- 172.21.15.7:0/959960142 shutdown_connections 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 --2- 172.21.15.7:0/959960142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 0x7f7628108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 --2- 172.21.15.7:0/959960142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628101920 0x7f7628101da0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 --2- 172.21.15.7:0/959960142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7628100730 0x7f7628100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 -- 172.21.15.7:0/959960142 >> 172.21.15.7:0/959960142 conn(0x7f76280fbc80 msgr2=0x7f76280fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 -- 172.21.15.7:0/959960142 shutdown_connections 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.654+0000 7f762f4f8640 1 -- 172.21.15.7:0/959960142 wait complete. 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 Processor -- start 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 -- start start 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 0x7f762806aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7628101920 0x7f762819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 0x7f762819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f762810c2c0 con 0x7f7628100730 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f762810c140 con 0x7f7628101920 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762f4f8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f762810c440 con 0x7f76281022e0 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 0x7f762806aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 0x7f762806aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35804/0 (socket says 172.21.15.7:35804) 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 -- 172.21.15.7:0/3987163993 learned_addr learned my addr 172.21.15.7:0/3987163993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762da95640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 0x7f762819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762ca93640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7628101920 0x7f762819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 -- 172.21.15.7:0/3987163993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 msgr2=0x7f762819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 0x7f762819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 -- 172.21.15.7:0/3987163993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7628101920 msgr2=0x7f762819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7628101920 0x7f762819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762d294640 1 -- 172.21.15.7:0/3987163993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762819f740 con 0x7f7628100730 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.655+0000 7f762da95640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 0x7f762819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.656+0000 7f762d294640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 0x7f762806aa50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f762400d770 tx=0x7f762400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.656+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624017070 con 0x7f7628100730 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.656+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7624009c30 con 0x7f7628100730 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.656+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762819f9d0 con 0x7f7628100730 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.656+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624003d00 con 0x7f7628100730 2024-03-19T00:50:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.656+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762819ff70 con 0x7f7628100730 2024-03-19T00:50:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.657+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628107af0 con 0x7f7628100730 2024-03-19T00:50:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.658+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f7624003ea0 con 0x7f7628100730 2024-03-19T00:50:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.658+0000 7f761e7fc640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76140a2800 0x7f76140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.659+0000 7f762ca93640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76140a2800 0x7f76140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.659+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8854+0+0 (secure 0 0 0) 0x7f76240cea60 con 0x7f7628100730 2024-03-19T00:50:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.659+0000 7f762ca93640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76140a2800 0x7f76140a4cc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7618009230 tx=0x7f7618005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.663+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f762409b830 con 0x7f7628100730 2024-03-19T00:50:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:08.846+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f76280624c0 con 0x7f7628100730 2024-03-19T00:50:08.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:08 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/456369775' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:08.964 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:08 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/456369775' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:09.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:08 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/456369775' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.530+0000 7f761e7fc640 1 -- 172.21.15.7:0/3987163993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f7624009de0 con 0x7f7628100730 2024-03-19T00:50:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-03-19T00:50:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76140a2800 msgr2=0x7f76140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76140a2800 0x7f76140a4cc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7618009230 tx=0x7f7618005fd0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 msgr2=0x7f762806aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 0x7f762806aa50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f762400d770 tx=0x7f762400dc30 comp rx=0 tx=0).stop 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 shutdown_connections 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f76140a2800 0x7f76140a4cc0 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7618009230 tx=0x7f7618005fd0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76281022e0 0x7f762819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7628101920 0x7f762819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 --2- 172.21.15.7:0/3987163993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7628100730 0x7f762806aa50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 >> 172.21.15.7:0/3987163993 conn(0x7f76280fbc80 msgr2=0x7f7628104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.535+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 shutdown_connections 2024-03-19T00:50:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.536+0000 7f762f4f8640 1 -- 172.21.15.7:0/3987163993 wait complete. 2024-03-19T00:50:09.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-19T00:50:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.676+0000 7f0bcb537640 1 -- 172.21.15.7:0/2146607437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc41022f0 msgr2=0x7f0bc4108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.676+0000 7f0bcb537640 1 --2- 172.21.15.7:0/2146607437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc41022f0 0x7f0bc4108b80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009ca0 tx=0x7f0bc002c0f0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- 172.21.15.7:0/2146607437 shutdown_connections 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 --2- 172.21.15.7:0/2146607437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc41022f0 0x7f0bc4108b80 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 --2- 172.21.15.7:0/2146607437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4101930 0x7f0bc4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 --2- 172.21.15.7:0/2146607437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bc4100740 0x7f0bc4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- 172.21.15.7:0/2146607437 >> 172.21.15.7:0/2146607437 conn(0x7f0bc40fbcb0 msgr2=0x7f0bc40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- 172.21.15.7:0/2146607437 shutdown_connections 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- 172.21.15.7:0/2146607437 wait complete. 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 Processor -- start 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- start start 2024-03-19T00:50:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 0x7f0bc4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4101930 0x7f0bc419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bc41022f0 0x7f0bc41a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410c250 con 0x7f0bc4100740 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410c0d0 con 0x7f0bc41022f0 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.677+0000 7f0bcb537640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410c3d0 con 0x7f0bc4101930 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 0x7f0bc4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 0x7f0bc4199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35834/0 (socket says 172.21.15.7:35834) 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 -- 172.21.15.7:0/3755907987 learned_addr learned my addr 172.21.15.7:0/3755907987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc9ad4640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bc41022f0 0x7f0bc41a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc8ad2640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4101930 0x7f0bc419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 -- 172.21.15.7:0/3755907987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4101930 msgr2=0x7f0bc419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4101930 0x7f0bc419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 -- 172.21.15.7:0/3755907987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bc41022f0 msgr2=0x7f0bc41a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bc41022f0 0x7f0bc41a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 -- 172.21.15.7:0/3755907987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41a17b0 con 0x7f0bc4100740 2024-03-19T00:50:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.678+0000 7f0bc92d3640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 0x7f0bc4199b90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00d950 tx=0x7f0bbc00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.679+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc017020 con 0x7f0bc4100740 2024-03-19T00:50:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.679+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bbc009c30 con 0x7f0bc4100740 2024-03-19T00:50:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.679+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc41a1aa0 con 0x7f0bc4100740 2024-03-19T00:50:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.679+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00f620 con 0x7f0bc4100740 2024-03-19T00:50:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.679+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a1ff0 con 0x7f0bc4100740 2024-03-19T00:50:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.680+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc41a2280 con 0x7f0bc4100740 2024-03-19T00:50:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.681+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137289+0+0 (secure 0 0 0) 0x7f0bbc01a020 con 0x7f0bc4100740 2024-03-19T00:50:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.681+0000 7f0bba7fc640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:50:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.681+0000 7f0bc8ad2640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:50:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.681+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8324+0+0 (secure 0 0 0) 0x7f0bbc0ce450 con 0x7f0bc4100740 2024-03-19T00:50:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.682+0000 7f0bc8ad2640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0bc419b0a0 tx=0x7f0bb400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:50:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.686+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0bbc021070 con 0x7f0bc4100740 2024-03-19T00:50:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:09 smithi044 ceph-mon[66366]: pgmap v718: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-03-19T00:50:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:09 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/456369775' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T00:50:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:09 smithi044 ceph-mon[66366]: osdmap e93: 12 total, 12 up, 12 in 2024-03-19T00:50:09.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:09 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3987163993' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:09.868+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0bc41a29a0 con 0x7f0bc4100740 2024-03-19T00:50:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:09 smithi007 ceph-mon[62854]: pgmap v718: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-03-19T00:50:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:09 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/456369775' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T00:50:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:09 smithi007 ceph-mon[62854]: osdmap e93: 12 total, 12 up, 12 in 2024-03-19T00:50:09.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:09 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3987163993' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:09 smithi204 ceph-mon[66572]: pgmap v718: 129 pgs: 129 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-03-19T00:50:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:09 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/456369775' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T00:50:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:09 smithi204 ceph-mon[66572]: osdmap e93: 12 total, 12 up, 12 in 2024-03-19T00:50:10.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:09 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3987163993' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:10.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.543+0000 7f0bba7fc640 1 -- 172.21.15.7:0/3755907987 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) ==== 178+0+0 (secure 0 0 0) 0x7f0bbc009de0 con 0x7f0bc4100740 2024-03-19T00:50:10.544 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0bb00a2800 msgr2=0x7f0bb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0bc419b0a0 tx=0x7f0bb400a2d0 comp rx=0 tx=0).stop 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 msgr2=0x7f0bc4199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 0x7f0bc4199b90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00d950 tx=0x7f0bbc00dd50 comp rx=0 tx=0).stop 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 shutdown_connections 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:6800/1790452807,v1:172.21.15.7:6801/1790452807] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bc41022f0 0x7f0bc41a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bc4101930 0x7f0bc419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 --2- 172.21.15.7:0/3755907987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc4100740 0x7f0bc4199b90 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:50:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 >> 172.21.15.7:0/3755907987 conn(0x7f0bc40fbcb0 msgr2=0x7f0bc40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:50:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 shutdown_connections 2024-03-19T00:50:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-19T00:50:10.547+0000 7f0bcb537640 1 -- 172.21.15.7:0/3755907987 wait complete. 2024-03-19T00:50:10.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:50:10.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:50:10.628 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:50:10.664 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-19T00:50:10.664 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T00:50:10.681 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T00:50:10.697 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T00:50:10.722 INFO:tasks.cephadm:Stopping all daemons... 2024-03-19T00:50:10.722 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-19T00:50:10.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a 2024-03-19T00:50:10.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:10 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3987163993' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T00:50:10.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:10 smithi044 ceph-mon[66366]: osdmap e94: 12 total, 12 up, 12 in 2024-03-19T00:50:10.856 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:10 smithi044 ceph-mon[66366]: from='client.? 172.21.15.7:0/3755907987' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3987163993' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T00:50:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:10 smithi007 ceph-mon[62854]: osdmap e94: 12 total, 12 up, 12 in 2024-03-19T00:50:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:10 smithi007 ceph-mon[62854]: from='client.? 172.21.15.7:0/3755907987' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:10.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:10 smithi007 systemd[1]: Stopping Ceph mon.a for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:11.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:10 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3987163993' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T00:50:11.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:10 smithi204 ceph-mon[66572]: osdmap e94: 12 total, 12 up, 12 in 2024-03-19T00:50:11.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:10 smithi204 ceph-mon[66572]: from='client.? 172.21.15.7:0/3755907987' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T00:50:11.444 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a[62850]: 2024-03-19T00:50:11.032+0000 7ff14f7f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:11.444 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a[62850]: 2024-03-19T00:50:11.032+0000 7ff14f7f7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-19T00:50:11.444 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:11 smithi007 podman[127749]: 2024-03-19 00:50:11.094864557 +0000 UTC m=+0.171297297 container died ffb69919e0d5cba101777691e5299ff97cd96cca7ce2dd2b96f8edeb9a8aec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-19T00:50:11.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:11 smithi007 podman[127749]: 2024-03-19 00:50:11.550808678 +0000 UTC m=+0.627241417 container cleanup ffb69919e0d5cba101777691e5299ff97cd96cca7ce2dd2b96f8edeb9a8aec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-19T00:50:11.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 19 00:50:11 smithi007 bash[127749]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-a 2024-03-19T00:50:11.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.a.service' 2024-03-19T00:50:12.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:12.030 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-19T00:50:12.030 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-19T00:50:12.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.b 2024-03-19T00:50:12.336 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:12 smithi044 systemd[1]: Stopping Ceph mon.b for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:12.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b[66362]: 2024-03-19T00:50:12.335+0000 7f737cfcf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:12.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b[66362]: 2024-03-19T00:50:12.335+0000 7f737cfcf700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-19T00:50:12.606 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:12 smithi044 podman[101035]: 2024-03-19 00:50:12.43746136 +0000 UTC m=+0.206983596 container died cbc1839b7815ff81225d8a184503f171452d4b35cb7d100de8f37abd5688c8b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:13.129 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:12 smithi044 podman[101035]: 2024-03-19 00:50:12.878002319 +0000 UTC m=+0.647524563 container cleanup cbc1839b7815ff81225d8a184503f171452d4b35cb7d100de8f37abd5688c8b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:50:13.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Mar 19 00:50:12 smithi044 bash[101035]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-b 2024-03-19T00:50:13.345 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.b.service' 2024-03-19T00:50:13.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:13.397 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-19T00:50:13.397 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-19T00:50:13.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.c 2024-03-19T00:50:13.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:13 smithi204 systemd[1]: Stopping Ceph mon.c for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-c[66566]: 2024-03-19T00:50:13.713+0000 7fdcb9068700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-c[66566]: 2024-03-19T00:50:13.713+0000 7fdcb9068700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-19T00:50:14.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:13 smithi204 podman[102745]: 2024-03-19 00:50:13.744964077 +0000 UTC m=+0.150186147 container died adb740de161f30ee8160ce8ef91ab211269111852e1275afc8680018951850be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:14.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:14 smithi204 podman[102745]: 2024-03-19 00:50:14.123183459 +0000 UTC m=+0.528405536 container cleanup adb740de161f30ee8160ce8ef91ab211269111852e1275afc8680018951850be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-19T00:50:14.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 19 00:50:14 smithi204 bash[102745]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mon-c 2024-03-19T00:50:14.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mon.c.service' 2024-03-19T00:50:14.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:14.642 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-19T00:50:14.642 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-19T00:50:14.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x 2024-03-19T00:50:14.956 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:50:14 smithi007 systemd[1]: Stopping Ceph mgr.x for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:15.214 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:50:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x[63088]: 2024-03-19T00:50:14.955+0000 7fc8fa3af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:15.215 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:50:14 smithi007 podman[127872]: 2024-03-19 00:50:14.986173018 +0000 UTC m=+0.147984501 container died d6d7ca8e28f62c602a66a70dab552d7668b6127aaadbd0cd1687ba161a7f4b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-19T00:50:15.715 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:50:15 smithi007 podman[127872]: 2024-03-19 00:50:15.489610945 +0000 UTC m=+0.651422435 container cleanup d6d7ca8e28f62c602a66a70dab552d7668b6127aaadbd0cd1687ba161a7f4b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-19T00:50:15.715 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 19 00:50:15 smithi007 bash[127872]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-x 2024-03-19T00:50:15.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-19T00:50:15.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:15.950 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-19T00:50:15.950 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-19T00:50:15.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.y 2024-03-19T00:50:16.297 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:50:15 smithi044 systemd[1]: Stopping Ceph mgr.y for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:16.602 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:50:16 smithi044 podman[101169]: 2024-03-19 00:50:16.295963682 +0000 UTC m=+0.152956213 container died bad536000e72beaddbfab2126649c4b249627488c3e76ab32f4c42fc89e233b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-19T00:50:16.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:50:16 smithi044 podman[101169]: 2024-03-19 00:50:16.692662468 +0000 UTC m=+0.549654992 container cleanup bad536000e72beaddbfab2126649c4b249627488c3e76ab32f4c42fc89e233b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True) 2024-03-19T00:50:16.856 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Mar 19 00:50:16 smithi044 bash[101169]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-y 2024-03-19T00:50:17.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-19T00:50:17.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:17.152 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-19T00:50:17.152 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-19T00:50:17.152 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.z 2024-03-19T00:50:17.492 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:50:17 smithi204 systemd[1]: Stopping Ceph mgr.z for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:17.765 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:50:17 smithi204 podman[102881]: 2024-03-19 00:50:17.491437861 +0000 UTC m=+0.155524409 container died 8ac1793993d0fbf0ad0855c9a609cf9b7e32894c12e92a57599656a6177fd95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:18.124 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:50:17 smithi204 podman[102881]: 2024-03-19 00:50:17.871034702 +0000 UTC m=+0.535121260 container cleanup 8ac1793993d0fbf0ad0855c9a609cf9b7e32894c12e92a57599656a6177fd95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:18.124 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 19 00:50:17 smithi204 bash[102881]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mgr-z 2024-03-19T00:50:18.288 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-19T00:50:18.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:18.328 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-19T00:50:18.328 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-19T00:50:18.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.0 2024-03-19T00:50:18.653 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:18 smithi007 systemd[1]: Stopping Ceph osd.0 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:18.965 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0[74813]: 2024-03-19T00:50:18.652+0000 7f77228cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:18.965 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0[74813]: 2024-03-19T00:50:18.652+0000 7f77228cc700 -1 osd.0 95 *** Got signal Terminated *** 2024-03-19T00:50:18.965 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0[74813]: 2024-03-19T00:50:18.652+0000 7f77228cc700 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:50:24.465 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:24 smithi007 podman[127991]: 2024-03-19 00:50:24.199293112 +0000 UTC m=+5.672151304 container died ddd238c9500fd8561b539d08a2cc88afdae194315ebc0d7ff5af7454f12ba4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:50:24.913 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:24 smithi007 podman[127991]: 2024-03-19 00:50:24.703230442 +0000 UTC m=+6.176088636 container cleanup ddd238c9500fd8561b539d08a2cc88afdae194315ebc0d7ff5af7454f12ba4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-19T00:50:24.913 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:24 smithi007 bash[127991]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0 2024-03-19T00:50:25.214 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:24 smithi007 podman[128009]: 2024-03-19 00:50:24.913110568 +0000 UTC m=+0.710486863 container remove ddd238c9500fd8561b539d08a2cc88afdae194315ebc0d7ff5af7454f12ba4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-19T00:50:25.495 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:25 smithi007 podman[128108]: 2024-03-19 00:50:25.246039064 +0000 UTC m=+0.026138400 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:50:25.495 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:25 smithi007 podman[128108]: 2024-03-19 00:50:25.36442693 +0000 UTC m=+0.144526246 container create c028d2179aa4dfdb547a5bcad5e8b19ad521410ef07c88d6812adce3124c8d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-19T00:50:25.938 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:25 smithi007 podman[128108]: 2024-03-19 00:50:25.635626297 +0000 UTC m=+0.415725629 container init c028d2179aa4dfdb547a5bcad5e8b19ad521410ef07c88d6812adce3124c8d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:25.938 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:25 smithi007 podman[128108]: 2024-03-19 00:50:25.639321647 +0000 UTC m=+0.419420969 container start c028d2179aa4dfdb547a5bcad5e8b19ad521410ef07c88d6812adce3124c8d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-19T00:50:25.938 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:25 smithi007 podman[128108]: 2024-03-19 00:50:25.742507356 +0000 UTC m=+0.522606682 container attach c028d2179aa4dfdb547a5bcad5e8b19ad521410ef07c88d6812adce3124c8d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-19T00:50:26.215 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 19 00:50:25 smithi007 podman[128108]: 2024-03-19 00:50:25.937825971 +0000 UTC m=+0.717925297 container died c028d2179aa4dfdb547a5bcad5e8b19ad521410ef07c88d6812adce3124c8d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:26.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.0.service' 2024-03-19T00:50:26.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:26.784 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-19T00:50:26.784 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-19T00:50:26.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.1 2024-03-19T00:50:27.106 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:26 smithi044 systemd[1]: Stopping Ceph osd.1 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:27.606 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1[70945]: 2024-03-19T00:50:27.113+0000 7fca3456d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:27.607 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1[70945]: 2024-03-19T00:50:27.113+0000 7fca3456d700 -1 osd.1 95 *** Got signal Terminated *** 2024-03-19T00:50:27.607 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1[70945]: 2024-03-19T00:50:27.113+0000 7fca3456d700 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:50:32.842 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:32 smithi044 podman[101294]: 2024-03-19 00:50:32.575464851 +0000 UTC m=+5.583349193 container died 5d26cf42c6dbca40a74234560759ccdadd5b4e348f2a7a240b4903ebd8733b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-19T00:50:33.106 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:32 smithi044 podman[101294]: 2024-03-19 00:50:32.931758516 +0000 UTC m=+5.939642862 container cleanup 5d26cf42c6dbca40a74234560759ccdadd5b4e348f2a7a240b4903ebd8733b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:50:33.106 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:32 smithi044 bash[101294]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1 2024-03-19T00:50:33.599 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:33 smithi044 podman[101311]: 2024-03-19 00:50:33.241511546 +0000 UTC m=+0.661590591 container remove 5d26cf42c6dbca40a74234560759ccdadd5b4e348f2a7a240b4903ebd8733b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:50:33.856 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:33 smithi044 podman[101403]: 2024-03-19 00:50:33.501300535 +0000 UTC m=+0.018302162 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:50:33.856 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:33 smithi044 podman[101403]: 2024-03-19 00:50:33.625970753 +0000 UTC m=+0.142972395 container create c808a7683ce128f59454793bf38a06faf930d7a5e3a2fd81ce7dffeb89e298ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-19T00:50:34.199 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:33 smithi044 podman[101403]: 2024-03-19 00:50:33.889126804 +0000 UTC m=+0.406128423 container init c808a7683ce128f59454793bf38a06faf930d7a5e3a2fd81ce7dffeb89e298ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-19T00:50:34.200 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:33 smithi044 podman[101403]: 2024-03-19 00:50:33.892430356 +0000 UTC m=+0.409431960 container start c808a7683ce128f59454793bf38a06faf930d7a5e3a2fd81ce7dffeb89e298ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:34.200 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:34 smithi044 podman[101403]: 2024-03-19 00:50:34.004288812 +0000 UTC m=+0.521290422 container attach c808a7683ce128f59454793bf38a06faf930d7a5e3a2fd81ce7dffeb89e298ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-19T00:50:34.478 INFO:journalctl@ceph.osd.1.smithi044.stdout:Mar 19 00:50:34 smithi044 podman[101403]: 2024-03-19 00:50:34.199806365 +0000 UTC m=+0.716807966 container died c808a7683ce128f59454793bf38a06faf930d7a5e3a2fd81ce7dffeb89e298ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-19T00:50:34.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.1.service' 2024-03-19T00:50:34.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:34.960 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-19T00:50:34.960 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-19T00:50:34.960 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.2 2024-03-19T00:50:35.265 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:35 smithi204 systemd[1]: Stopping Ceph osd.2 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:35.765 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2[71738]: 2024-03-19T00:50:35.274+0000 7f3d77958700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:35.765 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2[71738]: 2024-03-19T00:50:35.274+0000 7f3d77958700 -1 osd.2 95 *** Got signal Terminated *** 2024-03-19T00:50:35.765 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2[71738]: 2024-03-19T00:50:35.274+0000 7f3d77958700 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:50:40.892 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:40 smithi204 podman[103005]: 2024-03-19 00:50:40.624126306 +0000 UTC m=+5.465598082 container died 4c813985425b4384f125ea61c5ee6851160a62de435a912cc47b8c59654e65f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-19T00:50:41.185 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:40 smithi204 podman[103005]: 2024-03-19 00:50:40.892298228 +0000 UTC m=+5.733769993 container cleanup 4c813985425b4384f125ea61c5ee6851160a62de435a912cc47b8c59654e65f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-19T00:50:41.185 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:40 smithi204 bash[103005]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2 2024-03-19T00:50:41.513 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:41 smithi204 podman[103022]: 2024-03-19 00:50:41.185406582 +0000 UTC m=+0.556590882 container remove 4c813985425b4384f125ea61c5ee6851160a62de435a912cc47b8c59654e65f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-19T00:50:41.765 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:41 smithi204 podman[103113]: 2024-03-19 00:50:41.415583991 +0000 UTC m=+0.017797779 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:50:41.765 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:41 smithi204 podman[103113]: 2024-03-19 00:50:41.562585815 +0000 UTC m=+0.164799645 container create 52cdc0cee55b98a22f9495d57dffd64b8b283011bc9a38994922a2fff3e9e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:50:42.118 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:41 smithi204 podman[103113]: 2024-03-19 00:50:41.8234599 +0000 UTC m=+0.425673693 container init 52cdc0cee55b98a22f9495d57dffd64b8b283011bc9a38994922a2fff3e9e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:50:42.118 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:41 smithi204 podman[103113]: 2024-03-19 00:50:41.826667683 +0000 UTC m=+0.428881475 container start 52cdc0cee55b98a22f9495d57dffd64b8b283011bc9a38994922a2fff3e9e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-03-19T00:50:42.118 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:41 smithi204 podman[103113]: 2024-03-19 00:50:41.897862928 +0000 UTC m=+0.500076714 container attach 52cdc0cee55b98a22f9495d57dffd64b8b283011bc9a38994922a2fff3e9e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-19T00:50:42.393 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 19 00:50:42 smithi204 podman[103113]: 2024-03-19 00:50:42.118154827 +0000 UTC m=+0.720368618 container died 52cdc0cee55b98a22f9495d57dffd64b8b283011bc9a38994922a2fff3e9e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-19T00:50:42.702 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.2.service' 2024-03-19T00:50:42.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:42.744 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-19T00:50:42.744 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-19T00:50:42.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.3 2024-03-19T00:50:43.075 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:42 smithi007 systemd[1]: Stopping Ceph osd.3 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:43.465 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:43 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3[81603]: 2024-03-19T00:50:43.073+0000 7fbf14dca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:43.465 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:43 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3[81603]: 2024-03-19T00:50:43.073+0000 7fbf14dca700 -1 osd.3 95 *** Got signal Terminated *** 2024-03-19T00:50:43.465 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:43 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3[81603]: 2024-03-19T00:50:43.073+0000 7fbf14dca700 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:50:45.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:45 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:45.506+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:46.106 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:46 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:46.009+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:46.469 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:46.152+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:46.469 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:46.280+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:46.714 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:46 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:46.461+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:47.106 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:46 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:46.988+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:47.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:47.137+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:47.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:47.235+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:47.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:47 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:47.416+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:48.215 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:47 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3[81603]: 2024-03-19T00:50:47.854+0000 7fbf0d3b6700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.394302+0000 front 2024-03-19T00:50:22.394279+0000 (oldest deadline 2024-03-19T00:50:47.694479+0000) 2024-03-19T00:50:48.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:48 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:48.012+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:48.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:48.277+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:48.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:48.171+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:48.658 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:48 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:48.463+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:48.658 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:48 smithi007 podman[128220]: 2024-03-19 00:50:48.397551313 +0000 UTC m=+5.454145575 container died 7b1e09cedb53b4df748c93cd235e08b6454315811262cb57ca3190388ee27c8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-19T00:50:48.965 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:48 smithi007 podman[128220]: 2024-03-19 00:50:48.748658765 +0000 UTC m=+5.805253035 container cleanup 7b1e09cedb53b4df748c93cd235e08b6454315811262cb57ca3190388ee27c8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-19T00:50:48.965 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:48 smithi007 bash[128220]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3 2024-03-19T00:50:49.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:49 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:49.001+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:49.375 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128235]: 2024-03-19 00:50:49.025008287 +0000 UTC m=+0.630965153 container remove 7b1e09cedb53b4df748c93cd235e08b6454315811262cb57ca3190388ee27c8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-19T00:50:49.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:49.128+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:49.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:49.272+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:49.653 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:49 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:49.468+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:49.654 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128328]: 2024-03-19 00:50:49.277865025 +0000 UTC m=+0.017669904 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:50:49.654 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128328]: 2024-03-19 00:50:49.392643742 +0000 UTC m=+0.132448594 container create 602619e3283037402c6c6c029fc7f2ff77cc9642f7da5722a9823b6ade416364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-19T00:50:49.654 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128328]: 2024-03-19 00:50:49.653761551 +0000 UTC m=+0.393566419 container init 602619e3283037402c6c6c029fc7f2ff77cc9642f7da5722a9823b6ade416364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-19T00:50:49.953 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128328]: 2024-03-19 00:50:49.657197849 +0000 UTC m=+0.397002708 container start 602619e3283037402c6c6c029fc7f2ff77cc9642f7da5722a9823b6ade416364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:50:49.953 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128328]: 2024-03-19 00:50:49.73709305 +0000 UTC m=+0.476897930 container attach 602619e3283037402c6c6c029fc7f2ff77cc9642f7da5722a9823b6ade416364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-19T00:50:50.214 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 19 00:50:49 smithi007 podman[128328]: 2024-03-19 00:50:49.953460372 +0000 UTC m=+0.693265236 container died 602619e3283037402c6c6c029fc7f2ff77cc9642f7da5722a9823b6ade416364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-19T00:50:50.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:49 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:49.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:50.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:50.091+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:50.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:50.227+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:50.584 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:50 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:50.505+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:50.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.3.service' 2024-03-19T00:50:50.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:50.655 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-19T00:50:50.655 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-19T00:50:50.655 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.4 2024-03-19T00:50:50.971 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:50 smithi044 systemd[1]: Stopping Ceph osd.4 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:51.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:50 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:50.974+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:51.357 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:50 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:50.970+0000 7f126e783700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:51.357 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:50 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:50.970+0000 7f126e783700 -1 osd.4 95 *** Got signal Terminated *** 2024-03-19T00:50:51.357 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:50 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:50.970+0000 7f126e783700 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:50:51.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:51.112+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:51.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:51.273+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:51.838 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:51 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:51.486+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:52.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:52 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:52.020+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:52.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:52.092+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:52.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:52.263+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:52.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:52 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:52.459+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:53.082 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:52 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:52.780+0000 7f1266d6f700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.963530+0000 front 2024-03-19T00:50:30.963576+0000 (oldest deadline 2024-03-19T00:50:52.663688+0000) 2024-03-19T00:50:53.083 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:53 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:53.023+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:53.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:53 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:53.082+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:53.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:53 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:53.065+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:53.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:53 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:53.252+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:53.745 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:53.479+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:54.106 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:54 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:54.035+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:54.106 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:53 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:53.825+0000 7f1266d6f700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.963530+0000 front 2024-03-19T00:50:30.963576+0000 (oldest deadline 2024-03-19T00:50:52.663688+0000) 2024-03-19T00:50:54.107 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:54 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:54.060+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:54.107 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:54 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:54.060+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:50:54.214 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:53 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:53.745+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:50:54.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:54 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:54.053+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:54.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:54 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:54.212+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:54.754 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:54 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:54.477+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:55.106 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:55 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:55.053+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:55.106 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:54 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:54.828+0000 7f1266d6f700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.963530+0000 front 2024-03-19T00:50:30.963576+0000 (oldest deadline 2024-03-19T00:50:52.663688+0000) 2024-03-19T00:50:55.107 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:55 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:55.042+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:55.107 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:55 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:55.042+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:50:55.215 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:54 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:54.753+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:50:55.456 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:55 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:55.043+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:55.456 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:55 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:55.206+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:55.457 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:55 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:55.206+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:50:55.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:50:55 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:50:55.455+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:50:55.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:55.482+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:55.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:55.482+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:50:55.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:55.706+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:50:56.069 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:55 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4[77892]: 2024-03-19T00:50:55.816+0000 7f1266d6f700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.963530+0000 front 2024-03-19T00:50:30.963576+0000 (oldest deadline 2024-03-19T00:50:52.663688+0000) 2024-03-19T00:50:56.069 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:56 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:56.063+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:56.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:56 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:56.068+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:56.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:56 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:56.068+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:50:56.439 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:56 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:56.256+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:56.439 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:56 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:56.256+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:50:56.439 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:56 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:56.054+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:56.439 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:56 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:56.054+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:50:56.655 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:56 smithi044 podman[101534]: 2024-03-19 00:50:56.397450005 +0000 UTC m=+5.546214045 container died d20d6a76da49c71601127d597c5e7b8ebf26488faab3e19d8c1ebea80d9d5e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:56.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:50:56 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:50:56.438+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:50:56.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:56 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:56.723+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:50:56.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:56 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:56.483+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:56.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:56 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:56.483+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:50:57.047 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:56 smithi044 podman[101534]: 2024-03-19 00:50:56.746172919 +0000 UTC m=+5.894936958 container cleanup d20d6a76da49c71601127d597c5e7b8ebf26488faab3e19d8c1ebea80d9d5e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T00:50:57.047 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:56 smithi044 bash[101534]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4 2024-03-19T00:50:57.356 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:57 smithi044 podman[101551]: 2024-03-19 00:50:57.16454394 +0000 UTC m=+0.770648781 container remove d20d6a76da49c71601127d597c5e7b8ebf26488faab3e19d8c1ebea80d9d5e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-19T00:50:57.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:57 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:57.103+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:57.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:57 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:57.045+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:57.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:57 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:57.045+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:50:57.459 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:57 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:57.281+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:57.459 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:57 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:57.281+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:50:57.459 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:57 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:57.100+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:57.459 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:57 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:57.100+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:50:57.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:50:57 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:50:57.458+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:50:57.801 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:57 smithi044 podman[101649]: 2024-03-19 00:50:57.419452412 +0000 UTC m=+0.017237990 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:50:57.801 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:57 smithi044 podman[101649]: 2024-03-19 00:50:57.523762277 +0000 UTC m=+0.121547849 container create c3baf6c479ce748fba92f6ed4116814f30651722b563390c7d4779fca8680c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-19T00:50:57.801 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:57 smithi044 podman[101649]: 2024-03-19 00:50:57.801302322 +0000 UTC m=+0.399087894 container init c3baf6c479ce748fba92f6ed4116814f30651722b563390c7d4779fca8680c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:50:57.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:57 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:57.706+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:50:57.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:57 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:57.470+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:57.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:57 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:57.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:50:58.082 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:57 smithi044 podman[101649]: 2024-03-19 00:50:57.8044069 +0000 UTC m=+0.402192472 container start c3baf6c479ce748fba92f6ed4116814f30651722b563390c7d4779fca8680c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:50:58.082 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:57 smithi044 podman[101649]: 2024-03-19 00:50:57.901803964 +0000 UTC m=+0.499589552 container attach c3baf6c479ce748fba92f6ed4116814f30651722b563390c7d4779fca8680c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-19T00:50:58.357 INFO:journalctl@ceph.osd.4.smithi044.stdout:Mar 19 00:50:58 smithi044 podman[101649]: 2024-03-19 00:50:58.09975405 +0000 UTC m=+0.697539622 container died c3baf6c479ce748fba92f6ed4116814f30651722b563390c7d4779fca8680c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:50:58.358 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:58 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:58.142+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:58.358 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:58 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:58.080+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:58.358 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:58 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:58.080+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:50:58.426 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:58 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:58.300+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:58.427 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:58 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:58.300+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:50:58.427 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:58 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:58.066+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:58.427 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:58 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:58.066+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:50:58.717 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.4.service' 2024-03-19T00:50:58.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:50:58.761 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-19T00:50:58.761 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-19T00:50:58.761 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.5 2024-03-19T00:50:58.764 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:50:58 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:50:58.425+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:50:58.964 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:58.736+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:50:58.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:58.497+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:58.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:58.497+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:50:59.054 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:58 smithi204 systemd[1]: Stopping Ceph osd.5 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:50:59.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:50:59 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:50:59.159+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:50:59.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:59 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:59.043+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:50:59.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:50:59 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:50:59.043+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:50:59.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:59.275+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:50:59.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:50:59.275+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:50:59.394 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:59.053+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:50:59.395 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:59.053+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:50:59.395 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:59.255+0000 7ff13efe5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:50:59.395 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:59.255+0000 7ff13efe5700 -1 osd.5 95 *** Got signal Terminated *** 2024-03-19T00:50:59.395 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:50:59.255+0000 7ff13efe5700 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:50:59.746 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:59 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:59.492+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:50:59.747 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:50:59 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:50:59.492+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:50:59.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:50:59 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:50:59.393+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:00.214 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:50:59 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:50:59.746+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:00.302 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:00 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:00.039+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:51:00.302 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:00 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:00.039+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:51:00.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:00 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:00.147+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:00.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:00 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:00.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:00.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:00 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:00.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:00.715 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:00 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:00.701+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:00.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:00 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:00.456+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:00.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:00 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:00.456+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:00.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:00 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:00.368+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:00.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:00 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:00.301+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:00.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:00 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:00.301+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:01.265 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:01 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:01.011+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:51:01.265 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:01 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:01.011+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:51:01.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:01 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:01.128+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:01.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:01 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:01.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:01.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:01 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:01.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:01.714 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:01 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:01.408+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:01.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:01 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:01.408+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:01.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:01 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:01.408+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:01.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:01 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:01.321+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:01.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:01 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:01.321+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:02.215 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:01 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:01.726+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:02.215 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:01 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:01.726+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:02.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:02 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:02.046+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:51:02.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:02 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:02.046+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:51:02.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:02 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:02.130+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:02.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:02 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:02.130+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:02.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:01 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:01.981+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:02.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:01 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:01.981+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:02.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:01 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:01.981+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:02.710 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:02 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:02.404+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:02.711 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:02 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:02.404+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:02.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:02 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:02.400+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:02.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:02 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:02.338+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:02.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:02 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:02.338+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:02.964 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:02 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:02.709+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:02.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:02 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:02.709+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:03.315 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:03.035+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:51:03.316 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:03.035+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:51:03.316 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:03.035+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.029291+0000 front 2024-03-19T00:50:38.029224+0000 (oldest deadline 2024-03-19T00:51:02.729416+0000) 2024-03-19T00:51:03.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:03 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:03.134+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:03.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:03 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:03.134+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:03.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:02 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:02.949+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:03.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:02 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:02.949+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:03.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:02 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:02.949+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:03.714 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:03 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:03.671+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:03.715 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:03 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:03.671+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:03.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:03 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:03.426+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:03.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:03 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:03.426+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:03.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:03 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:03.426+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:03.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:03.386+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:03.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:03.314+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:03.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:03.314+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:03.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:03 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:03.314+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:04.354 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:04.057+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:20.526461+0000 front 2024-03-19T00:50:20.526619+0000 (oldest deadline 2024-03-19T00:50:45.226711+0000) 2024-03-19T00:51:04.354 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:04.057+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.327962+0000 front 2024-03-19T00:50:29.328095+0000 (oldest deadline 2024-03-19T00:50:55.228017+0000) 2024-03-19T00:51:04.354 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5[78673]: 2024-03-19T00:51:04.057+0000 7ff1375d1700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.029291+0000 front 2024-03-19T00:50:38.029224+0000 (oldest deadline 2024-03-19T00:51:02.729416+0000) 2024-03-19T00:51:04.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:04 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:04.085+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:04.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:04 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:04.085+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:04.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:03 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:03.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:04.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:03 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:03.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:04.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:03 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:03.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:04.733 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:04.420+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:04.733 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:04.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:04.733 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:04.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:04.733 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:04 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:04.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:04.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:04 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:04.693+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:04.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:04 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:04.693+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:04.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:04 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:04.473+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:04.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:04 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:04.473+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:04.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:04 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:04.473+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:05.015 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:04 smithi204 podman[103229]: 2024-03-19 00:51:04.732606606 +0000 UTC m=+5.782882658 container died da4399545a8d9e1e2cf8e3dfe12a0ebe54098cf10af8528ed5d34b26cf9c5808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-19T00:51:05.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:05 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:05.099+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:05.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:05 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:05.099+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:05.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:04 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:04.926+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:05.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:04 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:04.926+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:05.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:04 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:04.926+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:05.365 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 podman[103229]: 2024-03-19 00:51:05.114518328 +0000 UTC m=+6.164794371 container cleanup da4399545a8d9e1e2cf8e3dfe12a0ebe54098cf10af8528ed5d34b26cf9c5808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:05.366 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 bash[103229]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5 2024-03-19T00:51:05.675 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 podman[103246]: 2024-03-19 00:51:05.365795524 +0000 UTC m=+0.635075576 container remove da4399545a8d9e1e2cf8e3dfe12a0ebe54098cf10af8528ed5d34b26cf9c5808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T00:51:05.675 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:05.383+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:05.676 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:05.383+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:05.676 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:05.373+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:05.676 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:05.373+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:05.676 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:05 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:05.373+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:05.706 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:05 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:05.497+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:05.706 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:05 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:05.497+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:05.706 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:05 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:05.497+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:05.930 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 podman[103339]: 2024-03-19 00:51:05.675176816 +0000 UTC m=+0.106255215 container create 0b9d82a279db39efa3f402a968b581d0b8055c2d7dbe9a7d6a71cf5b13fee1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:51:05.930 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 podman[103339]: 2024-03-19 00:51:05.586732348 +0000 UTC m=+0.017810749 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:05.930 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 podman[103339]: 2024-03-19 00:51:05.929783191 +0000 UTC m=+0.360861597 container init 0b9d82a279db39efa3f402a968b581d0b8055c2d7dbe9a7d6a71cf5b13fee1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:05.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:05 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:05.705+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:05.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:05 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:05.705+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:06.225 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:05 smithi204 podman[103339]: 2024-03-19 00:51:05.933185763 +0000 UTC m=+0.364264160 container start 0b9d82a279db39efa3f402a968b581d0b8055c2d7dbe9a7d6a71cf5b13fee1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:06.225 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:06 smithi204 podman[103339]: 2024-03-19 00:51:06.021755247 +0000 UTC m=+0.452833654 container attach 0b9d82a279db39efa3f402a968b581d0b8055c2d7dbe9a7d6a71cf5b13fee1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5) 2024-03-19T00:51:06.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:06.142+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:06.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:06.142+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:06.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:05 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:05.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:06.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:05 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:05.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:06.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:05 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:05.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:06.515 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 19 00:51:06 smithi204 podman[103339]: 2024-03-19 00:51:06.225703119 +0000 UTC m=+0.656781515 container died 0b9d82a279db39efa3f402a968b581d0b8055c2d7dbe9a7d6a71cf5b13fee1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T00:51:06.516 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:06.390+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:06.516 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:06.390+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:06.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:06.418+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:06.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:06.418+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:06.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:06 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:06.418+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:06.723 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:06.472+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:06.724 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:06.472+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:06.724 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:06.472+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:06.908 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.5.service' 2024-03-19T00:51:06.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:06.994 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-19T00:51:06.994 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-19T00:51:06.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.6 2024-03-19T00:51:07.024 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:06.723+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:07.024 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:06 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:06.723+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:07.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:07.171+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:07.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:07.171+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:07.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:06.947+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:07.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:06.947+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:07.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:06 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:06.947+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:07.428 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:07 smithi007 systemd[1]: Stopping Ceph osd.6 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:07.697 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:07.427+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:07.697 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:07.427+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:07.697 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:07.427+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:07.697 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:07.434+0000 7fb585d8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:07.697 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:07.434+0000 7fb585d8f700 -1 osd.6 95 *** Got signal Terminated *** 2024-03-19T00:51:07.698 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:07.434+0000 7fb585d8f700 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:51:07.698 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:07.696+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:07.698 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:07.696+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:07.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:07.384+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:07.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:07.384+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:07.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:07.384+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:07.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:07.428+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:07.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:07 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:07.428+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:08.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:08.180+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:08.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:08.180+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:08.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:07.950+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:08.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:07.950+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:08.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:07 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:07.950+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:08.714 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:08 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:08.435+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:08.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:08 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:08.435+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:08.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:08 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:08.435+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:08.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:08.438+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:08.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:08.438+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:08.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:08.375+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:08.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:08.375+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:08.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:08 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:08.375+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:09.199 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:08.918+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:09.199 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:08.918+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:09.199 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:08 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:08.918+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:09.214 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:08 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:08.727+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:09.215 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:08 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:08.727+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:09.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:09.197+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:09.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:09.197+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:09.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:09.459+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:09.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:09.459+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:09.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:09.416+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:09.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:09.416+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:09.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:09 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:09.416+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:09.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:09 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:09.705+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:09.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:09 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:09.705+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:09.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:09 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:09.475+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:09.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:09 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:09.475+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:09.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:09 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:09.475+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:10.233 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:09.944+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:10.233 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:09.944+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:10.233 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:09.944+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:10.234 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:09 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:09.944+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:10.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:10.232+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:10.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:10.232+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:10.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:10.232+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:10.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:10.382+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:10.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:10.382+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:10.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:10.382+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:10.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:10.493+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:10.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:10.493+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:10.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:10 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:10.493+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:10.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:10.739+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:10.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:10.739+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:10.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:10.518+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:10.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:10.518+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:10.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:10.518+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:10.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:10 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:10.518+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:11.213 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:10.958+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:11.213 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:10.958+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:11.213 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:10.958+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:11.213 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:10 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:10.958+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:11.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:11.212+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:11.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:11.212+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:11.606 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:11.212+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:11.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:11.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:11.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:11.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:11.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:11.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:11.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:11.455+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:11.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:11.455+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:11.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:11 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:11.455+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:11.708+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.082535+0000 front 2024-03-19T00:50:30.082572+0000 (oldest deadline 2024-03-19T00:50:52.882913+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:11.708+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:35.784048+0000 front 2024-03-19T00:50:35.784070+0000 (oldest deadline 2024-03-19T00:51:01.684021+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6[88317]: 2024-03-19T00:51:11.708+0000 7fb57e37b700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:46.385088+0000 front 2024-03-19T00:50:46.385188+0000 (oldest deadline 2024-03-19T00:51:11.685243+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:11.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:11.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:11.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:11.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:11 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:11.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:12.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:12.203+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:12.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:12.203+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:12.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:12.203+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:12.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:11.993+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:12.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:11.993+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:12.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:11.993+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:12.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:11.993+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:12.737 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:12.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:12.737 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:12.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:12.737 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:12.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:12.737 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:12 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:12.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:12.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:12 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:12.354+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:12.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:12 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:12.354+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:12.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:12 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:12.354+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:12.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:12 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:12.418+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:12.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:12 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:12.418+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:12.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:12 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:12.418+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:13.128 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:12 smithi007 podman[128444]: 2024-03-19 00:51:12.736468457 +0000 UTC m=+5.539542239 container died 94df4489e41e676cca75dbe0c1b5178b2c6495e27a58df0f0ec3be0a45736fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:13.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:13.200+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:13.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:13.200+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:13.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:13.200+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:13.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:12.974+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:13.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:12.974+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:13.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:12.974+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:13.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:12 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:12.974+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:13.460 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:13 smithi007 podman[128444]: 2024-03-19 00:51:13.218009481 +0000 UTC m=+6.021083262 container cleanup 94df4489e41e676cca75dbe0c1b5178b2c6495e27a58df0f0ec3be0a45736fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-19T00:51:13.461 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:13 smithi007 bash[128444]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6 2024-03-19T00:51:13.715 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:13 smithi007 podman[128461]: 2024-03-19 00:51:13.460778339 +0000 UTC m=+0.718691290 container remove 94df4489e41e676cca75dbe0c1b5178b2c6495e27a58df0f0ec3be0a45736fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-19T00:51:13.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:13.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:13.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:13.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:13.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:13.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:13.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:13 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:13.471+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:13.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:13.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:13.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:13.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:13.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:13.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:13.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:13.387+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:13.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:13.387+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:13.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:13 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:13.387+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:13 smithi007 podman[128550]: 2024-03-19 00:51:13.687387699 +0000 UTC m=+0.018256515 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:13 smithi007 podman[128550]: 2024-03-19 00:51:13.820148632 +0000 UTC m=+0.151017431 container create f880331059ad88b9ca771418611e8c5b873a5dbbd00aef777f88a33312f228e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-19T00:51:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:14 smithi007 podman[128550]: 2024-03-19 00:51:14.005955187 +0000 UTC m=+0.336823988 container init f880331059ad88b9ca771418611e8c5b873a5dbbd00aef777f88a33312f228e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:51:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:14 smithi007 podman[128550]: 2024-03-19 00:51:14.009148241 +0000 UTC m=+0.340017041 container start f880331059ad88b9ca771418611e8c5b873a5dbbd00aef777f88a33312f228e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-19T00:51:14.081 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:14 smithi007 podman[128550]: 2024-03-19 00:51:14.080867352 +0000 UTC m=+0.411736167 container attach f880331059ad88b9ca771418611e8c5b873a5dbbd00aef777f88a33312f228e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD) 2024-03-19T00:51:14.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:14 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:14.158+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:14.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:14 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:14.158+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:14.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:14 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:14.158+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:14.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:13.987+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:14.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:13.987+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:14.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:13.987+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:14.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:13 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:13.987+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:14.465 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 19 00:51:14 smithi007 podman[128550]: 2024-03-19 00:51:14.30908993 +0000 UTC m=+0.639958731 container died f880331059ad88b9ca771418611e8c5b873a5dbbd00aef777f88a33312f228e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-03-19T00:51:14.751 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:14.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:14.751 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:14.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:14.752 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:14.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:14.752 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:14 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:14.474+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:14.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:14 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:14.358+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:14.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:14 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:14.358+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:14.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:14 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:14.358+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:14.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:14 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:14.353+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:14.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:14 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:14.353+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:14.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:14 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:14.353+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:15.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.6.service' 2024-03-19T00:51:15.135 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:15.135 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-19T00:51:15.135 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-19T00:51:15.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.7 2024-03-19T00:51:15.270 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:15.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:15.270 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:15.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:15.271 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:15.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:15.271 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:15.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:15.271 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:15.110+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:15.271 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:15.110+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:15.271 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:15.110+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:15.271 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 systemd[1]: Stopping Ceph osd.7 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:15.535 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:15.424+0000 7f6cab230700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:15.535 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:15.424+0000 7f6cab230700 -1 osd.7 95 *** Got signal Terminated *** 2024-03-19T00:51:15.535 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:15.424+0000 7f6cab230700 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:51:15.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:15 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:15.318+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:15.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:15 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:15.318+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:15.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:15 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:15.318+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:15.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:15 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:15.387+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:15.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:15 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:15.387+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:15.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:15 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:15.387+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:15.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:15.481+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:15.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:15.481+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:15.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:15.481+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:15.965 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:15 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:15.481+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:16.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:16.111+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:16.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:16.111+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:16.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:16.111+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:16.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:16.054+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:16.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:16.054+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:16.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:16.054+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:16.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:16 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:16.054+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:16.666 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:16.401+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:16.666 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:16.401+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:16.666 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:16.401+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:16.666 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:16.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:16.667 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:16.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:16.667 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:16 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:16.353+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:16.714 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:16.439+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:16.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:16.439+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:16.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:16.439+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:16.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:16 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:16.439+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:17.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:17.071+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:17.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:17.071+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:17.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:17.071+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:17.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:17.071+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:17.357 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:17.099+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:17.357 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:17.099+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:17.357 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:17 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:17.099+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:17.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:17 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:17.407+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:17.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:17 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:17.407+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:17.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:17 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:17.407+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:17.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:17 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:17.407+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:17.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:17 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:17.439+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:17.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:17 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:17.439+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:17.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:17 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:17.439+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:17.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:17 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:17.303+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:17.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:17 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:17.303+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:17.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:17 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:17.303+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:18.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:18.050+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:18.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:18.050+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:18.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:18.050+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:18.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:18.074+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:18.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:18.074+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:18.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:18.074+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:18.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:18.074+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:18.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:18 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:18.074+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:18.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:18.408+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:18.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:18.408+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:18.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:18.408+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:18.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:18 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:18.408+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:18.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:18 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:18.456+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:18.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:18 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:18.456+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:18.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:18 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:18.456+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:18.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:18 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:18.328+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:18.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:18 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:18.328+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:18.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:18 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:18.328+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:19.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:19.093+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:19.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:19.093+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:19.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:19.093+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:19.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:19.093+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.122237+0000 front 2024-03-19T00:50:53.122308+0000 (oldest deadline 2024-03-19T00:51:18.422345+0000) 2024-03-19T00:51:19.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:19.053+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:19.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:19.053+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:19.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:19.053+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:19.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:19.053+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:19.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:19.053+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:19.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:19 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:19.365+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:19.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:19 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:19.365+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:19.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:19 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:19.365+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:19.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:19 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:19.365+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:19.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:19 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:19.503+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:19.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:19 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:19.503+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:19.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:19 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:19.503+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:19.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:19 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:19.341+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:19.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:19 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:19.341+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:19.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:19 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:19.341+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:20.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:20.046+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:26.818628+0000 front 2024-03-19T00:50:26.818509+0000 (oldest deadline 2024-03-19T00:50:52.718690+0000) 2024-03-19T00:51:20.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:20.046+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:39.120564+0000 front 2024-03-19T00:50:39.120716+0000 (oldest deadline 2024-03-19T00:51:02.020391+0000) 2024-03-19T00:51:20.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:20.046+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.921311+0000 front 2024-03-19T00:50:47.921402+0000 (oldest deadline 2024-03-19T00:51:09.621426+0000) 2024-03-19T00:51:20.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7[84580]: 2024-03-19T00:51:20.046+0000 7f6ca381c700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.122237+0000 front 2024-03-19T00:50:53.122308+0000 (oldest deadline 2024-03-19T00:51:18.422345+0000) 2024-03-19T00:51:20.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:20.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:20.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:20.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:20.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:20.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:20.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:20.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:20.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:20 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:20.015+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:20.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:20 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:20.331+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:20.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:20 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:20.331+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:20.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:20 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:20.331+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:20.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:20 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:20.331+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:20.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:20 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:20.331+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:20.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:20.520+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:20.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:20.520+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:20.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:20.520+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:20.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:20.520+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:20.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:20.320+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:20.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:20.320+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:20.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:20.320+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:20.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:20 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:20.320+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:21.094 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:20 smithi044 podman[101756]: 2024-03-19 00:51:20.81685142 +0000 UTC m=+5.481884180 container died 71a1957287b63899cbf55c7c01c314d3e8f130f6d53a6d5efbc49280bc40f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-19T00:51:21.095 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:21.095 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:21.095 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:21.095 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:21.095 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.002+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:21.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:21 smithi044 podman[101756]: 2024-03-19 00:51:21.184149286 +0000 UTC m=+5.849182052 container cleanup 71a1957287b63899cbf55c7c01c314d3e8f130f6d53a6d5efbc49280bc40f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-03-19T00:51:21.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:21 smithi044 bash[101756]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7 2024-03-19T00:51:21.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:21 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:21.305+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:21.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:21 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:21.305+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:21.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:21 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:21.305+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:21.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:21 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:21.305+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:21.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:21 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:21.305+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:21.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:21.511+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:21.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:21.511+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:21.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:21.511+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:21.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:21.511+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:21.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:21.338+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:21.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:21.338+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:21.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:21.338+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:21.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:21 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:21.338+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:21.786 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:21 smithi044 podman[101785]: 2024-03-19 00:51:21.452115528 +0000 UTC m=+0.626078614 container remove 71a1957287b63899cbf55c7c01c314d3e8f130f6d53a6d5efbc49280bc40f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-03-19T00:51:22.090 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:21 smithi044 podman[101876]: 2024-03-19 00:51:21.688822638 +0000 UTC m=+0.017600916 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:21 smithi044 podman[101876]: 2024-03-19 00:51:21.836536034 +0000 UTC m=+0.165314352 container create 52cf601b95e3e1d1a827be12b0bd6e30140e781214086d0ac8e51eeac8a9e68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:22 smithi044 podman[101876]: 2024-03-19 00:51:22.08916739 +0000 UTC m=+0.417945677 container init 52cf601b95e3e1d1a827be12b0bd6e30140e781214086d0ac8e51eeac8a9e68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.971+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.971+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.971+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.971+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:22.091 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:21 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:21.971+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:22.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:22 smithi044 podman[101876]: 2024-03-19 00:51:22.093731627 +0000 UTC m=+0.422509904 container start 52cf601b95e3e1d1a827be12b0bd6e30140e781214086d0ac8e51eeac8a9e68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-19T00:51:22.356 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:22 smithi044 podman[101876]: 2024-03-19 00:51:22.189245845 +0000 UTC m=+0.518024125 container attach 52cf601b95e3e1d1a827be12b0bd6e30140e781214086d0ac8e51eeac8a9e68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-19T00:51:22.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:22 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:22.256+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:22.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:22 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:22.256+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:22.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:22 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:22.256+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:22.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:22 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:22.256+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:22.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:22 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:22.256+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:22.742 INFO:journalctl@ceph.osd.7.smithi044.stdout:Mar 19 00:51:22 smithi044 podman[101876]: 2024-03-19 00:51:22.391556631 +0000 UTC m=+0.720334898 container died 52cf601b95e3e1d1a827be12b0bd6e30140e781214086d0ac8e51eeac8a9e68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-19T00:51:22.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:22.518+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:22.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:22.518+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:22.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:22.518+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:22.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:22.518+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:22.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:22.363+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:22.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:22.363+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:22.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:22.363+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:22.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:22 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:22.363+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:23.043 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:22 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:22.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:23.044 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:22 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:22.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:23.044 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:22 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:22.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:23.044 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:22 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:22.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:23.044 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:22 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:22.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:23.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.7.service' 2024-03-19T00:51:23.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:23.109 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-19T00:51:23.109 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-19T00:51:23.109 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.8 2024-03-19T00:51:23.414 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 systemd[1]: Stopping Ceph osd.8 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:23.414 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.412+0000 7f559ea2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:23.414 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:23.324+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:23.414 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:23.324+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:23.414 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:23.324+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:23.414 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:23.324+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:23.714 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:23.283+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:23.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:23.283+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:23.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:23.283+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:23.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:23.283+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:23.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:23 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:23.283+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:23.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.412+0000 7f559ea2b700 -1 osd.8 95 *** Got signal Terminated *** 2024-03-19T00:51:23.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.412+0000 7f559ea2b700 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:51:23.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.473+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:23.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.473+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:23.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.473+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:23.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:23 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:23.473+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:24.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:23.907+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:24.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:23.907+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:24.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:23.907+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:24.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:23.907+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:24.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:23.907+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:24.616 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:24.293+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:24.617 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:24.293+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:24.617 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:24.293+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:24.617 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:24.293+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:24.617 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:24 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:24.293+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:24.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:24.478+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:24.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:24.478+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:24.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:24.478+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:24.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:24.478+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:24.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:24.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:24.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:24.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:24.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:24.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:24.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:24 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:24.359+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:25.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:24 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:24.942+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:25.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:24 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:24.942+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:25.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:24 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:24.942+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:25.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:24 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:24.942+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:25.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:24 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:24.942+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:25.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:25.307+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:25.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:25.307+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:25.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:25.307+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:25.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:25.307+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:25.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:25 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:25.307+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:25.398+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:25.399+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:25.399+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:25.399+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:25.501+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:25.501+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:25.501+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:25.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:25 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:25.501+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:26.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:25 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:25.917+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:26.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:25 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:25.917+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:26.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:25 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:25.917+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:26.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:25 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:25.917+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:26.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:25 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:25.917+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:26.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:26.316+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:26.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:26.316+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:26.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:26.316+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:26.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:26.316+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:26.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:26 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:26.316+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:26.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:26.532+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:26.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:26.532+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:26.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:26.532+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:26.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:26.532+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:26.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:26.532+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:50:59.674968+0000 front 2024-03-19T00:50:59.674992+0000 (oldest deadline 2024-03-19T00:51:25.575060+0000) 2024-03-19T00:51:26.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:26.412+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:26.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:26.412+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:26.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:26.412+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:26.767 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:26 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:26.412+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:27.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:26 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:26.934+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:27.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:26 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:26.934+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:27.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:26 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:26.934+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:27.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:26 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:26.934+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:27.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:26 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:26.934+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:27.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:27.326+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:27.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:27.326+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:27.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:27.326+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:27.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:27.326+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:27.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:27.326+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:27.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:27 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:27.326+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:27.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:27.515+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:30.669853+0000 front 2024-03-19T00:50:30.669840+0000 (oldest deadline 2024-03-19T00:50:54.769865+0000) 2024-03-19T00:51:27.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:27.515+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.671962+0000 front 2024-03-19T00:50:38.672047+0000 (oldest deadline 2024-03-19T00:51:04.572096+0000) 2024-03-19T00:51:27.765 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:27.515+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.572822+0000 front 2024-03-19T00:50:44.572841+0000 (oldest deadline 2024-03-19T00:51:09.872552+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:27.515+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:55.674105+0000 front 2024-03-19T00:50:55.673953+0000 (oldest deadline 2024-03-19T00:51:19.674337+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8[85245]: 2024-03-19T00:51:27.516+0000 7f5597017700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:50:59.674968+0000 front 2024-03-19T00:50:59.674992+0000 (oldest deadline 2024-03-19T00:51:25.575060+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:27.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:27.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:27.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:27.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:27.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:27 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:27.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:28.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:27.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:28.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:27.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:28.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:27.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:28.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:27.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:28.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:27.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:28.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:27 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:27.927+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:28.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:28.344+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:28.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:28.344+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:28.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:28.344+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:28.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:28.344+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:28.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:28.344+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:28.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:28 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:28.344+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:28.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:28.376+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:28.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:28.376+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:28.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:28.376+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:28.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:28.376+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:28.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:28.376+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:29.213 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:28 smithi204 podman[103458]: 2024-03-19 00:51:28.937232473 +0000 UTC m=+5.634330626 container died 3b85e986c1be948a308d0483fd3cf55a60834bea7cc3263c0cd1c4632e1c8583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T00:51:29.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:28 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:28.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:29.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:28 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:28.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:29.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:28 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:28.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:29.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:28 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:28.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:29.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:28 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:28.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:29.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:28 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:28.943+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:29.515 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:29 smithi204 podman[103458]: 2024-03-19 00:51:29.304420367 +0000 UTC m=+6.001518517 container cleanup 3b85e986c1be948a308d0483fd3cf55a60834bea7cc3263c0cd1c4632e1c8583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:29.515 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:29 smithi204 bash[103458]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8 2024-03-19T00:51:29.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:29 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:29.389+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:29.515 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:29 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:29.389+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:29 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:29.389+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:29 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:29.389+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:29.516 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:29 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:29.389+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:29.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:29.391+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:29.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:29.391+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:29.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:29.391+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:29.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:29.391+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:29.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:29.391+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:29.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:29 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:29.391+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:29.840 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:29 smithi204 podman[103474]: 2024-03-19 00:51:29.533989784 +0000 UTC m=+0.593218469 container remove 3b85e986c1be948a308d0483fd3cf55a60834bea7cc3263c0cd1c4632e1c8583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-19T00:51:30.110 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:29 smithi204 podman[103568]: 2024-03-19 00:51:29.839786601 +0000 UTC m=+0.102203241 container create 4aa23f4592f4fa0e6b4a20756720e2f35516e5dd1fe15df0a87393c380ce6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-03-19T00:51:30.110 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:29 smithi204 podman[103568]: 2024-03-19 00:51:29.756010984 +0000 UTC m=+0.018427653 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:30.110 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:30 smithi204 podman[103568]: 2024-03-19 00:51:30.109993349 +0000 UTC m=+0.372409990 container init 4aa23f4592f4fa0e6b4a20756720e2f35516e5dd1fe15df0a87393c380ce6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:51:30.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:29 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:29.978+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:30.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:29 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:29.978+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:30.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:29 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:29.978+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:30.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:29 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:29.978+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:30.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:29 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:29.978+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:30.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:29 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:29.978+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:30.408 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:30 smithi204 podman[103568]: 2024-03-19 00:51:30.113626333 +0000 UTC m=+0.376042974 container start 4aa23f4592f4fa0e6b4a20756720e2f35516e5dd1fe15df0a87393c380ce6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:30.408 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:30 smithi204 podman[103568]: 2024-03-19 00:51:30.168116589 +0000 UTC m=+0.430533264 container attach 4aa23f4592f4fa0e6b4a20756720e2f35516e5dd1fe15df0a87393c380ce6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-19T00:51:30.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:30 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:30.348+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:30.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:30 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:30.348+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:30.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:30 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:30.348+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:30.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:30 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:30.348+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:30.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:30 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:30.348+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:30.689 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 19 00:51:30 smithi204 podman[103568]: 2024-03-19 00:51:30.408488716 +0000 UTC m=+0.670905357 container died 4aa23f4592f4fa0e6b4a20756720e2f35516e5dd1fe15df0a87393c380ce6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:51:30.714 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:30.345+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:30.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:30.345+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:30.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:30.345+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:30.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:30.345+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:30.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:30.345+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:30.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:30 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:30.345+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:30.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.8.service' 2024-03-19T00:51:31.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:31.011 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-19T00:51:31.011 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-19T00:51:31.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.9 2024-03-19T00:51:31.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:30 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:30.956+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:31.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:30 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:30.956+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:31.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:30 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:30.956+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:31.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:30 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:30.956+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:31.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:30 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:30.956+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:31.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:30 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:30.956+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:31.387 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 systemd[1]: Stopping Ceph osd.9 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.386+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.386+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.386+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.386+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.386+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.386+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:31.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.427+0000 7facd4562700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:31.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.427+0000 7facd4562700 -1 osd.9 95 *** Got signal Terminated *** 2024-03-19T00:51:31.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:31 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:31.427+0000 7facd4562700 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:51:31.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:31.362+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:31.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:31.362+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:31.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:31.362+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:31.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:31.362+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:31.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:31.362+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:32.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.005+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:32.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.005+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:32.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.005+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:32.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.005+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:32.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.005+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:32.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.005+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:32.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:32 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:32.387+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:32.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:32 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:32.387+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:32.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:32 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:32.387+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:32.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:32 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:32.387+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:32.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:32 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:32.387+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:32.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:32 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:32.387+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:32.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:32 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:32.402+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:32.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:32 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:32.402+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:32.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:32 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:32.402+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:32.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:32 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:32.402+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:32.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:32 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:32.402+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:33.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:33.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:33.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:33.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:33.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:33.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:32 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:32.960+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:33.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:33.372+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:33.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:33.372+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:33.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:33.372+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:33.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:33.372+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:33.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:33.372+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:33.716 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:33 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:33.372+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:33.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:33.421+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:33.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:33.421+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:33.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:33.421+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:33.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:33.421+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:33.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:33 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:33.421+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:34.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:33 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:33.966+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:34.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:33 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:33.966+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:34.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:33 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:33.966+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:34.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:33 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:33.966+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:34.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:33 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:33.966+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:34.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:33 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:33.966+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:34.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:34.346+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:34.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:34.346+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:34.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:34.346+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:34.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:34.346+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:34.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:34.346+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:34.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:34 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:34.346+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:34.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:34 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:34.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:34.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:34 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:34.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:34.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:34 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:34.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:34.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:34 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:34.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:34.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:34 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:34.387+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:35.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:34 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:34.973+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:35.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:34 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:34.973+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:35.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:34 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:34.973+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:35.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:34 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:34.973+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:35.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:34 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:34.973+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:35.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:34 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:34.973+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:35.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:35 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:35.359+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:35.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:35 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:35.359+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:35.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:35 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:35.359+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:35.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:35 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:35.359+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:35.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:35 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:35.359+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:35.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:35 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:35.359+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:35.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:35.345+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:35.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:35.345+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:35.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:35.345+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:35.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:35.345+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:35.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:35.345+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:36.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:35.938+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:36.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:35.938+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:36.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:35.938+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:36.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:35.938+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:36.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:35.938+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:36.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:35 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:35.938+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:36.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:36.366+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.980651+0000 front 2024-03-19T00:50:22.980676+0000 (oldest deadline 2024-03-19T00:50:45.280797+0000) 2024-03-19T00:51:36.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:36.366+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.182040+0000 front 2024-03-19T00:50:31.181947+0000 (oldest deadline 2024-03-19T00:50:54.682011+0000) 2024-03-19T00:51:36.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:36.366+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:34.682737+0000 front 2024-03-19T00:50:34.682828+0000 (oldest deadline 2024-03-19T00:51:02.883186+0000) 2024-03-19T00:51:36.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:36.366+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:44.584078+0000 front 2024-03-19T00:50:44.584130+0000 (oldest deadline 2024-03-19T00:51:09.884178+0000) 2024-03-19T00:51:36.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:36.366+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:53.985383+0000 front 2024-03-19T00:50:53.985504+0000 (oldest deadline 2024-03-19T00:51:19.885228+0000) 2024-03-19T00:51:36.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9[94907]: 2024-03-19T00:51:36.366+0000 7facd0b56700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.487004+0000 front 2024-03-19T00:51:01.486825+0000 (oldest deadline 2024-03-19T00:51:26.687152+0000) 2024-03-19T00:51:36.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:36 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:36.312+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:36.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:36 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:36.312+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:36.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:36 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:36.312+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:36.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:36 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:36.312+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:36.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:36 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:36.312+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:37.215 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:36 smithi007 podman[128667]: 2024-03-19 00:51:36.953422 +0000 UTC m=+5.748465360 container died 926356c8b5ccf47191c8765cf4c686e0fdd89af710db4c518fecfd4e310096e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:51:37.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:36 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:36.986+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:37.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:36 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:36.986+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:37.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:36 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:36.986+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:37.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:36 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:36.986+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:37.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:36 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:36.986+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:37.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:36 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:36.986+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:37.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:37 smithi007 podman[128667]: 2024-03-19 00:51:37.39410913 +0000 UTC m=+6.189152492 container cleanup 926356c8b5ccf47191c8765cf4c686e0fdd89af710db4c518fecfd4e310096e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:51:37.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:37 smithi007 bash[128667]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9 2024-03-19T00:51:37.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:37 smithi007 podman[128684]: 2024-03-19 00:51:37.570679953 +0000 UTC m=+0.611209342 container remove 926356c8b5ccf47191c8765cf4c686e0fdd89af710db4c518fecfd4e310096e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:51:37.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:37 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:37.331+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:37.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:37 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:37.331+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:37.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:37 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:37.331+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:37.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:37 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:37.331+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:37.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:37 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:37.331+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:37.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:37 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:37.331+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:38 smithi007 podman[128779]: 2024-03-19 00:51:37.918126722 +0000 UTC m=+0.018373099 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:38 smithi007 podman[128779]: 2024-03-19 00:51:38.021347394 +0000 UTC m=+0.121593759 container create 0429a9dd98fde5b78d9147e991345c31ace0d677b83dc147b1b911e9bc6b4408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-19T00:51:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:38 smithi007 podman[128779]: 2024-03-19 00:51:38.223583746 +0000 UTC m=+0.323830124 container init 0429a9dd98fde5b78d9147e991345c31ace0d677b83dc147b1b911e9bc6b4408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-19T00:51:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:38 smithi007 podman[128779]: 2024-03-19 00:51:38.226746831 +0000 UTC m=+0.326993197 container start 0429a9dd98fde5b78d9147e991345c31ace0d677b83dc147b1b911e9bc6b4408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-19T00:51:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:38 smithi007 podman[128779]: 2024-03-19 00:51:38.298550295 +0000 UTC m=+0.398796661 container attach 0429a9dd98fde5b78d9147e991345c31ace0d677b83dc147b1b911e9bc6b4408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:38.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:37 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:37.998+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:38.715 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 19 00:51:38 smithi007 podman[128779]: 2024-03-19 00:51:38.527401032 +0000 UTC m=+0.627647415 container died 0429a9dd98fde5b78d9147e991345c31ace0d677b83dc147b1b911e9bc6b4408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-19T00:51:38.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:38.332+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:38.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:38.332+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:38.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:38.332+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:38.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:38.332+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:38.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:38.332+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:38.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:38 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:38.332+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:39.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.9.service' 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:39.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:38 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:38.992+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:39.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:39.393 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-19T00:51:39.393 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-19T00:51:39.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.10 2024-03-19T00:51:39.696 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 systemd[1]: Stopping Ceph osd.10 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:39.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:39.367+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:39.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:39.367+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:39.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:39.367+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:39.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:39.367+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:39.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:39.367+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:39.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:39.367+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:39.981 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.695+0000 7fc49a41f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:39.981 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.695+0000 7fc49a41f700 -1 osd.10 95 *** Got signal Terminated *** 2024-03-19T00:51:39.981 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.695+0000 7fc49a41f700 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:39.982 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:39 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:39.980+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:40.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:40 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:40.383+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:40.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:40 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:40.383+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:40.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:40 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:40.383+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:40.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:40 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:40.383+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:40.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:40 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:40.383+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:40.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:40 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:40.383+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:41.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:41.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:41.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:41.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:41.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:41.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:41.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:41 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:41.016+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:41.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:41.347+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:41.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:41.347+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:41.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:41.347+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:41.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:41.347+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:41.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:41.347+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:41.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:41 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:41.347+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:42.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:42.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:42.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:42.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:42.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:42.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:42.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:42 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:42.057+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:42.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:42 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:42.380+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:42.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:42 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:42.380+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:42.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:42 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:42.380+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:42.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:42 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:42.380+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:42.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:42 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:42.380+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:42.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:42 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:42.380+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:43.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:43.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:43.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:43.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:43.356 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:43.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:43.357 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:43 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:43.077+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:43.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:43 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:43.395+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:43.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:43 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:43.395+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:43.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:43 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:43.395+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:43.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:43 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:43.395+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:43.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:43 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:43.395+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:43.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:43 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:43.395+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:19.493333+0000 front 2024-03-19T00:50:19.493204+0000 (oldest deadline 2024-03-19T00:50:45.393299+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:29.995024+0000 front 2024-03-19T00:50:29.994927+0000 (oldest deadline 2024-03-19T00:50:53.495017+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:37.496340+0000 front 2024-03-19T00:50:37.496419+0000 (oldest deadline 2024-03-19T00:51:01.596492+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-19T00:50:47.398362+0000 front 2024-03-19T00:50:47.398421+0000 (oldest deadline 2024-03-19T00:51:09.098223+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:51.399075+0000 front 2024-03-19T00:50:51.399118+0000 (oldest deadline 2024-03-19T00:51:17.299159+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:03.700892+0000 front 2024-03-19T00:51:03.701025+0000 (oldest deadline 2024-03-19T00:51:27.800744+0000) 2024-03-19T00:51:44.606 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:44 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10[91344]: 2024-03-19T00:51:44.124+0000 7fc496a13700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:07.801485+0000 front 2024-03-19T00:51:07.801412+0000 (oldest deadline 2024-03-19T00:51:37.801635+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:44.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:44 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:44.443+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:45.455 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:45 smithi044 podman[101982]: 2024-03-19 00:51:45.187472277 +0000 UTC m=+5.602453118 container died fd889d5264917b637230168d554c20923ec5020e92183400771197a0076e47d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:45.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:45 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:45.396+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:45.822 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:45 smithi044 podman[101982]: 2024-03-19 00:51:45.553986392 +0000 UTC m=+5.968967243 container cleanup fd889d5264917b637230168d554c20923ec5020e92183400771197a0076e47d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-19T00:51:45.822 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:45 smithi044 bash[101982]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10 2024-03-19T00:51:46.106 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:45 smithi044 podman[101999]: 2024-03-19 00:51:45.821894688 +0000 UTC m=+0.632534191 container remove fd889d5264917b637230168d554c20923ec5020e92183400771197a0076e47d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-19T00:51:46.412 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:46 smithi044 podman[102086]: 2024-03-19 00:51:46.064330355 +0000 UTC m=+0.016888310 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:46.412 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:46 smithi044 podman[102086]: 2024-03-19 00:51:46.198329938 +0000 UTC m=+0.150887891 container create 84c2b8c1b5cd040546178658e99f8dfb2bad8799612c961eef4b65517d7d4d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-19T00:51:46.412 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:46 smithi044 podman[102086]: 2024-03-19 00:51:46.409169405 +0000 UTC m=+0.361727360 container init 84c2b8c1b5cd040546178658e99f8dfb2bad8799612c961eef4b65517d7d4d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-19T00:51:46.412 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:46 smithi044 podman[102086]: 2024-03-19 00:51:46.412133265 +0000 UTC m=+0.364691221 container start 84c2b8c1b5cd040546178658e99f8dfb2bad8799612c961eef4b65517d7d4d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:51:46.703 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:46 smithi044 podman[102086]: 2024-03-19 00:51:46.509222097 +0000 UTC m=+0.461780056 container attach 84c2b8c1b5cd040546178658e99f8dfb2bad8799612c961eef4b65517d7d4d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:46.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:46 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:46.447+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:46.995 INFO:journalctl@ceph.osd.10.smithi044.stdout:Mar 19 00:51:46 smithi044 podman[102086]: 2024-03-19 00:51:46.703445552 +0000 UTC m=+0.656003511 container died 84c2b8c1b5cd040546178658e99f8dfb2bad8799612c961eef4b65517d7d4d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-19T00:51:47.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.10.service' 2024-03-19T00:51:47.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:47.412 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-19T00:51:47.412 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-19T00:51:47.413 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.11 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.442+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 systemd[1]: Stopping Ceph osd.11 for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.716+0000 7f2c329ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.716+0000 7f2c329ec700 -1 osd.11 95 *** Got signal Terminated *** 2024-03-19T00:51:47.718 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:47 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:47.716+0000 7f2c329ec700 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:48.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:48 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:48.446+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:49.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:49.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:49 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:49.489+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-19T00:51:27.487484+0000 front 2024-03-19T00:51:27.487522+0000 (oldest deadline 2024-03-19T00:51:49.187523+0000) 2024-03-19T00:51:50.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:50.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:50.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:50.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:50.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:50.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:50.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:50.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:50 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:50.462+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-19T00:51:27.487484+0000 front 2024-03-19T00:51:27.487522+0000 (oldest deadline 2024-03-19T00:51:49.187523+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:51.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:51.766 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:51 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:51.505+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-19T00:51:27.487484+0000 front 2024-03-19T00:51:27.487522+0000 (oldest deadline 2024-03-19T00:51:49.187523+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-19T00:50:22.074224+0000 front 2024-03-19T00:50:22.074446+0000 (oldest deadline 2024-03-19T00:50:46.174307+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-03-19T00:50:31.876984+0000 front 2024-03-19T00:50:31.876962+0000 (oldest deadline 2024-03-19T00:50:54.776909+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-19T00:50:38.778457+0000 front 2024-03-19T00:50:38.778519+0000 (oldest deadline 2024-03-19T00:51:02.878575+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-03-19T00:50:50.380941+0000 front 2024-03-19T00:50:50.380849+0000 (oldest deadline 2024-03-19T00:51:19.781269+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-19T00:51:01.883644+0000 front 2024-03-19T00:51:01.883517+0000 (oldest deadline 2024-03-19T00:51:27.183630+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-19T00:51:10.585384+0000 front 2024-03-19T00:51:10.585339+0000 (oldest deadline 2024-03-19T00:51:36.485227+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-03-19T00:51:19.386118+0000 front 2024-03-19T00:51:19.386196+0000 (oldest deadline 2024-03-19T00:51:43.986479+0000) 2024-03-19T00:51:52.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:52 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11[92256]: 2024-03-19T00:51:52.461+0000 7f2c2afd8700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-19T00:51:27.487484+0000 front 2024-03-19T00:51:27.487522+0000 (oldest deadline 2024-03-19T00:51:49.187523+0000) 2024-03-19T00:51:53.467 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:53 smithi204 podman[103686]: 2024-03-19 00:51:53.15354853 +0000 UTC m=+5.561148704 container died 175da5bba7409397111e8635829545dc164ca7ac08345a44ad2b006f5bfe9c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-19T00:51:53.467 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:53 smithi204 podman[103686]: 2024-03-19 00:51:53.466913873 +0000 UTC m=+5.874514052 container cleanup 175da5bba7409397111e8635829545dc164ca7ac08345a44ad2b006f5bfe9c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-19T00:51:53.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:53 smithi204 bash[103686]: ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11 2024-03-19T00:51:53.765 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:53 smithi204 podman[103704]: 2024-03-19 00:51:53.669067658 +0000 UTC m=+0.509354581 container remove 175da5bba7409397111e8635829545dc164ca7ac08345a44ad2b006f5bfe9c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:51:54.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:53 smithi204 podman[103798]: 2024-03-19 00:51:53.88691397 +0000 UTC m=+0.017794649 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T00:51:54.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:53 smithi204 podman[103798]: 2024-03-19 00:51:53.986315771 +0000 UTC m=+0.117196449 container create 29f3d36e4afa5b85ac4fce1c5c9bc22a39677b526d8bfc372aa3f660cde9cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-19T00:51:54.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:54 smithi204 podman[103798]: 2024-03-19 00:51:54.196855391 +0000 UTC m=+0.327736081 container init 29f3d36e4afa5b85ac4fce1c5c9bc22a39677b526d8bfc372aa3f660cde9cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-19T00:51:54.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:54 smithi204 podman[103798]: 2024-03-19 00:51:54.200367641 +0000 UTC m=+0.331248329 container start 29f3d36e4afa5b85ac4fce1c5c9bc22a39677b526d8bfc372aa3f660cde9cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T00:51:54.758 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:54 smithi204 podman[103798]: 2024-03-19 00:51:54.272053174 +0000 UTC m=+0.402933857 container attach 29f3d36e4afa5b85ac4fce1c5c9bc22a39677b526d8bfc372aa3f660cde9cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T00:51:54.759 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:54 smithi204 podman[103798]: 2024-03-19 00:51:54.490586537 +0000 UTC m=+0.621467219 container died 29f3d36e4afa5b85ac4fce1c5c9bc22a39677b526d8bfc372aa3f660cde9cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-19T00:51:55.014 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 19 00:51:55 smithi204 podman[103824]: 2024-03-19 00:51:55.000589813 +0000 UTC m=+0.501445243 container remove 29f3d36e4afa5b85ac4fce1c5c9bc22a39677b526d8bfc372aa3f660cde9cc85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-19T00:51:55.020 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@osd.11.service' 2024-03-19T00:51:55.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:55.062 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-19T00:51:55.062 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-19T00:51:55.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.a 2024-03-19T00:51:55.371 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:51:55 smithi007 systemd[1]: Stopping Ceph mds.a for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:55.646 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:51:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a[101430]: 2024-03-19T00:51:55.370+0000 7f79734e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:55.646 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:51:55 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a[101430]: 2024-03-19T00:51:55.370+0000 7f79734e7700 -1 mds.a *** got signal Terminated *** 2024-03-19T00:51:57.437 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:51:57 smithi007 podman[128898]: 2024-03-19 00:51:57.174133077 +0000 UTC m=+1.916164093 container died 0b271c0ad50e6970e5eed27cb8d357abcf66bcf2b5e320e78bded2b178db1ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:51:57.714 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:51:57 smithi007 podman[128898]: 2024-03-19 00:51:57.52830906 +0000 UTC m=+2.270340076 container cleanup 0b271c0ad50e6970e5eed27cb8d357abcf66bcf2b5e320e78bded2b178db1ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-19T00:51:57.715 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 19 00:51:57 smithi007 bash[128898]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-a 2024-03-19T00:51:58.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.a.service' 2024-03-19T00:51:58.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:51:58.049 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-19T00:51:58.049 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-19T00:51:58.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.d 2024-03-19T00:51:58.392 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:51:58 smithi007 systemd[1]: Stopping Ceph mds.d for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:51:58.714 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:51:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d[100475]: 2024-03-19T00:51:58.390+0000 7fa28f284700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:51:58.715 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:51:58 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d[100475]: 2024-03-19T00:51:58.390+0000 7fa28f284700 -1 mds.d *** got signal Terminated *** 2024-03-19T00:52:01.465 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:52:01 smithi007 podman[129031]: 2024-03-19 00:52:01.142503622 +0000 UTC m=+2.880396014 container died 0912a57a2241f7a3737aa7842c5e41119ca882f367016b2339472d363cbc9bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:52:01.959 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:52:01 smithi007 podman[129031]: 2024-03-19 00:52:01.743278369 +0000 UTC m=+3.481170766 container cleanup 0912a57a2241f7a3737aa7842c5e41119ca882f367016b2339472d363cbc9bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-19T00:52:01.959 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 19 00:52:01 smithi007 bash[129031]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-d 2024-03-19T00:52:02.072 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.d.service' 2024-03-19T00:52:02.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:02.107 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-19T00:52:02.107 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-19T00:52:02.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.g 2024-03-19T00:52:02.214 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:52:02 smithi007 systemd[1]: Stopping Ceph mds.g for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:02.715 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:52:02 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g[101069]: 2024-03-19T00:52:02.419+0000 7f3d028a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:02.715 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:52:02 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g[101069]: 2024-03-19T00:52:02.419+0000 7f3d028a8700 -1 mds.g *** got signal Terminated *** 2024-03-19T00:52:06.465 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:52:06 smithi007 podman[129152]: 2024-03-19 00:52:06.29655236 +0000 UTC m=+3.973939670 container died 3e4143e76ce259631282ecaf9f2995752309fcea469cb88221440faa9bd49c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-19T00:52:06.965 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:52:06 smithi007 podman[129152]: 2024-03-19 00:52:06.769605124 +0000 UTC m=+4.446992440 container cleanup 3e4143e76ce259631282ecaf9f2995752309fcea469cb88221440faa9bd49c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-19T00:52:06.965 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 19 00:52:06 smithi007 bash[129152]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-g 2024-03-19T00:52:07.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.g.service' 2024-03-19T00:52:07.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:07.173 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-19T00:52:07.174 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-19T00:52:07.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.j 2024-03-19T00:52:07.465 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:07 smithi007 systemd[1]: Stopping Ceph mds.j for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:07.965 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j[100087]: 2024-03-19T00:52:07.478+0000 7f97a19ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:07.965 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:07 smithi007 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j[100087]: 2024-03-19T00:52:07.478+0000 7f97a19ee700 -1 mds.j *** got signal Terminated *** 2024-03-19T00:52:10.556 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:10 smithi007 podman[129274]: 2024-03-19 00:52:10.233366929 +0000 UTC m=+2.845730296 container died c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-19T00:52:10.557 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:10 smithi007 podman[129274]: 2024-03-19 00:52:10.556236919 +0000 UTC m=+3.168600284 container cleanup c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-19T00:52:10.964 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:10 smithi007 bash[129274]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j 2024-03-19T00:52:10.965 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 19 00:52:10 smithi007 podman[129288]: 2024-03-19 00:52:10.74915901 +0000 UTC m=+0.516283794 container remove c1ad7694215d58b42cb3c8e7d9d618d073e3e337485b24e88285c0db4b48a988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-19T00:52:10.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.j.service' 2024-03-19T00:52:11.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:11.029 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-19T00:52:11.030 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-19T00:52:11.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.b 2024-03-19T00:52:11.342 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:52:11 smithi044 systemd[1]: Stopping Ceph mds.b for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:11.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:52:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: 2024-03-19T00:52:11.341+0000 7f7138872700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:11.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:52:11 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b[96557]: 2024-03-19T00:52:11.341+0000 7f7138872700 -1 mds.b *** got signal Terminated *** 2024-03-19T00:52:14.643 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:52:14 smithi044 podman[102204]: 2024-03-19 00:52:14.374298912 +0000 UTC m=+3.149480240 container died 0e16aa255c9a720acd9f14537483eab50d66a49cb18fbf1ca3d5aa3fc33f0d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-19T00:52:15.002 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:52:14 smithi044 podman[102204]: 2024-03-19 00:52:14.741929077 +0000 UTC m=+3.517110405 container cleanup 0e16aa255c9a720acd9f14537483eab50d66a49cb18fbf1ca3d5aa3fc33f0d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-19T00:52:15.002 INFO:journalctl@ceph.mds.b.smithi044.stdout:Mar 19 00:52:14 smithi044 bash[102204]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-b 2024-03-19T00:52:15.179 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.b.service' 2024-03-19T00:52:15.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:15.217 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-19T00:52:15.217 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-19T00:52:15.217 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.e 2024-03-19T00:52:15.547 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:15 smithi044 systemd[1]: Stopping Ceph mds.e for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:15.856 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e[95937]: 2024-03-19T00:52:15.545+0000 7fdaa8851700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:15.856 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:15 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e[95937]: 2024-03-19T00:52:15.545+0000 7fdaa8851700 -1 mds.e *** got signal Terminated *** 2024-03-19T00:52:18.672 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:18 smithi044 podman[102326]: 2024-03-19 00:52:18.368873412 +0000 UTC m=+2.933468388 container died ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T00:52:18.981 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:18 smithi044 podman[102326]: 2024-03-19 00:52:18.788340541 +0000 UTC m=+3.352935515 container cleanup ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-19T00:52:18.981 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:18 smithi044 bash[102326]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e 2024-03-19T00:52:19.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.e.service' 2024-03-19T00:52:19.239 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:18 smithi044 podman[102352]: 2024-03-19 00:52:18.981321129 +0000 UTC m=+0.603543013 container remove ebebdf52cadc1c3bd9dcfd711a2a7f0aefbf65c658d4c44ab3beca76bccb156b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-e, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-19T00:52:19.239 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:19 smithi044 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.e.service: Deactivated successfully. 2024-03-19T00:52:19.239 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:19 smithi044 systemd[1]: Stopped Ceph mds.e for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:52:19.239 INFO:journalctl@ceph.mds.e.smithi044.stdout:Mar 19 00:52:19 smithi044 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.e.service: Consumed 1.731s CPU time. 2024-03-19T00:52:19.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:19.252 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-19T00:52:19.252 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-19T00:52:19.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.h 2024-03-19T00:52:19.576 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:52:19 smithi044 systemd[1]: Stopping Ceph mds.h for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:19.856 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:52:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h[96250]: 2024-03-19T00:52:19.574+0000 7fb0bffb1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:19.856 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:52:19 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h[96250]: 2024-03-19T00:52:19.574+0000 7fb0bffb1700 -1 mds.h *** got signal Terminated *** 2024-03-19T00:52:22.651 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:52:22 smithi044 podman[102458]: 2024-03-19 00:52:22.377563325 +0000 UTC m=+2.905652766 container died e3640b3b6b0d62efdb558e067756943a94ae38e6f5ea7bddc023da6ea1a2c998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-19T00:52:23.002 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:52:22 smithi044 podman[102458]: 2024-03-19 00:52:22.7505631 +0000 UTC m=+3.278652549 container cleanup e3640b3b6b0d62efdb558e067756943a94ae38e6f5ea7bddc023da6ea1a2c998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-19T00:52:23.002 INFO:journalctl@ceph.mds.h.smithi044.stdout:Mar 19 00:52:22 smithi044 bash[102458]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-h 2024-03-19T00:52:23.127 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.h.service' 2024-03-19T00:52:23.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:23.165 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-19T00:52:23.165 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-19T00:52:23.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.k 2024-03-19T00:52:23.356 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:23 smithi044 systemd[1]: Stopping Ceph mds.k for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:23.856 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k[95621]: 2024-03-19T00:52:23.471+0000 7f6c95bda700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:23.856 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:23 smithi044 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k[95621]: 2024-03-19T00:52:23.471+0000 7f6c95bda700 -1 mds.k *** got signal Terminated *** 2024-03-19T00:52:27.597 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:27 smithi044 podman[102578]: 2024-03-19 00:52:27.305389084 +0000 UTC m=+3.926579437 container died 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-19T00:52:27.856 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:27 smithi044 podman[102578]: 2024-03-19 00:52:27.614516191 +0000 UTC m=+4.235706545 container cleanup 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-03-19T00:52:27.856 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:27 smithi044 bash[102578]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k 2024-03-19T00:52:27.856 INFO:journalctl@ceph.mds.k.smithi044.stdout:Mar 19 00:52:27 smithi044 podman[102592]: 2024-03-19 00:52:27.832004431 +0000 UTC m=+0.523213450 container remove 8d088867846da57d0d205ad65aa2d43a8e6a481068b1f24f078483442372265d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-k, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-19T00:52:27.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.k.service' 2024-03-19T00:52:28.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:28.007 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-19T00:52:28.007 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-19T00:52:28.007 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.c 2024-03-19T00:52:28.310 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:52:28 smithi204 systemd[1]: Stopping Ceph mds.c for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:28.567 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:52:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c[97163]: 2024-03-19T00:52:28.308+0000 7fa972433700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:28.567 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:52:28 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c[97163]: 2024-03-19T00:52:28.308+0000 7fa972433700 -1 mds.c *** got signal Terminated *** 2024-03-19T00:52:30.652 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:52:30 smithi204 podman[103903]: 2024-03-19 00:52:30.369065541 +0000 UTC m=+2.165175458 container died 8af9b6feaac41ae7d05c302840b87bb4f683c13b76c34239caada13fd3ed5818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-19T00:52:30.952 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:52:30 smithi204 podman[103903]: 2024-03-19 00:52:30.759073945 +0000 UTC m=+2.555183854 container cleanup 8af9b6feaac41ae7d05c302840b87bb4f683c13b76c34239caada13fd3ed5818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:52:30.952 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 19 00:52:30 smithi204 bash[103903]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-c 2024-03-19T00:52:31.079 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.c.service' 2024-03-19T00:52:31.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:31.119 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-19T00:52:31.119 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-19T00:52:31.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.f 2024-03-19T00:52:31.265 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:31 smithi204 systemd[1]: Stopping Ceph mds.f for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:31.765 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f[96854]: 2024-03-19T00:52:31.461+0000 7f3b951ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:31.765 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:31 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f[96854]: 2024-03-19T00:52:31.461+0000 7f3b951ae700 -1 mds.f *** got signal Terminated *** 2024-03-19T00:52:34.748 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:34 smithi204 podman[104035]: 2024-03-19 00:52:34.376140659 +0000 UTC m=+3.040890725 container died 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-19T00:52:35.011 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:34 smithi204 podman[104035]: 2024-03-19 00:52:34.821010366 +0000 UTC m=+3.485760439 container cleanup 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-19T00:52:35.011 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:34 smithi204 bash[104035]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f 2024-03-19T00:52:35.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.f.service' 2024-03-19T00:52:35.265 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:35 smithi204 podman[104049]: 2024-03-19 00:52:35.011341161 +0000 UTC m=+0.629140626 container remove 4bf907b4949819b7464c21f65caa953bf5ab6199c5127fa6ebb610190537ea29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-19T00:52:35.265 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:35 smithi204 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-19T00:52:35.265 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:35 smithi204 systemd[1]: Stopped Ceph mds.f for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:52:35.265 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 19 00:52:35 smithi204 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.f.service: Consumed 2.094s CPU time. 2024-03-19T00:52:35.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:35.284 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-19T00:52:35.284 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-19T00:52:35.284 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.i 2024-03-19T00:52:35.575 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:35 smithi204 systemd[1]: Stopping Ceph mds.i for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:36.015 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i[97471]: 2024-03-19T00:52:35.573+0000 7f6357dc2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:36.015 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:35 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i[97471]: 2024-03-19T00:52:35.573+0000 7f6357dc2700 -1 mds.i *** got signal Terminated *** 2024-03-19T00:52:38.733 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:38 smithi204 podman[104154]: 2024-03-19 00:52:38.374696684 +0000 UTC m=+2.877820329 container died 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-19T00:52:39.000 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:38 smithi204 podman[104154]: 2024-03-19 00:52:38.807797693 +0000 UTC m=+3.310921338 container cleanup 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:52:39.000 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:38 smithi204 bash[104154]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i 2024-03-19T00:52:39.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.i.service' 2024-03-19T00:52:39.258 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:39 smithi204 podman[104168]: 2024-03-19 00:52:39.000767445 +0000 UTC m=+0.617420908 container remove 3a6f3fc144cc2c0843db2327be8466efa61540a4a5a4cd45baaa8591a026a7e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-19T00:52:39.258 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:39 smithi204 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.i.service: Deactivated successfully. 2024-03-19T00:52:39.258 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:39 smithi204 systemd[1]: Stopped Ceph mds.i for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:52:39.258 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 19 00:52:39 smithi204 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.i.service: Consumed 4min 4.917s CPU time. 2024-03-19T00:52:39.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:39.270 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-19T00:52:39.270 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-19T00:52:39.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.l 2024-03-19T00:52:39.514 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:39 smithi204 systemd[1]: Stopping Ceph mds.l for 60839b00-e587-11ee-95c9-87774f69a715... 2024-03-19T00:52:40.015 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l[96539]: 2024-03-19T00:52:39.594+0000 7ff69e383700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:52:40.015 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:39 smithi204 ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l[96539]: 2024-03-19T00:52:39.594+0000 7ff69e383700 -1 mds.l *** got signal Terminated *** 2024-03-19T00:52:43.854 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:43 smithi204 podman[104273]: 2024-03-19 00:52:43.60058689 +0000 UTC m=+4.107634832 container died 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:52:44.121 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:43 smithi204 podman[104273]: 2024-03-19 00:52:43.928175654 +0000 UTC m=+4.435223609 container cleanup 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True) 2024-03-19T00:52:44.121 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:43 smithi204 bash[104273]: ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l 2024-03-19T00:52:44.351 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.l.service' 2024-03-19T00:52:44.374 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:44 smithi204 podman[104287]: 2024-03-19 00:52:44.120676605 +0000 UTC m=+0.514476127 container remove 939651d69de54ed9ae9b28a32f480bab837e136a58b9e95b47698a0999847b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-60839b00-e587-11ee-95c9-87774f69a715-mds-l, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-19T00:52:44.375 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:44 smithi204 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.l.service: Deactivated successfully. 2024-03-19T00:52:44.375 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:44 smithi204 systemd[1]: Stopped Ceph mds.l for 60839b00-e587-11ee-95c9-87774f69a715. 2024-03-19T00:52:44.375 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 19 00:52:44 smithi204 systemd[1]: ceph-60839b00-e587-11ee-95c9-87774f69a715@mds.l.service: Consumed 1.014s CPU time. 2024-03-19T00:52:44.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T00:52:44.385 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-19T00:52:44.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 60839b00-e587-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T00:52:44.581 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:52:47.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 60839b00-e587-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T00:52:47.247 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:52:49.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 60839b00-e587-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T00:52:49.829 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:52:52.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:52:52.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:52:52.256 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:52:52.285 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-19T00:52:52.287 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi007/crash 2024-03-19T00:52:52.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash -- . 2024-03-19T00:52:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T00:52:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-03-19T00:52:52.337 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi044/crash 2024-03-19T00:52:52.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash -- . 2024-03-19T00:52:52.385 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T00:52:52.385 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-03-19T00:52:52.387 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi204/crash 2024-03-19T00:52:52.388 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash -- . 2024-03-19T00:52:52.421 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/60839b00-e587-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T00:52:52.421 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-03-19T00:52:52.422 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-19T00:52:52.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-19T00:52:52.460 INFO:teuthology.orchestra.run.smithi007.stdout:2024-03-19T00:29:53.200941+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T00:52:52.460 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-19T00:52:52.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-19T00:52:52.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-19T00:52:52.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-19T00:52:52.684 INFO:teuthology.orchestra.run.smithi007.stdout:2024-03-19T00:29:53.200941+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T00:52:52.685 INFO:tasks.cephadm:Compressing logs... 2024-03-19T00:52:52.685 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T00:52:52.727 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T00:52:52.729 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:: No such file or directory 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T00:52:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-19T00:52:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.a.log: /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-19T00:52:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-19T00:52:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-19T00:52:52.770 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T00:52:52.770 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-19T00:52:52.770 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.x.log: 94.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-19T00:52:52.770 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr: 94.7% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.j.log.gz/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.0.log: 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr: 82.7% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log: 87.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T00:52:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-19T00:52:52.772 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/cephadm.log: -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr: 90.2%/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.7.log: -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-19T00:52:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-19T00:52:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T00:52:52.774 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-19T00:52:52.774 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.i.log: /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.z.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T00:52:52.775 INFO:teuthology.orchestra.run.smithi204.stderr: 94.9%gzip -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.l.log.gz -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-19T00:52:52.776 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-03-19T00:52:52.776 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-19T00:52:52.776 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.2.log: 87.6% 92.9% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-19T00:52:52.776 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T00:52:52.776 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T00:52:52.776 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-19T00:52:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-19T00:52:52.778 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-19T00:52:52.779 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log 2024-03-19T00:52:52.779 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T00:52:52.779 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T00:52:52.779 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log: 87.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T00:52:52.779 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-19T00:52:52.780 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T00:52:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-19T00:52:52.781 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T00:52:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-19T00:52:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.7% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-19T00:52:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-19T00:52:52.794 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T00:52:52.837 INFO:teuthology.orchestra.run.smithi044.stderr: 92.9% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-19T00:52:52.837 INFO:teuthology.orchestra.run.smithi044.stderr: 94.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-19T00:52:52.860 INFO:teuthology.orchestra.run.smithi204.stderr: 93.1% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-19T00:52:52.894 INFO:teuthology.orchestra.run.smithi204.stderr: 95.0% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-19T00:52:53.045 INFO:teuthology.orchestra.run.smithi007.stderr: 89.8% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-19T00:52:53.055 INFO:teuthology.orchestra.run.smithi044.stderr: 92.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-19T00:52:53.145 INFO:teuthology.orchestra.run.smithi204.stderr: 91.9% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-19T00:52:53.174 INFO:teuthology.orchestra.run.smithi007.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-19T00:52:53.222 INFO:teuthology.orchestra.run.smithi044.stderr: 92.0% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-19T00:52:53.525 INFO:teuthology.orchestra.run.smithi007.stderr: 92.5% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-19T00:52:53.660 INFO:teuthology.orchestra.run.smithi007.stderr: 92.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-19T00:52:53.682 INFO:teuthology.orchestra.run.smithi204.stderr: 93.2% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-19T00:52:53.759 INFO:teuthology.orchestra.run.smithi007.stderr: 90.5% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-19T00:52:54.021 INFO:teuthology.orchestra.run.smithi007.stderr: 93.2% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-19T00:52:54.029 INFO:teuthology.orchestra.run.smithi044.stderr: 93.1% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-19T00:52:54.142 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-19T00:52:54.247 INFO:teuthology.orchestra.run.smithi204.stderr: 93.3% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-19T00:52:54.266 INFO:teuthology.orchestra.run.smithi204.stderr: 92.9% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-19T00:52:54.303 INFO:teuthology.orchestra.run.smithi007.stderr: 93.3% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-19T00:52:54.305 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-03-19T00:52:54.305 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m1.567s 2024-03-19T00:52:54.305 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m6.084s 2024-03-19T00:52:54.305 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.192s 2024-03-19T00:52:54.675 INFO:teuthology.orchestra.run.smithi044.stderr: 91.6% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-19T00:52:54.883 INFO:teuthology.orchestra.run.smithi044.stderr: 92.4% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-19T00:52:55.052 INFO:teuthology.orchestra.run.smithi044.stderr: 92.1% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-19T00:52:55.054 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-03-19T00:52:55.054 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m2.310s 2024-03-19T00:52:55.054 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m8.244s 2024-03-19T00:52:55.054 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m0.291s 2024-03-19T00:52:55.220 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/60839b00-e587-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-19T00:52:55.221 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-03-19T00:52:55.221 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m2.478s 2024-03-19T00:52:55.221 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m8.123s 2024-03-19T00:52:55.221 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.290s 2024-03-19T00:52:55.222 INFO:tasks.cephadm:Archiving logs... 2024-03-19T00:52:55.222 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi007/log 2024-03-19T00:52:55.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T00:52:56.062 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi044/log 2024-03-19T00:52:56.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T00:52:57.321 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi204/log 2024-03-19T00:52:57.322 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T00:52:58.488 INFO:tasks.cephadm:Removing cluster... 2024-03-19T00:52:58.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 60839b00-e587-11ee-95c9-87774f69a715 --force 2024-03-19T00:52:58.686 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:52:58.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 60839b00-e587-11ee-95c9-87774f69a715 --force 2024-03-19T00:52:59.132 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:52:59.390 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 60839b00-e587-11ee-95c9-87774f69a715 --force 2024-03-19T00:52:59.571 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 60839b00-e587-11ee-95c9-87774f69a715 2024-03-19T00:52:59.839 INFO:tasks.cephadm:Removing cephadm ... 2024-03-19T00:52:59.839 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T00:52:59.857 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T00:52:59.873 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T00:52:59.888 INFO:tasks.cephadm:Teardown complete 2024-03-19T00:52:59.889 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-19T00:52:59.901 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-19T00:52:59.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T00:52:59.903 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T00:52:59.916 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T00:53:00.054 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-19T00:53:00.067 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-19T00:53:00.107 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:53:00.110 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:53:00.114 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:53:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:53:00.127 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:53:00.128 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:53:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:53:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-19T00:53:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +1832us[+1843us] +/- 139ms 2024-03-19T00:53:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -733us[ -722us] +/- 56ms 2024-03-19T00:53:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +98us[ +109us] +/- 62ms 2024-03-19T00:53:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:53:00.131 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:53:00.131 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-03-19T00:53:00.131 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +1847us[+1854us] +/- 139ms 2024-03-19T00:53:00.131 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 60 -741us[ -735us] +/- 56ms 2024-03-19T00:53:00.132 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +93us[ +93us] +/- 62ms 2024-03-19T00:53:00.132 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:53:00.133 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:53:00.133 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-03-19T00:53:00.133 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 60 +1844us[+1844us] +/- 139ms 2024-03-19T00:53:00.134 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -745us[ -751us] +/- 56ms 2024-03-19T00:53:00.134 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +104us[ +104us] +/- 62ms 2024-03-19T00:53:00.134 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:53:00.134 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-19T00:53:00.179 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-19T00:53:00.180 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-19T00:53:00.200 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T00:53:00.295 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T00:53:00.374 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T00:53:00.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T00:53:00.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:00.481 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T00:53:00.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:00.513 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T00:53:00.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:00.543 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-19T00:53:00.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-19T00:53:00.590 INFO:teuthology.task.internal:Duration was 3499.439596 seconds 2024-03-19T00:53:00.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-19T00:53:00.624 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-19T00:53:00.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T00:53:00.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T00:53:00.629 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T00:53:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T00:53:00.672 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T00:53:00.672 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T00:53:01.149 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-19T00:53:01.150 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-03-19T00:53:01.150 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T00:53:01.171 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-03-19T00:53:01.171 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T00:53:01.193 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-03-19T00:53:01.193 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T00:53:01.213 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-19T00:53:01.213 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T00:53:01.215 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T00:53:01.237 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T00:53:01.281 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-19T00:53:01.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T00:53:01.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T00:53:01.309 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T00:53:01.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-19T00:53:01.691 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-19T00:53:01.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T00:53:01.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T00:53:01.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T00:53:01.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-19T00:53:01.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T00:53:01.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T00:53:01.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T00:53:01.810 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-03-19T00:53:01.812 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-03-19T00:53:01.838 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-03-19T00:53:01.867 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T00:53:01.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:01.881 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T00:53:01.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:01.894 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T00:53:01.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:01.921 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-19T00:53:01.932 INFO:teuthology.task.internal:Transferring archived files... 2024-03-19T00:53:01.933 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi007 2024-03-19T00:53:01.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T00:53:01.977 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi044 2024-03-19T00:53:01.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T00:53:02.015 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225/remote/smithi204 2024-03-19T00:53:02.015 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T00:53:02.054 INFO:teuthology.task.internal:Removing archive directory... 2024-03-19T00:53:02.054 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T00:53:02.056 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T00:53:02.058 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T00:53:02.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-19T00:53:02.116 INFO:teuthology.task.internal:Not uploading archives. 2024-03-19T00:53:02.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-19T00:53:02.141 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-19T00:53:02.141 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T00:53:02.143 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T00:53:02.145 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T00:53:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: 262170 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 00:53 /home/ubuntu/cephtest 2024-03-19T00:53:02.157 INFO:teuthology.orchestra.run.smithi044.stdout: 262170 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 00:53 /home/ubuntu/cephtest 2024-03-19T00:53:02.159 INFO:teuthology.orchestra.run.smithi204.stdout: 262170 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 00:53 /home/ubuntu/cephtest 2024-03-19T00:53:02.161 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-19T00:53:02.174 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-19T00:53:02.241 INFO:teuthology.nuke:Checking targets against current locks 2024-03-19T00:53:02.266 DEBUG:teuthology.nuke:shortname: smithi007 2024-03-19T00:53:02.266 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T00:53:02.288 DEBUG:teuthology.nuke:shortname: smithi044 2024-03-19T00:53:02.288 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T00:53:02.293 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 23:31:17.216453', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKT4V/jSmR2AUCPmV1XiqgbRRuNTuEtBjyRpbTG6vOINJ39jsdZ4avpnWXR0v+gT/lYh/TJ3u4Vp633n6CyKk6Q='} 2024-03-19T00:53:02.311 DEBUG:teuthology.nuke:shortname: smithi204 2024-03-19T00:53:02.311 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T00:53:02.318 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 23:31:17.219350', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQG07FLKYQrbnejlw3LW7M+r1RZzqt2qa+Kk61wdFO+IOonC412GvvNGqL+yuVjgq5TRduVwBLZR4MmRAfy0zE='} 2024-03-19T00:53:02.327 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 23:31:17.217886', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5kGPY/UwRYLKvcCPSzHA3TWRcRdxDHmrkVH05qGP5PI3OlHGm6zpz6ZMPrKWOEme1EGs/S5+3wi3x3DFKGkSk='} 2024-03-19T00:53:02.344 INFO:teuthology.orchestra.console.smithi007:Power off 2024-03-19T00:53:02.345 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T00:53:02.370 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-03-19T00:53:02.370 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:02.388 INFO:teuthology.orchestra.console.smithi204:Power off 2024-03-19T00:53:02.388 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T00:53:02.406 INFO:teuthology.orchestra.console.smithi044:Power off 2024-03-19T00:53:02.406 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T00:53:02.414 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-19T00:53:02.415 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-03-19T00:53:02.415 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:02.430 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-03-19T00:53:02.430 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:02.442 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-19T00:53:02.457 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-03-19T00:53:06.415 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:06.444 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:06.459 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:06.526 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-19T00:53:06.556 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-19T00:53:06.569 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-03-19T00:53:10.530 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:10.557 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:10.569 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:10.643 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-03-19T00:53:10.671 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-03-19T00:53:10.680 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-03-19T00:53:10.744 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-03-19T00:53:10.773 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-03-19T00:53:14.682 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T00:53:14.792 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-03-19T00:53:14.893 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-03-19T00:53:14.998 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} duration: 3499.439596414566 failure_reason: '"2024-03-19T00:29:53.200941+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-19T00:53:14.998 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-19T00:53:15.083 INFO:teuthology.run:FAIL